hadoop - 如何将 hadoop jar 添加到类路径？

Question

我的 mac 上的 Hadoop 2.7.3 安装在:

/usr/local/Cellar/hadoop/2.7.3

我写了一个 demo 来使用 java 从 HDFS 读取文件:

import java.io.*;
import java.net.URI;
import java.net.URISyntaxException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;

public class HDFSTest{

public static void main(String[] args) throws IOException, URISyntaxException{

  String file= "hdfs://localhost:9000/hw1/customer.tbl";
  Configuration conf = new Configuration();
  FileSystem fs = FileSystem.get(URI.create(file), conf);
  Path path = new Path(file);
  FSDataInputStream in_stream = fs.open(path);
  BufferedReader in = new BufferedReader(new      
  InputStreamReader(in_stream));
  String s;
     while ((s=in.readLine())!=null) {
        System.out.println(s);
     }
    in.close();
    fs.close();
  }
}

编译java文件时报错如下:

hero:Documents yaopan$ javac HDFSTest.java 
HDFSTest.java:8: error: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configuration;
                         ^
HDFSTest.java:10: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.FSDataInputStream;
                       ^
HDFSTest.java:12: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.FSDataOutputStream;
                       ^
HDFSTest.java:14: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.FileSystem;

我知道是找不到hadoop jar，怎么配置？ ^

Answer 1

在您的安装下找到一个名为“hadoop-common-2.7.3.jar”的 jar 文件(即 /usr/local/Cellar/hadoop/2.7.3 ) 并将其设置在类路径中，或直接在命令行中与 javac 一起提供。

javac -cp "/PATH/hadoop-common-2.7.3.jar" HDFSTest.java 

(用适当的路径替换PATH)