java - hadoop编译——在dfs文件中

Question

我想用下面的命令编译hadoop的例子，但是出现了错误:

$ mkdir wordcount_classes
$ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d wordcount_classes WordCount.java
$ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ .

Assuming that:

    /usr/joe/wordcount/input - input directory in HDFS
    /usr/joe/wordcount/output - output directory in HDFS

Sample text-files as input:

$ bin/hadoop dfs -ls /usr/joe/wordcount/input/
/usr/joe/wordcount/input/file01
/usr/joe/wordcount/input/file02

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file01
Hello World Bye World

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file02
Hello Hadoop Goodbye Hadoop 

Answer 1

/usr/joe 是本地的，正如您在第一行执行 ls 命令时所见。第二个命令需要在 HDFS 位置进行输入和输出，但 /usr/joe 在 HDFS 上不存在。您需要将数据移动到 HDFS 上，然后执行命令。例如:

#This creates a folder "wordcount/input" in your HDFS home directory
hdfs dfs -mkdir -p wordcount/input
hdfs dfs -put /usr/joe/wordcount/input/* wordcount/input