java - map 缩小 : Unabale to run the code due to number of errors

Question

请看下面的代码

Map.java

public class Map extends Mapper<longwritable, intwritable="" text,=""> {
 private final static IntWritable one = new IntWritable(1);
 private Text word = new Text();

 @Override
 public void map(LongWritable key, Text value, Context context)
   throws IOException, InterruptedException {
  String line = value.toString();
  StringTokenizer tokenizer = new StringTokenizer(line);
  while (tokenizer.hasMoreTokens()) {
   word.set(tokenizer.nextToken());
   context.write(word, one);
  }
 }
}
</longwritable,>

Reduce.java

public class Reduce extends Reducer<text, intwritable,="" intwritable="" text,=""> {
 @Override
 protected void reduce(
   Text key,
   java.lang.Iterable<intwritable> values,
   org.apache.hadoop.mapreduce.Reducer<text, intwritable,="" intwritable="" text,="">.Context context)
   throws IOException, InterruptedException {
  int sum = 0;
  for (IntWritable value : values) {
   sum += value.get();
  }
  context.write(key, new IntWritable(sum));
 }
}
</text,></intwritable></text,>

WordCount.java

public class WordCount {

    public static void main(String[] args) throws Exception {
          Configuration conf = new Configuration();

          Job job = new Job(conf, "wordcount");
          job.setJarByClass(WordCount.class);

          job.setOutputKeyClass(Text.class);
          job.setOutputValueClass(IntWritable.class);

          job.setMapperClass(Map.class);
          job.setReducerClass(Reduce.class);

          job.setInputFormatClass(TextInputFormat.class);
          job.setOutputFormatClass(TextOutputFormat.class);

          FileInputFormat.addInputPath(job, new Path(args[0]));
          FileOutputFormat.setOutputPath(job, new Path(args[1]));

          job.waitForCompletion(true);
        }

}

整个代码摘自this Map Reduce 教程 ( http://cloud.dzone.com/articles/how-run-elastic-mapreduce-job )

.一旦我将这些类复制到 Eclipse 中，它就会显示很多错误，例如 Cannot be Resolved By Type。这是合理的，因为这段代码用作实例的类在默认的 JDK 中找不到，而且教程也没有给出任何下载任何库的说明。我忽略了它，认为它与服务器端的 Elastic Map Reduce 有关。

当我将其上传到 Amazon Elastic Map Reduce、创建作业流并运行程序后，它立即出现以下错误。

Exception in thread "main" java.lang.Error: Unresolved compilation problems: 
    Configuration cannot be resolved to a type
    Configuration cannot be resolved to a type
    Job cannot be resolved to a type
    Job cannot be resolved to a type
    Text cannot be resolved to a type
    IntWritable cannot be resolved to a type
    TextInputFormat cannot be resolved to a type
    TextOutputFormat cannot be resolved to a type
    FileInputFormat cannot be resolved
    Path cannot be resolved to a type
    FileOutputFormat cannot be resolved
    Path cannot be resolved to a type

    at WordCount.main(WordCount.java:5)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:187)

我怎样才能使这段代码工作？我是否必须为此下载任何库？如何让这段代码运行并查看结果？这是我第一次体验 Amazon 和 Elastic Map reduce，是的，也是第一次体验大数据。

请帮忙。

Answer 1

所以，你的意思是，你没有在你的项目中添加任何hadoop jar，你忽略了编译错误，并希望它可以在安装了hadoop-client的服务器端运行？

如果是真的，那是不可能的。

你必须在你的项目中添加hadoop-client.XX.jar，任何版本都可以。