java - 使用 MapReduce Java 解析文件

Question

我正在尝试使用 Hadoop MapReduce 解析一个 json 文件，但在编译时遇到了我在以前的 MapReduce 项目中没有遇到的奇怪错误。

Mapper.java:43: error: type Mapper does not take parameters
public static class Map extends Mapper<LongWritable, Text, Text, Text> {
                                      ^
Mapper.java:45: error: cannot find symbol
    public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
                                                  ^
  symbol:   class Context
  location: class Map
Mapper.java:35: error: incompatible types: Class<Map> cannot be converted to Class<? extends Mapper>
    job.setMapperClass(Map.class);

我的代码如下:

import java.io.IOException;
import java.util.*;

import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.mapreduce.lib.input.*;
import org.apache.hadoop.mapreduce.lib.output.*;
import org.apache.hadoop.util.*;

public class Mapper extends Configured implements Tool {

public static void main(String args[]) throws Exception {
    int res = ToolRunner.run(new Mapper(), args);
    System.exit(res);
}

public int run(String[] args) throws Exception {
    Path inputPath = new Path(args[0]);
    Path outputPath = new Path(args[1]);

    Configuration conf = getConf();
    Job job = new Job(conf, this.getClass().toString());

    FileInputFormat.setInputPaths(job, inputPath);
    FileOutputFormat.setOutputPath(job, outputPath);

    job.setJobName("Mapper");
    job.setJarByClass(Mapper.class);

    job.setMapperClass(Map.class);

    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);

    return job.waitForCompletion(true) ? 0 : 1;
}

public static class Map extends Mapper<LongWritable, Text, Text, Text> {

    public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {

        String line = value.toString();
        Link link = Link.parse(line);
        context.write(new Text(link.url().toString()), new Text(link.tags().toString()));
    }
}
}

如果能帮助我确定为什么会收到这些错误，我们将不胜感激。谢谢。

Answer 1

您的主类称为 Mapper，它又具有扩展 Mapper 的嵌套类。将您的主类重命名为其他名称。还要将 Map 更改为其他内容，否则它可能会与 java.util.Map 混淆。