hadoop - 收到错误消息:- map 中的键键入不匹配:预期的org.apache.hadoop.io.Text，收到的org.apache.hadoop.io.LongWritable

Question

我已经写了一个mapreduce作业来进行日志文件分析，我的映射器将文本输出为键和值，并且在驱动程序类中明确设置了映射输出类。

但是我仍然收到错误:- map 中的键类型不匹配:预期的org.apache.hadoop.io.Text，收到的org.apache.hadoop.io.LongWritable

public class CompositeUserMapper extends Mapper<LongWritable, Text, Text, Text> {

IntWritable a = new IntWritable(1);
//Text txt = new Text();

@Override
protected void map(LongWritable key, Text value,
        Context context)
        throws IOException, InterruptedException {
    String line = value.toString();

    Pattern p = Pattern.compile("\bd{8}\b");
    Matcher m = p.matcher(line);
    String userId = "";
    String CompositeId = "";
    if(m.find()){

        userId = m.group(1);
    }

     CompositeId = line.substring(line.indexOf("compositeId :")+13).trim();

     context.write(new Text(CompositeId),new Text(userId));


    // TODO Auto-generated method stub
    super.map(key, value, context);
}    

我的驱动程序类如下:

public class CompositeUserDriver extends Configured implements Tool {

public static void main(String[] args) throws Exception {

    CompositeUserDriver wd = new CompositeUserDriver();
    int res = ToolRunner.run(wd, args);
    System.exit(res);

}

public int run(String[] arg0) throws Exception {
    // TODO Auto-generated method stub

    Job job=new Job();
    job.setJarByClass(CompositeUserDriver.class);
    job.setJobName("Composite UserId Count" );

    FileInputFormat.addInputPath(job, new Path(arg0[0]));
    FileOutputFormat.setOutputPath(job, new Path(arg0[1]));
    job.setMapperClass(CompositeUserMapper.class);
    job.setReducerClass(CompositeUserReducer.class);
    job.setMapOutputKeyClass(Text.class);
    job.setMapOutputValueClass(Text.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    return job.waitForCompletion(true) ? 0 : 1;
    //return 0;
}

}

请告知如何解决此问题。

Answer 1

从您的映射器代码中删除super.map(key, value, context);行:它调用父类的map方法，这是标识映射器，它返回键和传递给它的值，在这种情况下，键是从文件开头的字节偏移量