gpt4 book ai didi

java - Driver 中的 job.setOutputKeyClass 和 setOutputValueClass 与 reducer 的 context.write 方法不匹配,程序仍然运行正常。怎么办?

转载 作者:可可西里 更新时间:2023-11-01 14:29:00 25 4
gpt4 key购买 nike

驱动代码:

public class WcDriver {

public static void main(String[] args) throws IOException,
InterruptedException, ClassNotFoundException {
Configuration conf = new Configuration();

Job job = new Job(conf, "WcDriver");
job.setJarByClass(WcDriver.class);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(LongWritable.class);

job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);

FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.setMapperClass(WcMapper.class);
job.setReducerClass(WcReducer.class);

job.waitForCompletion(true);
}
}

reducer 代码

public class WcReducer extends Reducer<Text, LongWritable, Text,String>
{
@Override
public void reduce(Text key, Iterable<LongWritable> values, Context context) throws IOException, InterruptedException {
String key1 = null;
int total = 0;
for (LongWritable value : values) {
total += value.get();
key1= key.toString();
}
context.write(new Text(key1), "ABC");
}
}

在这里,在驱动程序类中我设置了 job.setOutputKeyClass(Text.class)job.setOutputValueClass(LongWritable.class),但在 reducer 类中我正在编写字符串 context.write(new Text(key1), "ABC");。我认为运行程序时应该会出现错误,因为输出类型不匹配,而且 reducer 的键应该实现 WritableComparable 并且值应该实现 Writable 接口(interface)。奇怪的是,这个程序运行良好。我不明白为什么没有异常(exception)。

最佳答案

尝试这样做:

 //  job.setOutputFormatClass(TextOutputFormat.class); 
// comment this line, and you'll sure get exception of casting.

这是因为,TextOutputFormat 假定 LongWritable 为键,Text 为值,如果您不定义 outPutFormat 类,它将期望获得可写的默认行为,这是默认情况,但如果您提及它, 它会将其隐式转换为给定类型。;

关于java - Driver 中的 job.setOutputKeyClass 和 setOutputValueClass 与 reducer 的 context.write 方法不匹配,程序仍然运行正常。怎么办?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/21968435/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com