gpt4 book ai didi

java - Reducer 代码是代码没有执行?

转载 作者:可可西里 更新时间:2023-11-01 16:24:32 26 4
gpt4 key购买 nike

在我的驱动程序类中,我正在运行两个作业,我的第一个作业按预期工作,但在我的第二个作业中,reducer 类没有执行。
下面是我的驱动程序类(JOb2 配置):

if(job.waitForCompletion(true)){
Configuration conf2 = new Configuration();
Job job2 = Job.getInstance(conf2);

MultipleInputs.addInputPath(job2, inOutPath, TextInputFormat.class, CombinedUserRatingMapper.class);
MultipleInputs.addInputPath(job2, new Path(oArgs[3]), TextInputFormat.class, MovieMapper.class);
FileOutputFormat.setOutputPath(job2, new Path(oArgs[4]));
job2.setReducerClass(GenreCombinedReducer.class);
job2.setJarByClass(GroupedAnalysisDriver.class);
job2.setNumReduceTasks(1);
job2.setMapOutputKeyClass(IntWritable.class);
job2.setMapOutputValueClass(Text.class);
job2.setOutputKeyClass(Text.class);
job2.setOutputValueClass(Text.class);

System.out.println(job2.waitForCompletion(true)?0:1);
}

下面是我的两个映射器:
映射器 1:

public class CombinedUserRatingMapper extends Mapper<LongWritable, Text, IntWritable, Text> {
//age occ mid rating
private IntWritable mid = new IntWritable();
private Text val = new Text();

public void map(LongWritable key,Text value,Context context) throws IOException, InterruptedException{
String[] token = value.toString().split("\t");
int i= Integer.parseInt(token[2]);
mid.set(i);
val.set("C"+"\t"+token[0]+"\t"+token[1]+"\t"+token[3]);
context.write(mid, val);
} // mid age occ rating

}

第二个映射器:

public class MovieMapper extends Mapper<LongWritable, Text, IntWritable, Text> {


public void map(LongWritable key, Text value,Context context) throws IOException, InterruptedException {
// movies.dat - 1::Toy Story (1995)::Animation|Children's|Comedy
// id name genre
String[] token = value.toString().split("::");
int id = Integer.parseInt(token[0]); //getting ID and name
String name = token[1];
String genre = token[2];
context.write(new IntWritable(id), new Text("G\t"+genre));
//mid G genre
}
}

最后是 Reduce 类:

public class GenreCombinedReducer extends Reducer<IntWritable, Text, Text, Text>{
// mid G genre
// mid C age occ rating
private Text outvalue=new Text();

public void Reduce(IntWritable key, Iterable<Text> values,Context context) throws IOException, InterruptedException{
String genre = "";
String combineInfo = "" ;
for(Text val : values){
if(val.charAt(0)=='G'){
genre = val.toString().split("\t")[1];

}else if(val.charAt(0)=='C'){
String[] CI = val.toString().split("\t");
combineInfo = CI[1]+CI[2]+CI[3];

}
}
context.write(new Text(combineInfo),new Text(genre));
}


}

在输出文件中,我得到映射器类的输出。我也试过调试,但光标从未出现在 Reducer 类中。

最佳答案

在您的 Reducer 类中,您应该覆盖“reduce”方法而不是“Reduce”。 (Java 区分大小写)

关于java - Reducer 代码是代码没有执行?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/48650465/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com