gpt4 book ai didi

hadoop - 运行mapreduce作业根本没有输出。它甚至没有运行。很奇怪。终端上没有错误

转载 作者:行者123 更新时间:2023-12-02 21:19:57 25 4
gpt4 key购买 nike

我编译了mapreduce代码(驱动程序,mapper和reducer类)并创建了Jar文件。当我在数据集上运行它时,它似乎没有运行。它只是返回到如图所示的提示。有什么建议吗?

非常感谢
basam

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;

//This driver program will bring all the information needed to submit this Map reduce job.

public class MultiLangDictionary {

public static void main(String[] args) throws Exception{

if (args.length !=2){

System.err.println("Usage: MultiLangDictionary <input path> <output path>");

System.exit(-1);

}



Configuration conf = new Configuration();



Job ajob = new Job(conf, "MultiLangDictionary");

//Assigning the driver class name
ajob.setJarByClass(MultiLangDictionary.class);


FileInputFormat.addInputPath(ajob, new Path(args[0]));

//first argument is the job itself
//second argument is the location of the output dataset
FileOutputFormat.setOutputPath(ajob, new Path(args[1]));


ajob.setInputFormatClass(TextInputFormat.class);



ajob.setOutputFormatClass(TextOutputFormat.class);


//Defining the mapper class name
ajob.setMapperClass(MultiLangDictionaryMapper.class);

//Defining the Reducer class name
ajob.setReducerClass(MultiLangDictionaryReducer.class);

//setting the second argument as a path in a path variable
Path outputPath = new Path(args[1]);

//deleting the output path automatically from hdfs so that we don't have delete it explicitly
outputPath.getFileSystem(conf).delete(outputPath);



}

}

最佳答案

尝试在命令中使用java packagename.classname

hadoop jar MultiLangDictionary.jar [yourpackagename].MultiLangDictionary input output

关于hadoop - 运行mapreduce作业根本没有输出。它甚至没有运行。很奇怪。终端上没有错误,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37685880/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com