gpt4 book ai didi

hadoop - 在多节点集群上运行mapreduce wordcount(Java代码)

转载 作者:行者123 更新时间:2023-12-02 21:23:15 26 4
gpt4 key购买 nike

我试图使用eclipse在hadoop多节点群集上运行wordcount java程序(该程序在单节点群集上工作正常,但在多节点上无法运行)。我正在返回关注信息

INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:54310. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 16/04/24 21:30:46 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:54310. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)


public static void main(String[] args) throws Exception 
{
Configuration conf = new Configuration();

Job job = new Job(conf, "wordcount");

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);

job.setMapperClass(Map.class);
job.setReducerClass(Reduce.class);

job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);

FileInputFormat.addInputPath(job, new Path("hdfs://localhost:54310/user/hduser/sam/"));
FileOutputFormat.setOutputPath(job, new Path("hdfs://localhost:54310/user/hduser/wc-output"));

job.waitForCompletion(true);

}

}
*

我认为路径有问题。
我在主端运行此代码

最佳答案

执行命令

hdfs dfs -ls hdfs://localhost:54310/user/hduser/sam/

工作?

关于hadoop - 在多节点集群上运行mapreduce wordcount(Java代码),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/36822166/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com