gpt4 book ai didi

hadoop - Hadoop未与Resourcemanager通信

转载 作者:行者123 更新时间:2023-12-02 21:45:31 25 4
gpt4 key购买 nike

嗨,目前我正在运行hadoop 2.4.1。我使用Eclipse创建了一个简单的Java程序DefaultMapperClass.java并将其打包到ex1.jar中

当我尝试使用命令通过hadoop shell调用该程序时,

**hadoop jar /home/Maddy/ex1.jar DefaultMapperClass hdfs://localhost/users/root/input/Hadoop.txt hdfs://localhost/users/root/output**

我在hadoop shell中获得以下输出
**[root@localhost Maddy]# hadoop jar /home/Maddy/ex1.jar DefaultMapperClass hdfs://localhost/users/root/input/Hadoop.txt hdfs://localhost/users/root/output
14/09/05 19:26:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Job started: Fri Sep 05 19:26:35 CDT 2014
14/09/05 19:26:35 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8032
[root@localhost Maddy]#**

好像hadoop shell正在尝试连接到资源管理器,但未成功,但是没有错误消息

mapred-site.xml文件:
**<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>**

yarn-site.xml:
**<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>localhost:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>localhost:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>localhost:8031</value>
</property>
</configuration>**

这里缺少什么?为什么在尝试连接到资源管理器后终止执行?

最佳答案

我建议从yarnsite.xml中删除以下配置,因为它们是不必要的:

<property>
<name>yarn.resourcemanager.address</name>
<value>localhost:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>localhost:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>localhost:8031</value>
</property>

您可以访问本地主机上的资源管理器:8088

关于hadoop - Hadoop未与Resourcemanager通信,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/25695750/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com