Hadoop 客户端.RMProxy : Connecting to ResourceManager

Question

Hadoop client.RMProxy:连接到 ResourceManager

我在 Linux 上设置单节点集群:http://tecadmin.net/setup-hadoop-2-4-single-node-cluster-on-linux/

当我像下面这样运行 mapreduce 应用程序时:hadoop jar hadoop-mapreduce-examples-2.6.0.jar grep input output 'dfs[a-z.]+

I got the ff INFO:
15/02/25 23:42:54 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/02/25 23:42:56 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/02/25 23:42:59 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/02/25 23:43:02 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

日本:

5232 SecondaryNameNode
6482 RunJar
5878 NodeManager
6521 Jps
4905 NameNode
5759 ResourceManager
5023 DataNode

搭建单机集群时如何连接ResourceManager？

我尝试添加到 yarn-site.xml，但没有成功。

<property>
<name>yarn.resourcemanager.address</name>
<value>127.0.0.1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>127.0.0.1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>127.0.0.1:8031</value>
</property>

谢谢

Answer 1

请记住有关运行 Hadoop 的一个方面。给出了三种模式:单机模式、伪分布式模式和全分布式模式。

单机和伪分布式运行在同一个节点。实际上，它们只在你的机器上运行。这不需要您显示的配置:http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html

先验，这就是 yarn-site.xml 中单个节点所需的全部内容:

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>

但也可以使用进一步的配置。我的伪分布式模式的 yarn-site 是这样的:

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>localhost:8025</value>
</property>
<property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>localhost:8030</value>
</property>
<property>
    <name>yarn.resourcemanager.address</name>
    <value>localhost:8050</value>
</property>

提示:请确定您在配置文件中输入的 IP。我建议您将此 IP 添加到您的 etc/hosts 并提供主机名。因此，在配置文件中使用主机名。