hadoop - 资源管理器没有节点

Question

编辑:我看过YARN Resourcemanager not connecting to nodemanager该解决方案对我不起作用。我附上了与资源管理器建立连接的节点管理器日志部分:

[main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at /0.0.0.0:8031

2016-06-17 19:01:04,697 INFO  [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(429)) - Sending out 0 NM container statuses: []

2016-06-17 19:01:04,701 INFO  [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(268)) - Registering with RM using containers :[]

2016-06-17 19:01:05,815 INFO  [main] ipc.Client (Client.java:handleConnectionFailure(867)) - Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 0 time(s); retry policy is     RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

2016-06-17 19:01:06,816 INFO  [main] ipc.Client (Client.java:handleConnectionFailure(867)) - Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

出于某种原因，它说它正在连接到 0.0.0.0。当我通过 ssh 进入其中一个数据节点并 ping 资源管理器时，我得到了一个响应，因此它能够解析主机名。

这让我相信我的 yarn-site.xml 中的选项不正确，因为我的节点正在尝试连接到 0.0.0.0:8031 而不是 resource-manager:8031

---

我在 docker 上运行一个 Cloudera hadoop 集群，我遇到了 Yarn 资源管理器无法看到其他节点的问题。它们的设置方式如下:

节点 1 - 名称节点 (hadoop-hdfs-namenode)

节点 2 - 辅助名称节点 (hadoop-hdfs-secondarynamenode)

节点 3 - yarn 资源管理器 (hadoop-yarn-resourcemanager)

节点 4 - 数据节点和节点管理器(hadoop-hdfs-datanode、hadoop-yarn-nodemanager)

节点 5 - 数据节点和节点管理器(hadoop-hdfs-datanode、hadoop-yarn-nodemanager)

当我转到 namenode:50070 时，我能够看到两个节点。但是，当我转到 resource-manager:8088 时，它显示我有零个节点。我在每个节点上的 yarn-site.xml 文件如下:

<configuration>
  <property>
    <name>yarn.resourcemanager.address</name>
    <value>resource-manager:8032</value>
  </property>
  <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>resource-manager:8030</value>
  </property>
  <property>
    <description>Classpath for typical applications.</description>
    <name>yarn.application.classpath</name>
    <value>
      $HADOOP_CONF_DIR,
      $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
      $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
      $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
      $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
    </value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.local-dirs</name>
    <value>file:///data/1/yarn/local,file:///data/2/yarn/local,file:///data/3/yarn/local</value>
  </property>
  <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value>file:///data/1/yarn/logs,file:///data/2/yarn/logs,file:///data/3/yarn/logs</value>
  </property>
  <property>
    <name>yarn.log.aggregation-enable</name>
    <value>true</value>
  </property>
  <property>
    <description>Where to aggregate logs</description>
    <name>yarn.nodemanager.remote-app-log-dir</name>
    <value>hdfs://namenode:8020/var/log/hadoop-yarn/apps</value>
  </property>
  <property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>resource-manager:8088</value>
  </property>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>resource-manager:8031</value>
  </property>
  <property>
    <name>yarn.resourcemanager.admin.address</name>
    <value>resource-manager:8033</value>
  </property>
  <property>
    <description>
    Number of seconds after an application finishes before the nodemanager's
    DeletionService will delete the application's localized file directory
    and log directory.

    To diagnose Yarn application problems, set this property's value large
    enough (for example, to 600 = 10 minutes) to permit examination of these
    directories. After changing the property's value, you must restart the
    nodemanager in order for it to have an effect.

    The roots of Yarn applications' work directories is configurable with
    the yarn.nodemanager.local-dirs property (see below), and the roots
    of the Yarn applications' log directories is configurable with the
    yarn.nodemanager.log-dirs property (see also below).
    </description>
    <name>yarn.nodemanager.delete.debug-delay-sec</name>
    <value>600</value>
  </property>
</configuration>

有人知道为什么会这样吗？

感谢阅读。

Answer 1

指定:

<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>master-1</value>
</property>