hadoop - Endless INFO Client:Spark上的application_xx的应用程序报告(状态:ACCEPTED)消息提交

Question

当我在集群模式下使用Hadoop与Yarn提交Spark应用程序时。
yarn 客户端状态卡在“接受”状态，并且永远不会更改为“正在运行”。我正在使用具有1个主控2个从属的Centos 7 Hadoop集群

我使用 float IP(我们在外部关联)登录到openstack，该IP与我们在系统中执行ifconfig所获得的IP地址不同。

以下是日志:

    18/01/21 16:34:19 INFO yarn.Client: Uploading resource file:/usr/local/spark/examples/jars/spark-examples_2.11-2.0.1.jar -> hdfs://192.168.198.10:8020/user/cloud-user/.sparkStaging/application_1516548465362_0014/spark-examples_2.11-2.0.1.jar
    18/01/21 16:34:19 INFO yarn.Client: Uploading resource file:/tmp/spark-f37b5cec-a81f-46c3-9b5e-6ce7854c6dd4/__spark_conf__2008488553335511154.zip -> hdfs://192.168.198.10:8020/user/cloud-user/.sparkStaging/application_1516548465362_0014/__spark_conf__.zip
    18/01/21 16:34:19 INFO spark.SecurityManager: Changing view acls to: cloud-user
    18/01/21 16:34:19 INFO spark.SecurityManager: Changing modify acls to: cloud-user
    18/01/21 16:34:19 INFO spark.SecurityManager: Changing view acls groups to: 
    18/01/21 16:34:19 INFO spark.SecurityManager: Changing modify acls groups to: 
    18/01/21 16:34:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(cloud-user); groups with view permissions: Set(); users  with modify permissions: Set(cloud-user); groups with modify permissions: Set()
    18/01/21 16:34:19 INFO yarn.Client: Submitting application application_1516548465362_0014 to ResourceManager
    18/01/21 16:34:19 INFO impl.YarnClientImpl: Submitted application application_1516548465362_0014
    18/01/21 16:34:20 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:20 INFO yarn.Client: 
                client token: N/A
                diagnostics: N/A
                ApplicationMaster host: N/A
                ApplicationMaster RPC port: -1
                queue: default
                start time: 1516552459599
                tracking URL: http://master.abc.com:8088/proxy/application_1516548465362_0014/
                user: cloud-user

    18/01/21 16:34:21 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:22 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:23 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:24 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:25 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:26 INFO yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)
    18/01/21 16:34:27 yarn.Client: Application report for application_1516548465362_0014 (state: ACCEPTED)

尝试了人们提出的所有选项，但没有任何效果。我看到节点有足够的空间，但不确定为什么它不起作用。任何帮助表示赞赏。谢谢

Answer 1

Unresolved datanode registration: hostname cannot be resolved (ip=192.168.198.11, hostname=192.168.198.11)

我不认为主机名应该是IP，但是您需要在每台计算机上更新 /etc/hosts来告诉它从机和主服务器在哪里，或者您需要静态IP或使用DNS服务器来解析 float 在其上的计算机的地址。网络