hadoop - 输出状态为 : ACCEPTED in YARN MODE 的 Spark Hang

Question

我的 Spark 环境

Spark  -> 2.1.0
Hadoop -> 2.8.1
Eclipse -> Neon 2

我在yarn模式下获取spark上下文时卡住了，如何在yarn模式下获取spark上下文，请帮我解决。

我的 Hadoop、Yarn 和 Spark 安装成功。

$ jps
3200 NameNode
5264 ExecutorLauncher
5328 CoarseGrainedExecutorBackend
3555 SecondaryNameNode
5316 CoarseGrainedExecutorBackend
7590 Jps
3357 DataNode
4045 NodeManager
5118 SparkSubmit
3727 ResourceManager

我在 yarn 模式下获取 spark 上下文的源代码:-

public class JavaClient {

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        SparkConf conf = new SparkConf().setAppName("SparkTest").setMaster("yarn-client");
        SparkSession spark = SparkSession.builder().config(conf).getOrCreate();


        System.out.println(spark.version() + " : " + spark.sparkContext());

    }
}

输出:-

17/09/22 10:24:11 INFO Client: Application report for application_1506052073594_0011 (state: ACCEPTED)

17/09/22 10:24:12 INFO Client: Application report for application_1506052073594_0011 (state: ACCEPTED) 

17/09/22 10:24:13 INFO Client: Application report for application_1506052073594_0011 (state: ACCEPTED)

.
.
.

依此类推，不返回 spark session 。

Answer 1

在 Resource Manager UI (localhost:8088) 中检查您是否有用于 Application Master 和执行器的可用内存/CPU 插槽