apache-spark - SparkDeploySchedulerBackend 错误 : Application has been killed. 所有主节点都没有响应

Question

当我启动 Spark shell 时:

bin>./spark-shell

我收到以下错误:

Spark assembly has been built with Hive, including Data nucleus jars on classpath
Welcome to SPARK VERSION 1.3.0
Using Scala version 2.10.4 (Java HotSpot(TM) Server VM, Java 1.7.0_75)
Type in expressions to have them evaluated.
Type :help for more information.
15/05/10 12:12:21 ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up.
15/05/10 12:12:21 ERROR TaskSchedulerImpl: Exiting due to error from cluster scheduler: All masters are unresponsive! Giving up.

我已经通过以下链接安装了 spark:- http://www.philchen.com/2015/02/16/how-to-install-apache-spark-and-cassandra-stack-on-ubuntu

Answer 1

您应该在启动 spark-shell 时提供 Spark 集群的主 URL

至少:

bin/spark-shell --master spark://master-ip:7077

所有选项构成了一个很长的列表，您可以自己找到合适的选项:

bin/spark-shell --help