java - 从 Eclipse IDE 在 YARN 上提交 Spark 应用程序

Question

当我尝试通过 Eclipse 在 Yarn 上提交我的 Spark 应用程序时，我遇到了一个问题。我尝试提交一个简单的 SVM 程序，但出现以下错误。我有macbook，如果有人能给我详细的答案，我将不胜感激

16/09/17 10:04:19 ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalStateException: Library directory '.../MyProject/assembly/target/scala-2.11/jars' does not exist; make sure Spark is built.
    at org.apache.spark.launcher.CommandBuilderUtils.checkState(CommandBuilderUtils.java:248)
    at org.apache.spark.launcher.CommandBuilderUtils.findJarsDir(CommandBuilderUtils.java:368)
    at org.apache.spark.launcher.YarnCommandBuilderUtils$.findJarsDir(YarnCommandBuilderUtils.scala:38)
    at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:500)
    at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:834)
    at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:167)
    at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
    at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:149)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:500)
    at SVM.main(SVM.java:21)

Answer 1

转到

Run Configurations --> Environment

在 Eclipse 中添加环境变量 SPARK_HOME。