java - NoClassDefFound : Scala/xml/metadata

Question

我正在用 Java(IDE:Eclipse Luna、Maven)运行一个简单的 spark 程序。

我的示例程序是

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;

public class testSpark {
    public static void main(String[] args) {
        SparkConf conf = new SparkConf().setAppName("Testing").setMaster("local");
        JavaSparkContext sc = new JavaSparkContext(conf);
        
        System.out.println(sc.appName());
    }
}

但是我在运行示例程序时遇到错误

Exception in thread "main" java.lang.NoClassDefFoundError:
scala/xml/MetaData  at
  org.apache.spark.ui.jobs.JobsTab.<init>(JobsTab.scala:30)     at
  org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:50)  at
  org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:61)  at
  iScope.testSpark.main(testSpark.java:9) 

Caused by:
  java.lang.ClassNotFoundException: scala.xml.MetaData  at
  java.net.URLClassLoader.findClass(Unknown Source)     at
  java.lang.ClassLoader.loadClass(Unknown Source)

我的pom.xml文件是

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.11</artifactId>
    <version>1.2.1</version>
</dependency>
<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>18.0</version>
    </dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-mllib_2.11</artifactId>
    <version>1.2.1</version>
</dependency>

Answer 1

SparkUI 似乎使用了一个 scala 包，请尝试将此 dsependency 添加到您的 pom 文件以将 scala.xml 包放在您的类路径中。

<dependency>
    <groupId>org.scala-lang</groupId>
    <artifactId>scala-xml</artifactId>
    <version>2.11.0-M4</version>
</dependency>