java - Apache Spark maven tomcat :run

Question

我想从具有多个 maven 模块的 spring 项目运行 Apache Spark。Spring 项目在没有 apache spark 依赖项的情况下正常运行，但是当我添加 spark 依赖项时，我看到以下内容:

java.lang.ClassCastException: org.springframework.web.servlet.DispatcherServlet cannot be cast to javax.servlet.Servlet

Spark 依赖:

 <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_2.10</artifactId>
      <version>1.2.0</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
    <groupId>org.apache.spark</groupId>
       <artifactId>spark-mllib_2.10</artifactId>
       <version>1.2.0</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-core</artifactId>
      <version>0.20.2</version>
      <scope>provided</scope>
    </dependency>
   <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-client</artifactId>
      <version>2.5.2</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-assembly-jar</artifactId>
      <version>1.3.0</version>
    </dependency>

Answer 1

这是对我有用的。原因是 spark 引擎对特定版本的 servlet api 和实现具有传递依赖性。在类路径中，它首先发现不一样的地方并提示它。

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>${spark.version}</version>
    <exclusions>
        <exclusion>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.glassfish</groupId>
            <artifactId>javax.servlet</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.eclipse.jetty.orbit</groupId>
            <artifactId>javax.servlet</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-mllib_2.10</artifactId>
    <version>${spark.version}</version>
</dependency>

此外，为了将 Web 服务用作驱动程序，我认为需要使用 spark 库的编译范围。