Maven 无法解析依赖关系 spark

Question

我尝试构建一个简单的 Java 程序:JavaWordCount for spark-1.1.0。

I get this error: Building JavaWordCount 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.279 s
[INFO] Finished at: 2014-10-23T11:28:30-04:00
[INFO] Final Memory: 9M/156M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project JavaWordCount: Could not resolve dependencies for     project spark.examples:JavaWordCount:jar:1.0-SNAPSHOT: Failure to find org.apache.spark:spark-assembly_2.10:jar:1.1.0 in http://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

这是我从 pom.xml 中的依赖项

<dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
     </dependency>
    <dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-assembly_2.10</artifactId>
    <version>1.1.0</version>
   </dependency>
   <dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-examples_2.10</artifactId>
    <version>1.1.0</version>
    </dependency>
 <dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.1.0</version>
 </dependency>

它包括 Spark 组件。

如有任何想法，我们将不胜感激。

谢谢!

Answer 1

问题在于依赖:

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-assembly_2.10</artifactId>
  <version>1.1.0</version>
</dependency>

不是 jar 它是pom file only which means你不能这样定义它。您可以在错误消息中看到它:

Failure to find org.apache.spark:spark-assembly_2.10:jar:1.1.0

这表明 Maven 将尝试下载一个 jar 文件。这意味着你必须这样定义它:

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-assembly_2.10</artifactId>
  <version>1.1.0</version>
  <type>pom</type>
</dependency>

但我不确定这是否能解决所有问题。如果这是正确的路径，您应该深入查看文档。

更新:您也可以use that as BOM via:

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-assembly_2.10</artifactId>
  <version>1.1.0</version>
  <type>pom</type>
  <scope>import</scope>
</dependency>