elasticsearch - 如何将Elasticsearch-Hadoop依赖项导入Spark脚本

Question

我试图通过将Build.scala文档设置为使用spark-es connector:

libraryDependencies ++= Seq(
    "com.datastax.spark" %% "spark-cassandra-connector" % "1.2.1",
    "org.elasticsearch" %% "elasticsearch-hadoop" % "2.2.0"
  )

但是我得到了错误:

[error] (*:update) sbt.ResolveException: unresolved dependency: org.elasticsearch#elasticsearch-hadoop_2.10;2.2.0: not found

我可以看到它存在 here ...

编辑:

当我将Build.scala更改为:

"org.elasticsearch" % "elasticsearch-hadoop" % "2.2.0"

我收到以下错误:

[error] impossible to get artifacts when data has not been loaded. IvyNode = org.scala-lang#scala-library;2.10.3
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.scala-lang#scala-library;2.10.3

怎么了？

Answer 1

elasticsearch-hadoop不是Scala依赖项，因此它没有Scala特定版本，因此不能与%%一起使用。尝试

"org.elasticsearch" % "elasticsearch-hadoop" % "2.2.0"