maven - 哪个 jar 包含 org.apache.spark.sql.api.java.JavaSQLContext

Question

pom 中包含以下依赖项:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.10</artifactId>
    <version>1.3.0</version>
</dependency>

我希望 jar 包含以下类:

org.apache.spark.sql.api.java.JavaSQLContext

虽然它包含包 org.apache.spark.sql.api.java，但该包似乎包含名为 UDF1- UDSF22 的接口(interface).

获取 JavaSQLContext 的正确依赖项是什么？

谢谢。

Answer 1

从版本 1.3.0 开始，JavaSQLContext 类已被删除。您应该使用 org.apache.spark.sql.SQLContext 类来代替。 documentation声明如下:

Prior to Spark 1.3 there were separate Java compatible classes (JavaSQLContext and JavaSchemaRDD) that mirrored the Scala API. In Spark 1.3 the Java API and Scala API have been unified. Users of either language should use SQLContext and DataFrame. In general theses classes try to use types that are usable from both languages (i.e. Array instead of language specific collections). In some cases where no common type exists (e.g., for passing in closures or Maps) function overloading is used instead.

Additionally the Java specific types API has been removed. Users of both Scala and Java should use the classes present in org.apache.spark.sql.types to describe schema programmatically.

顺便说一句，如果您想搜索哪些 jar 包含特定类，您可以使用 Advanced Search Maven Central 并搜索“按类名”。所以这里是对 JavaSQLContext 的搜索: http://search.maven.org/#search|ga|1|fc%3A%22org.apache.spark.sql.api.java.JavaSQLContext%22