scala - Apache Spark 错误: not found: value sqlContext

Question

我正在尝试在 Windows 10 中设置 Spark。最初，我遇到 this error启动时，链接中的解决方案有所帮助。现在我仍然无法运行 import sqlContext.sql 因为它仍然抛出错误

----------------------------------------------------------------
Fri Mar 24 12:07:05 IST 2017:
Booting Derby version The Apache Software Foundation - Apache Derby - 10.12.1.1 - (1704137): instance a816c00e-015a-ff08-6530-00000ac1cba8
on database directory C:\metastore_db with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@37606fee
Loaded from file:/F:/Soft/spark/spark-2.1.0-bin-hadoop2.7/bin/../jars/derby-10.12.1.1.jar
java.vendor=Oracle Corporation
java.runtime.version=1.8.0_101-b13
user.dir=C:\
os.name=Windows 10
os.arch=amd64
os.version=10.0
derby.system.home=null
Database Class Loader started - derby.database.classpath=''
17/03/24 12:07:09 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Spark context Web UI available at http://10.128.18.22:4040
Spark context available as 'sc' (master = local[*], app id = local-1490337421381).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.0
      /_/

Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101)
Type in expressions to have them evaluated.
Type :help for more information.

scala> import sqlContext.sql
<console>:23: error: not found: value sqlContext
       import sqlContext.sql
              ^

Answer 1

Spark context available as 'sc' (master = local[*], app id = local-1490337421381).

Spark session available as 'spark'.

在Spark 2.0.x中，Spark的入口点是SparkSession这在 Spark shell 中可以作为 spark 使用，所以请尝试以下方式:

spark.sqlContext.sql(...)

您还可以像这样创建 Spark 上下文

val sqlContext = new org.apache.spark.sql.SQLContext(sc)

第一个选项是我的选择，因为 Spark shell 已经为您创建了一个选项，所以请使用它。