apache-spark - Spark-Shell 启动错误

Question

我在启动时看到错误 spark-shell , 使用 spark-1.6.0-bin-hadoop2.6 .这是刚刚出现的新行为。

下面的日志消息中显示的失败结果是 sqlContext 不可用(但 sc 可用)。

是否存在某种可以释放的 Derby 锁？ Another instance of Derby may have already booted the database /root/spark-1.6.0-bin-hadoop2.6/bin/metastore_db.

<console>:16: error: not found: value sqlContext
         import sqlContext.implicits._
                ^
<console>:16: error: not found: value sqlContext
         import sqlContext.sql

16/05/25 11:00:00 ERROR Schema: Failed initialising database.
Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@c2191a8, see the next exception for details.
org.datanucleus.exceptions.NucleusDataStoreException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@c2191a8, see the next exception for details.


16/05/25 11:06:02 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient


16/05/25 11:06:02 ERROR Schema: Failed initialising database.
Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@372e972d, see the next exception for details.
org.datanucleus.exceptions.NucleusDataStoreException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@372e972d, see the next exception for details.

Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@c2191a8, see the next exception for details.
        at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
        ... 134 more
Caused by: java.sql.SQLException: Another instance of Derby may have already booted the database /root/spark-1.6.0-bin-hadoop2.6/bin/metastore_db.
        at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
        ... 131 more
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /root/spark-1.6.0-bin-hadoop2.6/bin/metastore_db.

Answer 1

在尝试创建 DataFrame 时，我在 Spark 2.0.0 shell 中遇到了类似的问题。 ，只需删除 metastore_db/dbex.lck问题已解决。