个人中心
gpt4 book ai didi

java - Spark 2.0.0 抛出 AlreadyExistsException(消息 :Database default already exists) when interact with Hive 1. 0.0

转载 作者：可可西里 更新时间：2023-11-01 16:42:10 25 4
gpt4 key购买 nike

我正在尝试使用 Spark Java 连接到 Hive。当我通过 Spark 在 Hive 中运行任何查询时，它会返回如下异常:

16/10/06 09:37:56 ERROR metastore.RetryingHMSHandler: AlreadyExistsException(message:Database default already exists)

我的版本是:

星火 2.0.0

hive 1.0.0

这是我的全部堆栈:

16/10/06 09:37:56 ERROR metastore.RetryingHMSHandler: AlreadyExistsException(message:Database default already exists)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:891)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
    at com.sun.proxy.$Proxy14.create_database(Unknown Source)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:644)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156)
    at com.sun.proxy.$Proxy15.createDatabase(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:306)
    at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply$mcV$sp(HiveClientImpl.scala:291)
    at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:291)
    at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:291)
    at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:262)
    at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:209)
    at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:208)
    at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:251)
    at org.apache.spark.sql.hive.client.HiveClientImpl.createDatabase(HiveClientImpl.scala:290)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply$mcV$sp(HiveExternalCatalog.scala:99)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:99)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:99)
    at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:72)
    at org.apache.spark.sql.hive.HiveExternalCatalog.createDatabase(HiveExternalCatalog.scala:98)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createDatabase(SessionCatalog.scala:147)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.<init>(SessionCatalog.scala:89)
    at org.apache.spark.sql.hive.HiveSessionCatalog.<init>(HiveSessionCatalog.scala:51)
    at org.apache.spark.sql.hive.HiveSessionState.catalog$lzycompute(HiveSessionState.scala:49)
    at org.apache.spark.sql.hive.HiveSessionState.catalog(HiveSessionState.scala:48)
    at org.apache.spark.sql.hive.HiveSessionState$$anon$1.<init>(HiveSessionState.scala:63)
    at org.apache.spark.sql.hive.HiveSessionState.analyzer$lzycompute(HiveSessionState.scala:63)
    at org.apache.spark.sql.hive.HiveSessionState.analyzer(HiveSessionState.scala:62)
    at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:49)
    at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
    at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:582)
    at in.inndata.sparkjoinsexamples.SparkJoinExample.main(SparkJoinExample.java:10)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

16/10/06 09:37:57 INFO metastore.HiveMetaStore: 0: get_database: default
16/10/06 09:37:57 INFO HiveMetaStore.audit: ugi=karuturi    ip=unknown-ip-addr  cmd=get_database: default   
16/10/06 09:37:57 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=aaa
16/10/06 09:37:57 INFO HiveMetaStore.audit: ugi=karuturi    ip=unknown-ip-addr  cmd=get_table : db=default tbl=aaa  
Exception in thread "main" org.apache.spark.sql.AnalysisException: Table or view not found: `default`.`aaa`; line 1 pos 14
    at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:71)
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:67)
    at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:126)
    at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$foreachUp$1.apply(TreeNode.scala:125)
    at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$foreachUp$1.apply(TreeNode.scala:125)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:125)
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:67)
    at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:58)
    at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:49)
    at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
    at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:582)
    at in.inndata.sparkjoinsexamples.SparkJoinExample.main(SparkJoinExample.java:10)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/10/06 09:37:57 INFO spark.SparkContext: Invoking stop() from shutdown hook
16/10/06 09:37:57 INFO server.ServerConnector: Stopped ServerConnector@3f20e4fa{HTTP/1.1}{0.0.0.0:4040}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1bdf8190{/stages/stage/kill,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4f8969b0{/api,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6fefce9e{/,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@74cec793{/static,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@f9b7332{/executors/threadDump/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@18e7143f{/executors/threadDump,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@209775a9{/executors/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5db4c359{/executors,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2c177f9e{/environment/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@33617539{/environment,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@47874b25{/storage/rdd/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@290b1b2e{/storage/rdd,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1fc0053e{/storage/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@77307458{/storage,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@389adf1d{/stages/pool/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@7bf9b098{/stages/pool,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@72e34f77{/stages/stage/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6e9319f{/stages/stage,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6fa590ba{/stages/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2416a51{/stages,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@293bb8a5{/jobs/job/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@37ebc9d8{/jobs/job,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5217f3d0{/jobs/json,null,UNAVAILABLE}
16/10/06 09:37:57 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@32232e55{/jobs,null,UNAVAILABLE}
16/10/06 09:37:57 INFO ui.SparkUI: Stopped Spark web UI at http://192.168.1.131:4040
16/10/06 09:37:57 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/10/06 09:37:57 INFO memory.MemoryStore: MemoryStore cleared
16/10/06 09:37:57 INFO storage.BlockManager: BlockManager stopped
16/10/06 09:37:57 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
16/10/06 09:37:57 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/10/06 09:37:57 INFO spark.SparkContext: Successfully stopped SparkContext
16/10/06 09:37:57 INFO util.ShutdownHookManager: Shutdown hook called
16/10/06 09:37:57 INFO util.ShutdownHookManager: Deleting directory /private/var/folders/6n/nrvn14r50tvdvcfdds6jxyx40000gn/T/spark-b0f5733d-a475-4289-956d-c2650d9792d0

这是我的 Spark 代码:

public static void main(String[] arr){
        SparkSession session = new SparkSession.Builder().appName("SparkJoinExample").master("local").enableHiveSupport().getOrCreate();
        Dataset<Row> dset = session.sql("select * from test.results");

    }

最佳答案

它将在您的 $HOME 文件夹中创建一个 Metastore，在 Metastore 中只需删除 dbex.lck

关于java - Spark 2.0.0 抛出 AlreadyExistsException(消息 :Database default already exists) when interact with Hive 1. 0.0，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/39887437/

25 4 0
文章推荐： c++ - 以编程方式复制 Windows 10 上的桌面
文章推荐： http - 循环发送多个http post请求
文章推荐： java - 如何停止 Apache HttpClient 针对单个请求三次请求凭据？
文章推荐： hadoop - 如何在我的 ZooKeeper 中获取节点名称？
可可西里
个人简介

我是一名优秀的程序员,十分优秀！

滴滴打车优惠券免费领取
滴滴打车优惠券
全站热门文章
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com