hadoop - Hive无法启动:创建路径/hive/cluster/delegation/METASTORE/keys时出错

Question

我在无法启动配置单元的kerberized群集上遇到了一个问题。

症状:

服务成功启动(并且没有停止)

在Ambari中出现了一条警报，其中提到Hive Metastore失败

在命令行上启动配置单元失败(只是一直挂着)

通过beeline，我可以看到元数据，但无法获取实际数据

我在/var/log/hive/hivemetastore.log中发现以下错误

2016-08-29 10:12:49,047 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(5934)) - Metastore Thrift Server threw an exception...
org.apache.hadoop.hive.thrift.DelegationTokenStore$TokenStoreException: Error creating path /hive/cluster/delegation/METASTORE/keys
    at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:166)
    at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.initClientAndPaths(ZooKeeperTokenStore.java:236)
    at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:469)
    at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server.startDelegationTokenSecretManager(HadoopThriftAuthBridge.java:444)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6015)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:5930)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /hive/cluster/delegation/METASTORE/keys
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:123)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
    at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:691)
    at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:675)
    at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
    at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:672)
    at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:453)
    at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:443)
    at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:423)
    at org.apache.curator.framework.imps.CreateBuilderImpl$3.forPath(CreateBuilderImpl.java:257)
    at org.apache.curator.framework.imps.CreateBuilderImpl$3.forPath(CreateBuilderImpl.java:205)
    at org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:160)
    ... 11 more

Answer 1

请注意，我实际上已经尝试了几件事，所以我不确定这是否是完整的解决方案，但这是最后一步，我认为这是关键的一步:

经过长时间的搜索，我间接找到了此站点:https://community.hortonworks.com/articles/49040/hive-metastore-crashes-on-nullpointerexception-wit.html

以下是有助于我解决问题的相关片段:

This is a known issue being tracked in the following Hortonworks bug:

https://hortonworks.jira.com/browse/BUG-42602

WORKAROUND:

Set the hive.cluster.delegation.token.store.class to the following:

hive.cluster.delegation.token.store.class=org.apache.hadoop.hive.thrift.DBTokenStore

If using Ambari, this setting can be changed by clicking on the Hive service on the Ambari Dashboard, navigating to the "Configs" tab, and modifying the parameter in the "Advanced Hive-site" section of the Hive configs. Save the changes and restart Hive from the Ambari User Interface when prompted.

If not using ambari, this setting can be located in the /etc/hive/conf/hive-site.xml file. Make sure this change is made on all applicable nodes on the cluster. Once the changes are made, the Hive services must be restarted.