个人中心
gpt4 book ai didi

hadoop - 无法在配置单元中创建外部表以指向 hbase 一个

转载 作者：可可西里 更新时间：2023-11-01 14:54:21 26 4
gpt4 key购买 nike

我是一名学生，试图了解所有 hadoop 的工作原理。所以，我在 15 台机器上运行 cloudera。配置没问题，所有服务都是绿色的。我在 hbase 下导入了一个 mysql 12k 行，一切都很顺利。我想对这些数据进行查询，但我知道我不能使用 hbase。这就是为什么我想使用以下代码创建外部 View :

CREATE EXTERNAL TABLE ViewSimulation2 (
  id int,
  eol int,
  sensor int,
  value1 float,
  value2 float,
  value3 float,
  value4 float,
  value5 float,
  value6 float)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  "hbase.columns.mapping" =
  ":key,data:eol,data:sensor,data:value1,data:value2,data:value3,data:value4,data:value5,data:value6"
)
TBLPROPERTIES("hbase.table.name" = "Simulation");

当我在控制台中运行它时，它卡住了，我必须按 ctrl-c 来取消它。在色调中，我所拥有的只是这些消息一次又一次地循环:

13/12/04 07:33:48 INFO ql.Driver: </PERFLOG method=TimeToSubmit start=1386171228376 end=1386171228483 duration=107>
13/12/04 07:33:48 INFO exec.DDLTask: Use StorageHandler-supplied org.apache.hadoop.hive.hbase.HBaseSerDe for table ViewSimulation2
13/12/04 07:33:48 INFO hive.metastore: Trying to connect to metastore with URI thrift://insset-5.cloudera.insset.fr:9083
13/12/04 07:33:48 INFO hive.metastore: Waiting 1 seconds before next connection attempt.
13/12/04 07:33:49 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
13/12/04 07:33:49 WARN conf.Configuration: dfs.https.address is deprecated. Instead, use dfs.namenode.https-address
13/12/04 07:33:49 WARN conf.Configuration: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
13/12/04 07:33:49 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/12/04 07:33:49 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 16770@insset-8.cloudera.insset.fr
13/12/04 07:33:49 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:33:49 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
13/12/04 07:33:51 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:33:51 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
13/12/04 07:33:56 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:33:56 INFO util.RetryCounter: Sleeping 8000ms before retry #3...
13/12/04 07:34:05 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:34:05 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
13/12/04 07:34:05 WARN zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:684)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:428)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
    at $Proxy13.createTable(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3719)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:344)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:609)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:598)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1388)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:598)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
13/12/04 07:34:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:684)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:428)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
    at $Proxy13.createTable(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3719)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:344)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:609)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:598)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1388)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:598)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

在那之后，我还有其他类似这样的错误:

13/12/04 07:49:53 INFO client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found
13/12/04 07:49:53 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 8 of 10 failed; retrying after sleep of 16069
org.apache.hadoop.hbase.MasterNotRunningException
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:706)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:428)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
    at $Proxy13.createTable(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3719)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute

这很奇怪，因为正如我所说，一切都运行良好，所以作为 hbase master...这会是动物园管理员配置错误吗？我进行了很多次搜索，但没有找到任何对我有帮助的东西。

最佳答案

ZooKeeper 似乎没有正确设置。也许 dataDir 不存在:http://archive.cloudera.com/cdh4/cdh/4/zookeeper/zookeeperAdmin.html#sc_zkMulitServerSetup

关于hadoop - 无法在配置单元中创建外部表以指向 hbase 一个，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/20379867/

26 4 0
文章推荐： c++ - move 操作的条件编译
文章推荐： html - 购物车增长到覆盖它下面的 div
文章推荐： c++ - 何时将 C++ 中的函数标记为虚函数？
文章推荐： Hadoop MapReduce : MapTasks vs. 映射器
可可西里
个人简介

我是一名优秀的程序员,十分优秀！

滴滴打车优惠券免费领取
滴滴打车优惠券
全站热门文章
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com