Hadoop Hive 集成 INSERT 查询

Question

我是 hadoop 新手，我正在尝试这个教程: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration

1.使用参数成功启动hive:配置单元 --auxpath/cygdrive/c/Hadoop/hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar,/cygdrive/c/javaHBase/hbase-0.94.6/hbase-0.94.6。 jar,/cygdrive/c/Hadoop/hive-0.9.0/lib/zookeeper-3.4.3.jar,/cygdrive/c/Hadoop/hive-0.9.0/lib/guava-r09.jar -hiveconf hbase.master =本地主机:60010

2.启动hbase成功

3."CREATE TABLE hbase_table_1"成功完成

4.我用命令list和show tables验证，一切正常

这是我的问题“INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM pokes WHERE foo=98;”

我在“htp://localhost:50060/tasklog?attemptid...”中搜索时收到此错误消息

java.lang.ClassNotFoundException: org/apache/hadoop/hive/hbase/HBaseSerDe
Continuing ...
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat
Continuing ...
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat
Continuing ...
java.lang.NullPointerException
Continuing ...
java.lang.NullPointerException
 at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:280)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
    at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:78)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
...

我试图将配置单元 jars 复制到 hbase 安装，副 vesa ...注意:我使用 hive 命令添加了必要的 JARS:ADD JAR C:...\hive-hbase-handler-0.9.0.jar etc.

hbase 版本:0.94.6配置单元版本:0.9.0

任何额外的导出？或配置？我需要帮助!非常感谢!

Answer 1

尝试通过在 hive CLI 中运行以下命令将这些 jar 添加到分布式缓存中，或者将这些行包含在 $HOME/.hiverc 文件中。这应该解决 ClassNotFoundException。

ADD JAR ...../hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar;
ADD JAR ...../hbase-0.94.1/hbase-0.94.6.jar;
ADD JAR ...../hbase-0.94.1/lib/zookeeper-3.4.3.jar;
ADD JAR ...../hbase-0.94.1/lib/guava-11.0.2.jar;
ADD JAR ...../hbase-0.94.1/lib/protobuf-java-2.4.0a.jar;

在 mapred 模式(非本地)下运行配置单元查询时，这是必需的。