java - 从MapReduce导入数据到HBase(TableOutputFormat错误)

Question

我正在尝试将 MapReduce 作业中的数据保存到 HBase 中。我们制作的脚本在旧版本的 Hadoop (CDH3u4) 上运行良好。现在我们升级到最新版本 (CDH 5.0.2)，但脚本无法运行。

当我在最新版本上运行该程序时，出现以下错误:

Exception in thread "main" java.lang.RuntimeException: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:211)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
        at com.nrholding.t0_mr.main.ELogHBaseImport.main(ELogHBaseImport.java:89)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:389)
        at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:366)
        at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:247)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:188)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
        ... 17 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:387)
        ... 22 more
Caused by: java.lang.NoClassDefFoundError: org/cloudera/htrace/Trace
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:195)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479)
        at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:801)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:633)
        ... 27 more
Caused by: java.lang.ClassNotFoundException: org.cloudera.htrace.Trace
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 33 more

看来问题出在 TableOutputFormat 中。所以我检查了一下:

• 正确的库在 libpath 中。
• Zookeeper quorum 和 zookeeper 端口在 hbase-site.xml 中设置
• 表wp_json存在于hbase

这是有问题的代码:

public static void main(String args[]) throws Exception {

    Configuration conf = new Configuration();
    conf.set("hbase.zookeeper.quorum", "zookeeper_server1,zookeeper_server2,zookeeper_server3");
    conf.set(TableOutputFormat.OUTPUT_TABLE, "wp_json");

    String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
    String input = otherArgs[0];
    Job job = Job.getInstance(conf, "ELogHBaseImport");
    // Input is just text files in HDFS
    FileInputFormat.addInputPath(job, new Path(input));
    job.setJarByClass(ELogHBaseImport.class);
    job.setMapperClass(Map.class);
    job.setNumReduceTasks(0);
    job.setOutputFormatClass(TableOutputFormat.class);
    job.waitForCompletion(true);
}

当我使用 NullOutputFormat 时，一切正常，但没有任何内容写入 hbase。

负责错误的TableOutputFormat部分在这里:

163   /**
164    * Returns the output committer.
165    *
166    * @param context  The current context.
167    * @return The committer.
168    * @throws IOException When creating the committer fails.
169    * @throws InterruptedException When the job is aborted.
170    * @see org.apache.hadoop.mapreduce.OutputFormat#getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext)
171    */
172   @Override
173   public OutputCommitter getOutputCommitter(TaskAttemptContext context)
174   throws IOException, InterruptedException {
175     return new TableOutputCommitter();
176   }
177 
178   public Configuration getConf() {
179     return conf;
180   }
181 
182   @Override
183   public void setConf(Configuration otherConf) {
184     this.conf = HBaseConfiguration.create(otherConf);
185 
186     String tableName = this.conf.get(OUTPUT_TABLE);
187     if(tableName == null || tableName.length() <= 0) {
188       throw new IllegalArgumentException("Must specify table name");
189     }
190 
191     String address = this.conf.get(QUORUM_ADDRESS);
192     int zkClientPort = this.conf.getInt(QUORUM_PORT, 0);
193     String serverClass = this.conf.get(REGION_SERVER_CLASS);
194     String serverImpl = this.conf.get(REGION_SERVER_IMPL);
195 
196     try {
197       if (address != null) {
198         ZKUtil.applyClusterKeyToConf(this.conf, address);
199       }
200       if (serverClass != null) {
201         this.conf.set(HConstants.REGION_SERVER_IMPL, serverImpl);
202       }
203       if (zkClientPort != 0) {
204         this.conf.setInt(HConstants.ZOOKEEPER_CLIENT_PORT, zkClientPort);
205       }
206       this.table = new HTable(this.conf, tableName);
207       this.table.setAutoFlush(false, true);
208       LOG.info("Created table instance for "  + tableName);
209     } catch(IOException e) {
210       LOG.error(e);
211       throw new RuntimeException(e);
212     }
213   }

Answer 1

错误实际上是由这条消息引起的:

引起:java.lang.ClassNotFoundException:org.cloudera.htrace.Trace*

您可能在类路径中缺少一个 jar。上面提到的类可能是从您的代码中间接引用的。尝试将包含此类的 jar 放在类路径中。

希望这有帮助!!!