hbase - 在 Cloudera 4.7.0 集群上的 hbase 类路径中找不到 Jar

Question

我正在升级运行 hbase 版本 0.94.15-cdh-4.7.0 的 CDH 4.7.0 集群上使用的 Apache Phoenix 版本。我的目标是将我们从 Phoenix 3.1.0 版迁移到 3.3.1，但是当我升级到 3.2.2 版时，我的 join 语句开始失败。

Phoenix Upgrade Documentation充其量是简洁的，但我的理解是使用 Cloudera Manager 推出带有升级后的 jar 的新包裹，然后使用升级的客户端连接到集群应该是从一个版本升级到另一个版本所必需的。

升级后，我所有的测试都通过了保存连接测试。一个简单的连接语句:

SELECT A."id", A."group", B."group", B."name" 
  FROM "Test_Table_1" AS A 
  INNER JOIN "Test_Table_2" AS B 
  ON A."group" = B."group" 
  LIMIT 10

导致以下异常(帖子末尾的完整堆栈跟踪):

Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=14, exceptions:
Mon Aug 24 09:21:18 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException

在 3.2 版本中，phoenix 将其 snappy 版本升级为使用 org.iq80.snappy 版本 0.3，但 jar tf phoenix-core-3.2.2.jar不会显示任何活泼的类。这让我认为类路径中缺少活泼的 jar，尽管它是 phoenix 3.2 pom 中的一个依赖项。这就是问题开始的地方:我可以将 snappy-0.3.jar 添加到我的 hbase 集群中，但无法弄清楚如何修改类路径以包含 jar。

我在 Cloudera 的 HBase 服务环境高级配置(安全阀)中包含了以下内容:
HBASE_CLASSPATH="/opt/cloudera/parcels/phoenix_core/lib/java/*:$HBASE_CLASSPATH"
请注意，我的 Phoenix jar 在 /opt/cloudera/parcels/phoenix_core/lib/java 中目录，所以我知道这个目录在 hbase 类路径中，即使我输入 hbase classpath 时它没有出现。进入其中一个数据节点上的终端。

所以在这个背景下， 我如何验证 snappy-0.3.jar 在类路径中，更广泛地说，我如何让我的 join 语句正常工作？

加入时的完整堆栈跟踪:

java.sql.SQLException: Encountered exception in sub plan [0] execution.
    at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:151)
    at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:230)
    at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:221)
    at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:221)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:1032)
    at MainTest.execute(MainTest.java:87)
    at MainTest.join1(MainTest.java:297)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.sql.SQLException: java.util.concurrent.ExecutionException: java.lang.reflect.UndeclaredThrowableException
    at org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:202)
    at org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:83)
    at org.apache.phoenix.execute.HashJoinPlan$HashSubPlan.execute(HashJoinPlan.java:333)
    at org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:130)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.UndeclaredThrowableException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:202)
    at org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:194)
    ... 7 more
Caused by: java.lang.reflect.UndeclaredThrowableException
    at com.sun.proxy.$Proxy20.addServerCache(Unknown Source)
    at org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:172)
    at org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:167)
    ... 4 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=14, exceptions:
Mon Aug 24 09:21:18 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:20 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:22 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:24 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:27 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:31 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:36 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:44 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:22:01 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:22:34 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:23:38 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:24:43 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:25:48 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:26:52 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException

    at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:187)
    at org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
    ... 7 more

Answer 1

我想出了一个 hacky 解决方法来防止错误。我解压了 snappy-0.3.jar 文件，然后将生成的类文件添加到 phoenix-core-3.3.1.jar

jar xf snappy-0.3.jar

jar uf phoenix-core-3.3.1.jar org/iq80/snappy/*

修改后的 Phoenix 核心 jar 然后被插入一个新的包裹中，现在加入工作!

如果有人有更好的答案而不涉及“滚动我自己的”可以这么说，我仍然很乐意听到它。