java - IncompleteClassChangeError : Class hbase. client.ConnectionManager$HConnectionImplementation 未实现接口(interface) hbase.client.Connection

Question

我得到以下异常:

java.io.IOException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
    at com.itworx.bigdata.mediasensor.cascade.dao.impl.ImpalaAuthorDao.queryUserFollowers(ImpalaAuthorDao.java:107)
    at com.itworx.bigdata.mediasensor.cascade.service.impl.AuthorServiceImpl.getNodesByHashtag(AuthorServiceImpl.java:46)
    at com.itworx.bigdata.mediasensor.cascade.service.test.AuthorServiceTest.testJob(AuthorServiceTest.java:26)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.springframework.test.context.junit4.SpringTestMethod.invoke(SpringTestMethod.java:198)
    at org.springframework.test.context.junit4.SpringMethodRoadie.runTestMethod(SpringMethodRoadie.java:274)
    at org.springframework.test.context.junit4.SpringMethodRoadie$2.run(SpringMethodRoadie.java:207)
    at org.springframework.test.context.junit4.SpringMethodRoadie.runBeforesThenTestThenAfters(SpringMethodRoadie.java:254)
    at org.springframework.test.context.junit4.SpringMethodRoadie.runWithRepetitions(SpringMethodRoadie.java:234)
    at org.springframework.test.context.junit4.SpringMethodRoadie.runTest(SpringMethodRoadie.java:204)
    at org.springframework.test.context.junit4.SpringMethodRoadie.run(SpringMethodRoadie.java:146)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.invokeTestMethod(SpringJUnit4ClassRunner.java:151)
    at org.junit.internal.runners.JUnit4ClassRunner.runMethods(JUnit4ClassRunner.java:51)
    at org.junit.internal.runners.JUnit4ClassRunner$1.run(JUnit4ClassRunner.java:44)
    at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:27)
    at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:37)
    at org.junit.internal.runners.JUnit4ClassRunner.run(JUnit4ClassRunner.java:42)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
    ... 28 more
Caused by: java.lang.IncompatibleClassChangeError: Class org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation does not implement the requested interface org.apache.hadoop.hbase.client.Connection
    at org.apache.hadoop.hbase.client.RegistryFactory.getRegistry(RegistryFactory.java:35)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.setupRegistry(ConnectionManager.java:882)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:647)
    ... 33 more

当我尝试在我的机器上运行测试用例时，需要连接到服务器才能连接到 hbase。我的代码是这样的:

Configuration conf = HBaseConfiguration.create();
        conf.set(HBASE_ZOOKEEPER_QUORUM, HBASE_ZOOKEEPER_QUORUM_VAL);
        Connection connection = ConnectionFactory.createConnection(conf);
        Table table = connection.getTable(TableName.valueOf("schemaname.tablename"));
        FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
        Scan s = new Scan();

        for(String user:users){
            PrefixFilter filter=new PrefixFilter(Bytes.toBytes(user));
            filterList.addFilter(filter);
        }
        s.setFilter(filterList);
        ResultScanner scanner = table.getScanner(s);
        for (Result r = scanner.next(); r != null; r = scanner.next()) {
            for (KeyValue kv : r.raw()) {
                System.out.println(kv.getValue()+" "+kv.getKey());
            }
        }
        scanner.close();

我不明白为什么会导致这个异常，任何人都可以帮我解决这个问题

更新以下是使用的依赖项:

<repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
        </repository>
    </repositories>
    <dependencies>
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>1.2.1</version>
            <exclusions>
                <exclusion>
                    <groupId>jdk.tools</groupId>
                    <artifactId>jdk.tools</artifactId>

                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.velocity</groupId>
            <artifactId>velocity-tools</artifactId>
            <version>2.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>2.5</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>19.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase -->
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase</artifactId>
            <version>0.90.3</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-examples</artifactId>
            <version>1.0.0-cdh5.4.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-prefix-tree</artifactId>
            <version>1.0.0-cdh5.4.2</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-client -->
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>1.0.0-cdh5.4.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.htrace</groupId>
            <artifactId>htrace-core</artifactId>
            <version>3.1.0-incubating</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-server</artifactId>
            <version>1.0.0-cdh5.4.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-annotations</artifactId>
            <version>1.0.0-cdh5.4.2</version>
        </dependency>
    </dependencies>

Answer 1

这是由于以下原因之一造成的。

1. 类路径中存在同一 jar 的多个版本。检查容器日志。它会列出类路径。该类路径显示运行时加载 jar 的目录。敏锐地观察每一个目录。至少其中一个目录应该有多个版本的 hbase jar。保留特定于您的 hadoop 提供商(HDP、CDH、Mapr 等)的一个。删除其余所有 jar。

2. 依赖项中列出了多个版本的 hbase jar。

   <dependency>
       <groupId>org.apache.hbase</groupId>
       <artifactId>hbase</artifactId>
       <version>0.90.3</version>
   </dependency>
   <dependency>
       <groupId>org.apache.hbase</groupId>
       <artifactId>hbase-client</artifactId>
       <version>1.0.0-cdh5.4.2</version>
   </dependency>
   

hbase 0.90.3没有ConnectionFactory、Connection、Table类和接口(interface)。请查看http://www.java2s.com/Code/Jar/h/Downloadhbase0903jar.htm 。 hbase-client 1.0.0-cdh5.4.2 拥有所有这些。我从https://javalibs.com/artifact/org.apache.hbase/hbase-client手动下载了jar并使用java反编译器进行验证。

这在编译时不会成为问题。但在运行时，提交 yarn 作业后，它会加载 hbase 0.90.3 jar，并且无法找到必要的类和接口(interface)。请从依赖项中删除此条目。这应该可以解决问题。