hadoop - 使用 Teradata 连接器的 Teradata 到 Hive 表导入工具

Question

我正在使用 TDCH 将 TD 表导入 Hive。

使用以下命令:- hadoop jar teradata-connector-1.3.4.jar com.teradata.hadoop.tool.TeradataImportTool -url jdbc:teradata://URL -username **** -password ****** -jobtype hive -fileformat textfile - separator ","-method split.by.hash -sourcetable test -sourcefieldnames "name, id"-targettable test_td -targetfieldnames "name, id"

即使我在 hive/lib 路径中放置了正确的 jar hive-common-1.1.0.jar，我仍然遇到错误。

INFO tool.ConnectorImportTool: java.lang.NoSuchMethodError: org.apache.hive.common.util.HiveStringUtils.normalizeIdentifier(Ljava/lang/String;)Ljava/lang/String;
        at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.<init>(ObjectStore.java:2330)
        at org.apache.hadoop.hive.metastore.ObjectStore$GetDbHelper.<init>(ObjectStore.java:2466)
        at org.apache.hadoop.hive.metastore.ObjectStore$1.<init>(ObjectStore.java:555)
        at org.apache.hadoop.hive.metastore.ObjectStore.getDatabaseInternal(ObjectStore.java:555)
        at org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:539)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98)
        at com.sun.proxy.$Proxy10.getDatabase(Unknown Source)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:566)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:588)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:430)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
        at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5503)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:178)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:161)
        at com.teradata.connector.hive.utils.HiveUtils.isHiveOutputTablePartitioned(HiveUtils.java:1039)
        at com.teradata.connector.common.tool.ConnectorImportTool.processArgs(ConnectorImportTool.java:607)
        at com.teradata.connector.common.tool.ConnectorImportTool.run(ConnectorImportTool.java:57)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at com.teradata.hadoop.tool.TeradataImportTool.main(TeradataImportTool.java:24)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

15/05/14 06:16:54 INFO tool.ConnectorImportTool: job completed with exit code 10000

Answer 1

您使用的版本似乎不存在该方法。请检查此方法所在的 jar 版本。您应该具有 Hive 通用 1.1.0 API 依赖项，如果它存在，那么您的模块正在选择一个不同的版本。