hadoop - sqoop import-all-tables失败

Question

我试图在hortonworks VM沙箱上使用以下命令导入所有表。

[root@sandbox ~]# sqoop import-all-tables -m 12 --connect "jdbc:mysql://sandbox.hortonworks.com:3306/retail_db" --username retail_dba -P --warehouse-dir ./HDPCD/sqoop_import --driver com.mysql.jdbc.Driver

我收到以下错误:

16/05/07 23:00:19 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM categories AS t WHERE 1=0
16/05/07 23:00:19 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.4.0.0-169/hadoop-mapreduce
error: error reading /usr/hdp/2.4.0.0-169/hadoop/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/hadoop-yarn/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/sqoop/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/hive/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/hadoop/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/hadoop-yarn/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/sqoop/lib/mysql-connector-java.jar; error in opening zip file
error: error reading /usr/hdp/2.4.0.0-169/hive/lib/mysql-connector-java.jar; error in opening zip file
Note: /tmp/sqoop-root/compile/6cc7791885bc4e096dc470d761af0400/categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/05/07 23:00:23 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/6cc7791885bc4e096dc470d761af0400/categories.jar
16/05/07 23:00:23 INFO mapreduce.ImportJobBase: Beginning import of categories
16/05/07 23:00:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM categories AS t WHERE 1=0
16/05/07 23:00:27 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
16/05/07 23:00:27 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8050
16/05/07 23:00:29 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/10.0.2.15:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
16/05/07 23:00:30 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/10.0.2.15:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
16/05/07 23:00:31 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/10.0.2.15:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
16/05/07 23:00:32 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/10.0.2.15:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
16/05/07 23:00:33 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/10.0.2.15:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)

在运行此命令之前，我已将mysql-connector jar文件升级到6.0.2

[root@sandbox ~]# find / -name "mysql*.jar" | xargs ls -ltr
lrwxrwxrwx 1 root   root        40 2016-03-14 14:17 /usr/hdp/2.4.0.0-169/sqoop/lib/mysql-connector-java.jar -> /usr/share/java/mysql-connector-java.jar
lrwxrwxrwx 1 root   root        64 2016-03-14 14:38 /var/lib/ambari-server/resources/mysql-jdbc-driver.jar -> /var/lib/ambari-server/resources/mysql-connector-java-5.1.17.jar
-rw-r--r-- 1 root   root    819803 2016-03-14 14:38 /var/lib/ambari-server/resources/mysql-connector-java-5.1.17.jar
-rw-r--r-- 1 root   root    819803 2016-03-14 14:38 /var/lib/ambari-agent/tmp/mysql-jdbc-driver.jar
-rw-r--r-- 1 root   root    819803 2016-03-14 14:38 /var/lib/ambari-agent/tmp/mysql-connector-java.jar
-rw-r--r-- 1 ranger ranger  819803 2016-03-14 14:38 /usr/hdp/2.4.0.0-169/ranger-admin/ews/lib/mysql-connector-java.jar
-rw-r--r-- 1 root   root    819803 2016-03-14 14:43 /usr/hdp/2.4.0.0-169/knox/ext/mysql-connector-java.jar
-rw-r--r-- 1 root   root   1853083 2016-03-29 18:58 /root/mysql-connector-java-6.0.2.jar
-rw-r--r-- 1 root   root    819803 2016-05-07 14:42 /usr/hdp/2.4.0.0-169/hbase/lib/mysql-connector-java.jar
-rw-r--r-- 1 root   root      3287 2016-05-07 16:20 /usr/share/java/mysql-connector-java-6.0.2.jar
lrwxrwxrwx 1 root   root        46 2016-05-07 16:22 /usr/share/java/mysql-connector-java.jar -> /usr/share/java/mysql-connector-java-6.0.2.jar
-rw-r--r-- 1 root   root      3287 2016-05-07 16:28 /usr/hdp/2.4.0.0-169/hadoop/lib/mysql-connector-java.jar
-rw-r--r-- 1 root   root      3287 2016-05-07 16:28 /usr/hdp/2.4.0.0-169/hadoop-yarn/lib/mysql-connector-java.jar
-rw-r--r-- 1 root   root      3287 2016-05-07 16:29 /usr/hdp/2.4.0.0-169/hive/lib/mysql-connector-java.jar
-r-x------ 1 yarn   hadoop    3287 2016-05-07 21:38 /hadoop/yarn/local/usercache/root/filecache/185/mysql-connector-java.jar
lrwxrwxrwx 1 yarn   hadoop      72 2016-05-07 21:38 /hadoop/yarn/local/usercache/root/appcache/application_1462630000548_0001/container_e07_1462630000548_0001_01_000001/mysql-connector-java.jar -> /hadoop/yarn/local/usercache/root/filecache/185/mysql-connector-java.jar
-rw-r--r-- 1 root   root   1853086 2016-05-07 22:46 /root/mysql-connector-java-6.0.2-bin.jar
-rw-r--r-- 1 root   root   1853086 2016-05-07 22:47 /usr/share/java/mysql-connector-java-6.0.2-bin.jar

请帮忙。

Answer 1

我发现启动Yarn后解决了相同的问题。请检查并启动YARN。

命令:

$ sudo service hadoop-yarn-resourcemanager start
$ sudo service hadoop-yarn-nodemanager start