java - sqoop导入java io异常: Mkdir failed to create a file

Question

通过 sqoop java 将 MySQL 数据导入 HDFS 时出现错误。但是我授予了 hdfs 的所有权限，然后我也遇到了同样的问题。任何人都可以解决这个问题。Hadoop版本:2.6.0Sqoop:1.4.6请准确地建议我应该给本地文件系统什么权限。提前致谢。

16/12/28 13:14:54 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
16/12/28 13:14:54 INFO manager.SqlManager: Using default fetchSize of 1000
16/12/28 13:14:54 INFO tool.CodeGenTool: Beginning code generation
16/12/28 13:14:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0
16/12/28 13:14:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0
16/12/28 13:14:56 INFO orm.CompilationManager: HADOOP_HOME is /usr/local/hadoop
Note: /tmp/sqoop-hadoop/compile/779c01454836e023b4d99d8ebb4e3add/SHIPPER.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/12/28 13:14:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/779c01454836e023b4d99d8ebb4e3add/SHIPPER.jar
16/12/28 13:14:57 INFO mapreduce.ImportJobBase: Beginning import of SHIPPER
16/12/28 13:14:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/12/28 13:14:58 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/12/28 13:14:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0
16/12/28 13:14:58 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/12/28 13:14:58 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
16/12/28 13:14:58 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
16/12/28 13:14:58 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/12/28 13:14:59 INFO mapreduce.JobSubmitter: number of splits:1
16/12/28 13:14:59 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1242973550_0001
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar <- /home/hadoop/Workspaces/BigData/avro-mapred-1.7.5-hadoop2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/avro-mapred-1.7.5-hadoop2.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-hive-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-hive-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar <- /home/hadoop/Workspaces/BigData/parquet-format-2.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-format-2.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar <- /home/hadoop/Workspaces/BigData/sqoop-1.4.6-hadoop200.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/sqoop-1.4.6-hadoop200.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-common-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-common-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar <- /home/hadoop/Workspaces/BigData/sqoop-1.4.1-incubating.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/home/hadoop/.m2/repository/org/apache/sqoop/sqoop/1.4.1-incubating/sqoop-1.4.1-incubating.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar <- /home/hadoop/Workspaces/BigData/mysql-connector-java-5.1.40-bin.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/mysql-connector-java-5.1.40-bin.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar <- /home/hadoop/Workspaces/BigData/ojdbc6-11.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ojdbc6-11.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar <- /home/hadoop/Workspaces/BigData/commons-logging-1.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-logging-1.1.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-encoding-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-encoding-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar <- /home/hadoop/Workspaces/BigData/mysql-connector-java-5.1.40.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/home/hadoop/.m2/repository/mysql/mysql-connector-java/5.1.40/mysql-connector-java-5.1.40.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar <- /home/hadoop/Workspaces/BigData/opencsv-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/opencsv-2.3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar <- /home/hadoop/Workspaces/BigData/snappy-java-1.0.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/snappy-java-1.0.5.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar <- /home/hadoop/Workspaces/BigData/commons-codec-1.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-codec-1.4.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-column-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-column-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar <- /home/hadoop/Workspaces/BigData/jackson-mapper-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-mapper-asl-1.9.13.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar <- /home/hadoop/Workspaces/BigData/jackson-databind-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-databind-2.3.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-generator-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-generator-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar <- /home/hadoop/Workspaces/BigData/commons-io-2.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-io-2.4.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar <- /home/hadoop/Workspaces/BigData/commons-jexl-2.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-jexl-2.1.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar <- /home/hadoop/Workspaces/BigData/jackson-core-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-core-2.3.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-avro-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-avro-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-mapreduce-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-mapreduce-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar <- /home/hadoop/Workspaces/BigData/avro-1.7.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/avro-1.7.5.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar <- /home/hadoop/Workspaces/BigData/ant-contrib-1.0b3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ant-contrib-1.0b3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar <- /home/hadoop/Workspaces/BigData/paranamer-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/paranamer-2.3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar <- /home/hadoop/Workspaces/BigData/slf4j-api-1.6.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/slf4j-api-1.6.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-jackson-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-jackson-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar <- /home/hadoop/Workspaces/BigData/xz-1.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/xz-1.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-hadoop-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-hadoop-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar <- /home/hadoop/Workspaces/BigData/jackson-core-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-core-asl-1.9.13.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-hadoop-compatibility-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-hadoop-compatibility-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-core-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-core-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar <- /home/hadoop/Workspaces/BigData/jackson-annotations-2.3.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-annotations-2.3.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar <- /home/hadoop/Workspaces/BigData/ant-eclipse-1.0-jvm1.2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar <- /home/hadoop/Workspaces/BigData/commons-compress-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-compress-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar <- /home/hadoop/Workspaces/BigData/hsqldb-1.8.0.10.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/hsqldb-1.8.0.10.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar
16/12/28 13:15:00 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
16/12/28 13:15:00 INFO mapreduce.Job: Running job: job_local1242973550_0001
16/12/28 13:15:00 INFO mapred.LocalJobRunner: OutputCommitter set in config null
16/12/28 13:15:00 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
16/12/28 13:15:00 ERROR output.FileOutputCommitter: Mkdirs failed to create file:/shipper/_temporary/0
16/12/28 13:15:00 INFO mapred.LocalJobRunner: Waiting for map tasks
16/12/28 13:15:00 INFO mapred.LocalJobRunner: Starting task: attempt_local1242973550_0001_m_000000_0
16/12/28 13:15:00 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
16/12/28 13:15:00 INFO mapred.MapTask: Processing split: com.cloudera.sqoop.mapreduce.db.DataDrivenDBInputFormat$DataDrivenDBInputSplit@3d93720d
16/12/28 13:15:00 INFO mapred.LocalJobRunner: map task executor complete.
16/12/28 13:15:00 WARN mapred.LocalJobRunner: job_local1242973550_0001
java.lang.Exception: java.io.IOException: Mkdirs failed to create file:/shipper/_temporary/0/_temporary/attempt_local1242973550_0001_m_000000_0 (exists=false, cwd=file:/home/hadoop/Workspaces/BigData)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Mkdirs failed to create file:/shipper/_temporary/0/_temporary/attempt_local1242973550_0001_m_000000_0 (exists=false, cwd=file:/home/hadoop/Workspaces/BigData)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:442)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:428)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
    at org.apache.sqoop.mapreduce.RawKeyTextOutputFormat.getRecordWriter(RawKeyTextOutputFormat.java:98)
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:644)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
16/12/28 13:15:01 INFO mapreduce.Job: Job job_local1242973550_0001 running in uber mode : false
16/12/28 13:15:01 INFO mapreduce.Job:  map 0% reduce 0%
16/12/28 13:15:01 INFO mapreduce.Job: Job job_local1242973550_0001 failed with state FAILED due to: NA
16/12/28 13:15:01 INFO mapreduce.Job: Counters: 0
16/12/28 13:15:01 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
16/12/28 13:15:01 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 3.3552 seconds (0 bytes/sec)
16/12/28 13:15:01 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
16/12/28 13:15:01 INFO mapreduce.ImportJobBase: Retrieved 0 records.
16/12/28 13:15:01 ERROR tool.ImportTool: Error during import: Import job failed!

代码如下:

package org.sqoop.com;

import java.io.FileInputStream;
import java.util.Properties;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;

import com.cloudera.sqoop.SqoopOptions;
import com.cloudera.sqoop.SqoopOptions.FileLayout;
import com.cloudera.sqoop.tool.ImportTool;

public class MysqlExportHDFS {

    public static void importMySQLToHDFS() throws Exception {
        try {
            String driver = "com.mysql.jdbc.Driver";
            Class.forName(driver).newInstance();

            Configuration config = new Configuration();
            config.addResource(new Path("/usr/local/hadoop/etc/hadoop/conf/core-site.xml"));
            config.addResource(new Path("/usr/local/hadoop/etc/hadoop/conf/hdfs-site.xml"));

            Properties properties = new Properties();
            properties.load(new FileInputStream("/home/hadoop/Workspaces/BigData/resources/sqoopImport.properties"));

            SqoopOptions sqoopOptions = new SqoopOptions();
            sqoopOptions.setDriverClassName(driver);
            sqoopOptions.setHadoopHome("/usr/local/hadoop");
            sqoopOptions.setConnectString(properties.getProperty("db_connection_string"));
            sqoopOptions.setTableName(properties.getProperty("db_table_name"));
            sqoopOptions.setUsername(properties.getProperty("db_username"));
            sqoopOptions.setPassword(properties.getProperty("db_password"));
            sqoopOptions.setNumMappers(1);
            sqoopOptions.setTargetDir(properties.getProperty("db_output_directory"));
            sqoopOptions.setFileLayout(FileLayout.TextFile);
            new ImportTool().run(sqoopOptions);
            } catch (Exception e) {
            // TODO: handle exception
          }
       }

    public static void main(String[] args) throws Exception{

        MysqlExportHDFS mySqlImport = new MysqlExportHDFS();
        mySqlImport.importMySQLToHDFS();
    }
}

Answer 1

正在本地磁盘上创建此文件位置以解压缩您的作业 jar。它不在 HDFS 上。检查您是否有权在本地磁盘目录上执行 mkdir。您可以为用户设置本地文件系统的读取、写入、执行、删除权限，您正在使用该用户运行此作业。请访问以下链接以获得更详细的答案。

Hadoop java.io.IOException: Mkdirs failed to create /some/path

MKDirs-failed-to-create-file