hadoop - 问题:将表从hadoop导出到mysql

Question

这是我的sqoop脚本，用于将表从hadoop导出到mysql:

export
## Database details
--connect
jdbc:mysql://mktgcituspoc1.cisco.com:3306/poc
--username
pocuser
--password
pocuser
## Table to export to
--table
mktg_site_pub
--export-dir
##/app/MarketingIT/warehouse/mktg_mbd.db/performance_tst
/app/dev/MarketingIt/warehouse/hddvmktg/mktg_mbd.db/mktg_site_pub
--input-fields-terminated-by
'|'
--input-null-string
'\\N'
--input-null-non-string
'\\N'
-m

运行上面的sqoop脚本后出现以下错误。

15/02/19 02:16:38 INFO mapred.JobClient: Task Id : attempt_201502172305_2648_m_000016_1, Status : FAILED on node hdnprd-c01-r01-05
java.io.IOException: Can't export data, please check task tracker logs
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:680)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:282)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1117)
        at org.apache.hadoop.mapred.Child.main(Child.java:271)
Caused by: java.lang.IllegalArgumentException
        at java.sql.Date.valueOf(Date.java:140)
        at mktg_site_pub.__loadFromFields(mktg_site_pub.java:3622)
        at mktg_site_pub.parse(mktg_site_pub.java:3549)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
        ... 10 more

任何人都可以帮助找出确切的问题吗？

Answer 1

极有可能在MySQL中有一个与日期相关的字段，您试图将非日期值导出到该字段中。

检查MySQL表中的哪些字段具有该类型，然后检查Hadoop数据集中的相应值是否可以使用 Date 转换为Java Date.valueOf()。