hive - 当我有\N 作为数据时，sqoop-export 失败

Question

我在运行我的 sqoop 导出命令时遇到错误。

这是我要通过 sqoop 命令导出的内容

00001|Content|1|Content-article|\N|2015-02-1815:16:04|2015-02-1815:16:04|1      |\N|\N|\N|\N|\N|\N|\N|\N|\N00002|Content|1|Content-article|\N|2015-02-1815:16:04|2015-02-1815:16:04|1      |\N|\N|\N|\N|\N|\N|\N|\N|\N

sqoop command

sqoop export --connect jdbc:postgresql://10.11.12.13:1234/db --table table1 --username user1 --password pass1--export-dir /hivetables/table/   --fields-terminated-by '|' --lines-terminated-by '\n'  -- --schema schema

15/06/09 08:05:16 INFO mapreduce.Job: Task Id : attempt_1431442954745_1210_m_000001_0, Status : FAILED Error: java.io.IOException: Can't export data, please check failed map task logs at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.lang.RuntimeException: Can't parse input data: '\N' at duser.__loadFromFields(duser.java:690) at duser.parse(duser.java:558) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83) ... 10 more Caused by: java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff] at java.sql.Timestamp.valueOf(Timestamp.java:202) at duser.__loadFromFields(duser.java:627)

你能帮我解决吗？

Answer 1

尝试将这些参数添加到导出语句中

--input-null-string "\\\\N" --input-null-non-string "\\\\N"

来自文档:

If --input-null-string is not specified, then the string "null" will be interpreted as null for string-type columns. If --input-null-non-string is not specified, then both the string "null" and the empty string will be interpreted as null for non-string columns.

如果不添加这些参数，它将无法理解您数据中的 \N 实际上是 null。