postgresql - 如何使用Sqoop将Postgres表(几何格式)导入Hive？

Question

我有具有几何类型数据的Postgres表。现在我正尝试使用Sqoop将Postgres表导入Hive。然后出现此错误，

sqoop import --connect jdbc:postgresql://192.168.1.130:5432/Analyst_Bangalore --username postgres --password track@123 --table state -m 1 --warehouse-dir /user/hive/warehouse/analytics.db

 ERROR orm.ClassWriter (ClassWriter.java:generateFields(560)) - Cannot resolve SQL type 1111
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateEquals(645)) - Cannot resolve SQL type 1111
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateEquals(645)) - Cannot resolve SQL type 1111
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbRead(718)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbRead(718)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateLoadLargeObjects(798)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateLoadLargeObjects(798)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,115 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbWrite(884)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,116 ERROR orm.ClassWriter (ClassWriter.java:myGenerateDbWrite(884)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,116 ERROR orm.ClassWriter (ClassWriter.java:myGenerateHadoopRead(955)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,116 ERROR orm.ClassWriter (ClassWriter.java:myGenerateHadoopWrite(1595)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR orm.ClassWriter (ClassWriter.java:myGenerateHadoopWrite(1595)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR orm.ClassWriter (ClassWriter.java:myGenerateToString(1314)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR orm.ClassWriter (ClassWriter.java:myGenerateToString(1314)) - No Java type for SQL type 1111 for column the_geom
2016-10-20 15:52:30,117 ERROR sqoop.Sqoop (Sqoop.java:runSqoop(181)) - Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.sqoop.orm.ClassWriter.parseNullVal(ClassWriter.java:1377)
        at org.apache.sqoop.orm.ClassWriter.parseColumn(ClassWriter.java:1402)
        at org.apache.sqoop.orm.ClassWriter.myGenerateParser(ClassWriter.java:1528)

任何人都不知道如何将几何数据导入到 hive 中，请给我建议。任何输入表示赞赏。

Answer 1

Sqoop默认不支持Geometry数据类型。您需要为Java(对于POJO类)手动添加它的映射。

在您的代码中添加此标签:

--map-column-java the_geom=String

最终命令:

sqoop import \
 --connect jdbc:postgresql://192.168.1.130:5432/Analyst_Bangalore \
 --username postgres \
 --password track@123 \
 --map-column-java the_geom=String \
 --table state \
 -m 1 \
 --warehouse-dir /user/hive/warehouse/analytics.db

如果使用 配置单元，请添加--map-column-hive the_geom=string导入。