hadoop - 使用HQL将数据从一个表转移到另一个表

Question

我目前有一个看起来像这样的表:

表格1

ID|NAME|INFO_CODE_1|INFO_CODE_2|INFO_CODE_3|INFO_CODE_4|INFO_TEXT|DESCRIPTION
1 |Test|123        |254        |556        |867        |Test Text|Test Description

我正在尝试将Table_1中的数据插入Table_2中，以使第一个表中的每个记录变成第二个表中的四个单独的记录，如下所示:

表_2

ID|NAME|INFO_CODE|INFO_TEXT|DESCRIPTION
1 |Test|123      |Test Text|Test Description
1 |Test|254      |Test Text|Test Description
1 |Test|556      |Test Text|Test Description
1 |Test|867      |Test Text|Test Description

Table_1中有成千上万条记录，还有其他列我们不关心的数据，INFO_CODE_1至INFO_CODE_4中的值因记录而异(甚至可能为null)。我正在考虑写这样的东西:

INSERT OVERWRITE TABLE example.TABLE_2
SELECT
A.ID AS ID,
A.NAME AS NAME,
A.INFO_CODE_1 AS INFO_CODE,
A.INFO_TEXT AS INFO_TEXT,
A.DESCRIPTION AS DESCRIPTION
FROM
example.TABLE_1 A
UNION ALL
SELECT
A.ID AS ID,
A.NAME AS NAME,
A.INFO_CODE_2 AS INFO_CODE,
A.INFO_TEXT AS INFO_TEXT,
A.DESCRIPTION AS DESCRIPTION
FROM
example.TABLE_1 A
UNION ALL
SELECT
A.ID AS ID,
A.NAME AS NAME,
A.INFO_CODE_3 AS INFO_CODE,
A.INFO_TEXT AS INFO_TEXT,
A.DESCRIPTION AS DESCRIPTION
FROM
example.TABLE_1 A
UNION ALL
SELECT
A.ID AS ID,
A.NAME AS NAME,
A.INFO_CODE_4 AS INFO_CODE,
A.INFO_TEXT AS INFO_TEXT,
A.DESCRIPTION AS DESCRIPTION
FROM
example.TABLE_1 A;

这行得通吗？

如果可以，是否有更好的方法编写
声明？

是否可以确保将具有相同ID的所有记录一个接一个插入？

预先感谢大家提供的任何帮助。

Answer 1

没有太多工会的另一种方式是。这将需要1个mapreduce作业，而不是几个节省资源的作业。您可以合并代码，将其拆分以生成一个数组并分解该数组。

INSERT OVERWRITE TABLE example.TABLE_2
SELECT
a.id,
a.name,
CODE
a.INFO_TEXT,
A.DESCRIPTION from 
(
SELECT
A.ID AS ID,
A.NAME AS NAME,
split(concat(A.INFO_CODE_1, ",",A.INFO_CODE_2,",",A.INFO_CODE_3,",",A.INFO_CODE_4),",") AS INFO_CODE_ARRAY,
A.INFO_TEXT AS INFO_TEXT,
A.DESCRIPTION AS DESCRIPTION
from TABLE_1
) t LATERAL VIEW explode(INFO_CODE_ARRAY) codea AS code;