hadoop - 如何加入 pig 袋

Question

首先，我有两个数据文件。

largefile.txt:

1001    {(1,-1),(2,-1),(3,-1),(4,-1)}

smallfile.txt:

1002    {(1,0.04),(2,0.02),(4,0.03)}

我想要这样的smallfile.txt:

1002    {(1,0.04),(2,0.02),(3,-1),(4,0.03)}

我可以执行哪种类型的联接？

A = LOAD './largefile.txt' USING PigStorage('\t') AS (id:int, a:bag{tuple(time:int,value:float)});

B = LOAD './smallfile.txt' USING PigStorage('\t') AS (id:int, b:bag{tuple(time:int,value:float)});

Answer 1

您能稍微清除一下要求吗？是否要以相同的值(例如1002)从largefile.txt和smallfile.txt加入第一列/字段。如果是这样，您可以简单地做到这一点:

A =使用PigStorage('\ t')AS(id:int，a:bag {tuple(time:int，value:float)})来加载'./largefile.txt'。

A = Foreach A生成id，FLATTEN(a)作为time，value;

B =使用PigStorage('\ t')AS(id:int，b:bag {tuple(time:int，value:float)})来加载'./smallfile.txt'。

B = Foreach B生成id，FLATTEN(b)作为time，value;

join =通过A.id加入A，通过B.id加入B；