hadoop - PIG 拉丁文脚本 - 使用组和 TOBAG

Question

我有一个包含以下内容的文件

输入:

TOYID;TOYSeries;ModuleID;ID;PART_NUMBER;SUPPLIER;LAND
394107;C204; 731305; 69807402;A0001532122;ABC;AT
394107;C204; 731307; 69807402;A0001532122;ABC;AT
394107;C204; 731315; 69807402;A0001532122;ABC;AT
394107;C204; 731325; 69807402;A0001532122;ABC;AT
394107;C204; 731335; 69807402;A0001532122;ABC;AT
394107;C204; 731345; 69807402;A0001532122;ABC;AT

我想要这样的输出输出:

SUPPLIER;LAND; COUNT(SUPPLIER,LAND);  TOYID         TOYSeries;   ModuleID;   ID;          PART_NUMBER
ABC;AT;             6 ;               394107          C204;       731305; 69807402;      A0001532122
ABC;AT              6 ;               394107          C204;       731307; 69807402;      A0001532122

我试过:

A = LOAD 'hdfs://localhost:8020/BigData_POC/....../TOY_Detail.txt' USING PigStorage(';') AS (TOYID:chararray,TOYSeries:chararray,ModuleID:chararray,ID:c‌​hararray,DESCRIPTION‌​:chararray,PART_NUMB‌​ER:chararray,SUPPLIE‌​R:chararray,LAND:cha‌​rarray);
B = FOREACH A GENERATE TOYID,ModuleID,DESCRIPTION,PART_NUMBER,SUPPLIER,LAND;
C = GROUP B by (SUPPLIER,LAND);
D = foreach C generate group, COUNT(B) as cnt, B.TOYID,B.ModuleID,B.PART_NUMBER; 

我得到这样的输出:

(SUPPLIER,LAND) COUNT {(TOYID) (TOYID) (TOYID)...(TOYID) (MODULEID) (MODULEID) (MODULEID)... (MODULEID)(PARTNUMBER) (PARTNUMBER)... (PARTNUMBER)}

你知道任何可用的 pig 拉丁文字吗？

Answer 1

根据您的评论，您可以试试这个作为解决方案吗？我自己还没有验证过，所以可能还需要一些调整。

D = foreach C generate group, COUNT(B) as cnt; 
E = foreach D generate group.supplier as supplier, group.land as land, cnt;
F = Join B by (supplier,land),E by (supplier,land)