hadoop - Pig - 分组后 MAX 不工作

Question

我正在使用 Pig 0.12.1 和 Map-R。在对其他字段的关系进行分组后，我试图找到一个字段的 ma​​x 。在评论中引用以下 pig 脚本和关系结构-

r1 = foreach SomeRelation generate flatten(group) as (c1 , c2);
-- r1: {c1: biginteger,c2: biginteger}
r2 = group r1 by c1;
-- r2: {group: chararray,r1: {(c1: chararray,c2: biginteger)}}

DUMP r2;
/* output -
1234|{(1234,9876)}
2345|{(2345,8765)}
3456|{(3456,7654)}
4567|{(4567,6543)}
*/

r3 = foreach r2 generate group as c1, MAX(r1.c2) as c2;

我收到以下错误

Could not infer the matching function for org.apache.pig.builtin.MAX as multiple or none of them fit. Please use an explicit cast.

脚本解释-

I am flattening group of SomeRelation into c1, c2 and then regrouping on c1 to generate max of c2 with each c1 group.

请提出建议。

Answer 1

我不确定你是否可以在 flatten 下使用 group 关键字。此外，您是否考虑过在展平该组之前对其进行标记化。看这个例子:

       load_data = LOAD '/PIG_TESTS_ALL/WordCount' as (line); 
       tokenizing_data = FOREACH load_data generate flatten(TOKENIZE(line)) as word; 
       group_data = GROUP tokenizing_data by word; 
       Result = FOREACH group_data generate group,COUNT(tokenizing_data); 
       dump Result;

这实际上是为了字数统计，你可以在此基础上根据你想做的事情找到最大值。