hadoop - 如何使用 Pig 按键和值分组

Question

我正在使用 pig，这是我要分析的文本的一部分:

SciTePress: 32    
Springer: 10    
Springer: 13    
Springer: 14    
Springer: 1571

我想要实现的是以上升的方式对文本进行排序。例如，我希望输出看起来像这样:

Springer: 1608  //( i.e. the sum of 10+13+14+1571)
SciTePress: 32

有没有办法使用 pig 来实现这一点？

这是我现在得到的输出:

Springer: 1571
SciTePress: 32  
Springer: 14  
Springer: 13    
Springer: 10  

这些是我用过的命令:

    WORDS = LOAD '../filename' using PigStorage(':') AS (title: chararray, count:int);
    grpd = GROUP WORDS BY count;
    sorted = order WORDS by count desc;
    top5 = limit sorted 5;
    dump top5;

Answer 1

我们必须根据标题对数据进行分组，对于每个组，我们可以调用 SUM 函数来获取总和。

输入:

SciTePress: 32    
Springer: 10    
Springer: 13    
Springer: 14    
Springer: 1571

pig 脚本:

words = LOAD '/Users/muralirao/learning/pig/a.csv'  USING PigStorage(':') AS (title: chararray, title_count:int);
grp_by_title = GROUP  words BY title;
req_data = FOREACH grp_by_title GENERATE group AS title, SUM(words.title_count) AS total_count;
req_data_ordered = ORDER req_data BY total_count;

输出:DUMP req_data_ordered

(SciTePress,32)
(Springer,1608)