hadoop - 使用Pig Latin进行列到行的转换

Question

A = load 'input.txt'; 
dump A;
"0,1, 2,3,4 
5, 6,7, 8,9
B = foreach A generate FLATTEN(TOBAG(*));
dump B
("0)
(1)
( 2)
(3)
(4)
(5)
( 6)
(7)
( 8)
(9)

我想对上面的每个字段执行一些替换和修剪操作。我如何将其转换回原始格式？

预期产量

0,1,2,3,4

5,6,7,8,9

Answer 1

是的，这确实是一个实验性问题。

行到列的转换和列到行的转换!

是的，通过从RANK运营商那里获得很少的帮助，我认为我们可以实现这一目标

我为下面的输入尝试了下面的代码

输入:

 0,1,2,3,4 
 5,6,7,8,9

在 pig 以下脚本中，有两个转储语句

numbers = LOAD '/home/inputfiles/col_to_row.txt' USING PigStorage() As(line:chararray);

numbers_rank = RANK numbers;

numbers_each = FOREACH numbers_rank GENERATE  $0 as rank_key,FLATTEN(TOKENIZE(line)) as each_number;

rows_to_columns = FOREACH numbers_each GENERATE each_number;

dump rows_to_columns;--Will give you each number in a separate row..


numbers_grp = GROUP numbers_each BY rank_key;

columns_to_rows = FOREACH numbers_grp GENERATE FLATTEN(BagToTuple(numbers_each.each_number));

dump columns_to_rows; -- Will give you as Per original input data set

输出:

   dump rows_to_columns;

         (0)
         (1)
         (2)
         (3)
         (4)
         (5)
         (6)
         (7)
         (8)
         (9)


   dump columns_to_rows;

         (0,1,2,3,4)
         (5,6,7,8,9)