hadoop - 如何访问Pig Latin中Date的每个元素？

Question

查询:

records = LOAD 'input' using PigStorage(' ') as (id:int, name:chararray, desination:chararray, date:chararray, salary: long);

输入样例:

    (10102,neha,developer,14/02/13,32000)

    (10103,deva,admin,02/02/14,40000)

    (10102,neha,developer,01/01/14,45000)

    (10245,sasi,developer,01/01/14,20000)

    (10109,surya,manager,01/02/2014,56000)

    (10102,neha,developer,01/02/2014,45000)

    (10245,sasi,developer,02/01/2014,25000)

我想根据日期的年份(不是整个日期)过滤以上数据。

Answer 1

检查是否适合您。

records = LOAD '/home/abhijit/Downloads/movies.txt' using PigStorage(',') as (id:int, name:chararray, desination:chararray, date:chararray, salary:int);

todate_data = foreach records generate id,name,destination,date, salary,ToDate(date,'yyyy/MM/dd HH:mm:ss') as (date_time:DateTime );

todate_data = foreach records generate name,desination,ToDate(date,'dd/MM/yyyy') as (date_time:DateTime );

getyear_data = foreach todate_data generate name,desination,GetYear(date_time);

groupByYear = group getyear_data by $3;

最终输出将是:

(2013,{(neha,developer,2013)})
(2014,{(deva,admin,2014),(neha,developer,2014),(sasi,developer,2014),(surya,manager,2014),(neha,developer,2014),(sasi,developer,2014)})