java - Pig 脚本在 0.12.0 上工作但不在 0.11.1 上工作

Question

我写了这个在 0.12.0 版本上完美运行的 pig 脚本，但我不能让它在 0.11.1 上运行我无法确定真正缺少的是什么。

data = LOAD '<file_name>' USING PigStorage(',') AS 
(Year,Month:int,DayofMonth,DayOfWeek,DepTime,CRSDepTime,ArrTime,CRSArrTime,UniqueCarrier,
    FlightNum,TailNum,ActualElapsedTime,CRSElapsedTime,AirTime,ArrDelay:int,DepDelay,Origin,
    Dest,Distance,TaxiIn,TaxiOut,Cancelled,CancellationCode,Diverted,CarrierDelay,
    WeatherDelay,NASDelay,SecurityDelay,LateAircraftDelay);
A = FILTER data BY (ArrDelay > 0);
X = GROUP A BY (Dest, Year, 
        (
            Case
                when Month>2 AND Month<6 THEN 'SPRING'
                when Month>5 AND Month<9 THEN 'SUMMER'
                when Month>8 AND Month<12 THEN 'FALL'
                when Month==12 OR (Month<3 AND Month>0) THEN 'WINTER'
            END
        )
    );
Y = FOREACH X GENERATE group.Dest, group.Year, group.$2, SUM(A.ArrDelay);
STORE Y INTO 'myoutput';

这是运行脚本时给出的异常。

Pig Stack Trace
---------------
ERROR 1200: <file DelayBySeasonPerYear.pig, line 7, column 16>  Syntax error, unexpected symbol at or near 'Dest'

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file DelayBySeasonPerYear.pig, line 7, column 16>  Syntax error, unexpected symbol at or near 'Dest'
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:991)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:604)
    at org.apache.pig.Main.main(Main.java:157)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Caused by: Failed to parse: <file DelayBySeasonPerYear.pig, line 7, column 16>  Syntax error, unexpected symbol at or near 'Dest'
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:235)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:177)
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
    ... 14 more

Answer 1

CASE 最近才在 0.12 版中实现。它在 0.11 中不可用。

参见:

• https://issues.apache.org/jira/browse/PIG-3268 (通知修复版本 0.12)
• http://hortonworks.com/blog/announcing-apache-pig-0-12/ (关于 0.12 中新特性的博文，其中之一是 CASE)

---

建议的解决方法:

写一个UDF它接受一个 Month 值，然后返回一个包含写出季节的 chararray。然后，在 FILTER 之后的 FOREACH 语句中使用此 UDF。

...
A = FILTER data BY (ArrDelay > 0);
A = FOREACH A GENERATE MySeasonUDF(Month) as Season, Dest, Year, ArrDelay;
X = GROUP A BY (Dest, Year, Season);
...