hadoop - 执行使用UNION和RANK的PIG脚本。 (错误2017:内部错误创建作业配置。)

Question

我执行如上所示的简单PIG脚本，该脚本使用RANK运算符和UNION。它引发:错误2017:内部错误创建作业配置。

A = LOAD '/location/to/my/file';
SPLIT A INTO 
B IF $0 IS NULL,
C IF $0 IS NOT NULL;

B2 = RANK B;
C2 = RANK C;

D = UNION B2, C2;
STORE D INTO '/location/to/my/outputs/1';

有人知道如何解决这个问题吗？

完整的错误日志为:

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration.
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:873)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:298)
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
  at org.apache.pig.PigServer.launchPlan(PigServer.java:1334)
  at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1319)
  at org.apache.pig.PigServer.execute(PigServer.java:1309)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:387)
  at org.apache.pig.PigServer.executeBatch(PigServer.java:365)
  at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
  at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
  at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
  at org.apache.pig.Main.run(Main.java:478)
  at org.apache.pig.PigRunner.run(PigRunner.java:49)
  at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:285)
  at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:228)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
  at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
  at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
  Caused by: java.lang.NullPointerException
  at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:817)
  ... 34 more

Answer 1

这将起作用..在应用等级然后合并关系之后，只需写一个foreach

A = LOAD '/location/to/my/file';
SPLIT A INTO 
B IF $0 IS NULL,
C IF $0 IS NOT NULL;

B2 = RANK B;
C2 = RANK C;

B_EACH = FOREACH B2 *;
C_EACH = FOREACH C2 *;

D = UNION B_EACH,C_EACH;
STORE D INTO '/location/to/my/outputs/1';