python - Java错误:org.apache.hadoop.mapreduce.lib.partition.KeyFieldBasedPartitioner不是org.apache.hadoop.mapred.Partitioner

Question

  Exception in thread "main" java.lang.RuntimeException: class org.apache.hadoop.mapreduce.lib.partition.KeyFieldBasedPartitioner not org.apache.hadoop.mapred.Partitioner

at org.apache.hadoop.conf.Configuration.setClass(Configuration.java:2273)

at org.apache.hadoop.mapred.JobConf.setPartitionerClass(JobConf.java:1165)

at org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:944)

at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:128)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

尝试从日志中获取计数器...

无法获取历史记录日志；缺少工作ID

找不到计数器

正在扫描日志以查找可能的故障原因...

无法获取历史记录日志；缺少工作ID

无法获取任务日志；缺少应用程序ID

当我尝试使用基于keyfieldbasedpartitioner的partitioner类时遇到此错误。我的输入和输出格式是rawvalueprotocol，输出是制表符分隔的。
有没有人见过这个？请帮忙。

Answer 1

这是我使用时固定的
--partitioner", "org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner",