java - Cloudera MapReduce计数器getValue错误

Question

我目前正在Cloudera上使用Counters开发MapReduce仅 map 程序。 Mapper类将增加特定的Counter，并且我想在MapReduce作业完成后显示每个Counter的最终值。以下是我的Mapper类代码:

public class MyMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
    public static enum MY_COUNTER {
        C1,
        C2
    }

    //mapper logic that produces String variable 'final'

    if (final.equals("Foo")) context.getCounter(MY_COUNTER.C1).increment(1);
    else context.getCounter(MY_COUNTER.C2).increment(1);

    //context.write() method
}

下面是我的驱动程序类代码:

public class MyDriver extends Configured implements Tool {
    public static void main(String[] args) throws Exception {
        int exitCode = ToolRunner.run(new MyDriver(), args);
        System.exit(exitCode);
    }

    public int run(String[] args) throws Exception {
        Job job = Job.getInstance(getConf(), "My MapReduce");

        //Job configuration:
        //Sets mapper to MyMapper class
        //Sets num of Reduce tasks to 0
        //Other necessary job config

        boolean success = job.waitForCompletion(true);
        if (success) {
            Counter counter1 = job.getCounters().findCounter("MY_COUNTER", "C1");
            System.out.println(counter1.getDisplayName() + ": " + counter1.getValue());
            Counter counter2 = job.getCounters().findCounter("MY_COUNTER", "C2");
            System.out.println(counter2.getDisplayName() + ": " + counter2.getValue());
            return 0;
        }
        else return 1;
    }
}

当我运行jar文件时，作业成功执行。因为我将 job.waitForCompletion()参数设置为true，所以它将所有MapReduce进度打印到终端。我可以从那里看到我的柜台的值(value)。

18/03/27 09:59:58 INFO mapreduceJob: Counters: 35

    //all built-in counters

    MyMapper$MY_COUNTER
        C1=837
        C2=119

但是，当我在作业完成后打印计数器的值时(来自MyDriver类的 if(success)部分)，打印的值全为零。

C1: 0
C2: 0

关于我可能在哪里错的任何建议？

注意:我正在使用Hadoop 2.6.0-cdh5.12.0

Answer 1

找到了问题。我应该使用字符串参数而不是枚举来增加计数器。增量过程将类似于:

context.getCounter("MY_COUNTER","C1").increment(1);

这样，我什至无需在Mapper类中为我的计数器声明一个枚举。感谢 Amita帮助我。