hadoop - 映射代码中的 ArrayIndexOutOfBoundException

Question

输入是 1000:rohit:male:dev:2500

在此我想数一数男性和女性的数量。当我使用 split 时，将每个性别字段分配给 reducer 显示 ArrayIndexOutOfBounfException:2

public class DeptEmpcountMapper extends
        Mapper<LongWritable, Text, Text, LongWritable> {

    @Override
    protected void map(LongWritable key, Text value, Context context)

            throws IOException, InterruptedException {

        String st = value.toString();

        String[] field = st.split(":");
        String st1 = field[2];

        context.write(new Text(st1), new LongWritable(1));
    }

}

Answer 1

您正在做出危险的假设，即每条输入记录都采用 1000:rohit:male:dev:2500 形式。从你的错误来看，很明显情况并非如此。坏数据总是需要考虑的问题。

事先考虑一些简单的输入验证:

String[] field = st.split(":");
if(ArrayUtils.getLength(field) == 5) {
    String st1 = field[2];
    context.write(new Text(st1), new LongWritable(1));
} else {
    //Consider printing "st" to see what the bad input looks like
}