hadoop - mapreduce 中的 NoSuchElementException

Question

我是 map reduce 的新手，遇到 NoSuchElementException，请帮忙。

在文本下方输入文件容器:

this is a hadoop program
i am writing it for first time

映射器类:

public class Mappers extends MapReduceBase implements Mapper<LongWritable, Text, IntWritable, IntWritable>{
    private Text word = new Text();
    private IntWritable singleWordCount = new IntWritable();
    private IntWritable one = new IntWritable(1);

    @Override
    public void map(LongWritable key, Text value, OutputCollector<IntWritable, IntWritable> output, Reporter reporter) throws IOException {
         StringTokenizer wordList = new StringTokenizer(value.toString());
         while (wordList.hasMoreTokens()) {
             int wordSize = wordList.nextToken().length();
             singleWordCount.set(wordSize);
             if(word != null && wordList != null && wordList.nextToken() != null){
                 word.set(wordList.nextToken());
                 output.collect(singleWordCount, one);
             }
        }
    }

}

This is the error I am getting

Answer 1

您在循环中为每次迭代调用了三次 wordList.nextToken()。每次你调用它 StringTokenizer 都会返回下一个标记，当你的程序在你的文本中遇到单词 first 时会导致异常，因为你检索 first 然后 time 然后尝试检索下一个不存在的单词，导致异常。

您需要做的是在每次迭代中检索一次并将其存储在变量中。或者，如果您确实需要在一次迭代中检索两个词，请始终调用 hasMoreTokens() 以检查在您实际调用 nextToken() 之前是否确实有另一个词要处理。