ant - Amazon EMR 教程示例不使用 ant 进行编译

Question

我正在尝试做 Amazon EMR (Elastic MapReduce) 的教程 https://www.youtube.com/embed/ZSu8N2PoVbM?rel=0&hd=1;autoplay=1

我下载了教程示例http://elasticmapreduce.s3.amazonaws.com/training/wikipedia-lab.tgz ，它计算维基百科文档集合中二元组的数量。我一开始就卡住了:

2. Build the job jar

   [On your dev machine]

   % ant clean job

   This will create the wikipedia-ngrams-job.jar Hadoop job jar
   file in your build sub-directory. If you have Hadoop installed on
   your development machine, you can try running it locally via:

   % hadoop jar build/wikipedia-ngrams-job.jar -inputfile src/test/resources/enwiki-split.xml -outputdir build/test

   This will generate text output files in build/test/raw-counts and build/test/sorted-counts.
   To view the results, you can dump the output (these are text files), e.g.

   % cat build/test/sorted-counts/part-r-00000

这是终端输出:

> ant clean job
Buildfile: /Users/jfk/work/data/aws_s3/wikipedia-lab/build.xml

clean:
     [echo] cleaning wikipedia-ngrams

mvn-init:
[artifact:dependencies] [WARNING] POM for 'org.codehaus.janino:janino:pom:2.5.16:compile' is invalid. It will be ignored for artifact resolution. Reason: Not a v4.0.0 POM. for project org.codehaus.janino:janino at /Users/jfk/.m2/repository/org/codehaus/janino/janino/2.5.16/janino-2.5.16.pom

[...]

compile:
    [mkdir] Created dir: /Users/jfk/work/data/aws_s3/wikipedia-lab/build/classes-main
    [javac] Compiling 14 source files to /Users/jfk/work/data/aws_s3/wikipedia-lab/build/classes-main
    [javac] error: error reading /Users/jfk/.m2/repository/org/codehaus/janino/janino/2.5.16/janino-2.5.16.jar; cannot read zip file
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:9: package org.apache.commons.lang.builder does not exist
    [javac] import org.apache.commons.lang.builder.ReflectionToStringBuilder;
    [javac]                                       ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:10: package org.apache.commons.lang.builder does not exist
    [javac] import org.apache.commons.lang.builder.ToStringStyle;
    [javac]                                       ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:3: package org.apache.commons.lang.builder does not exist
    [javac] import org.apache.commons.lang.builder.ReflectionToStringBuilder;
    [javac]                                       ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:4: package org.apache.commons.lang.builder does not exist
    [javac] import org.apache.commons.lang.builder.ToStringStyle;
    [javac]                                       ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:3: package org.apache.commons.lang.builder does not exist
    [javac] import org.apache.commons.lang.builder.ReflectionToStringBuilder;
    [javac]                                       ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:4: package org.apache.commons.lang.builder does not exist
    [javac] import org.apache.commons.lang.builder.ToStringStyle;
    [javac]                                       ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlTool.java:17: package org.apache.commons.io does not exist
    [javac] import org.apache.commons.io.IOUtils;
    [javac]                             ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:84: cannot find symbol
    [javac] symbol  : variable ToStringStyle
    [javac] location: class com.amazon.aws.training.emr.wikipedia.NgramsJobOptions
    [javac]         return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
    [javac]                                                         ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/amazon/aws/training/emr/wikipedia/NgramsJobOptions.java:84: cannot find symbol
    [javac] symbol  : variable ReflectionToStringBuilder
    [javac] location: class com.amazon.aws.training.emr.wikipedia.NgramsJobOptions
    [javac]         return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
    [javac]                ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:41: cannot find symbol
    [javac] symbol  : variable ToStringStyle
    [javac] location: class com.scaleunlimited.wikipedia.ProcessXmlOptions
    [javac]         return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
    [javac]                                                         ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlOptions.java:41: cannot find symbol
    [javac] symbol  : variable ReflectionToStringBuilder
    [javac] location: class com.scaleunlimited.wikipedia.ProcessXmlOptions
    [javac]         return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
    [javac]                ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:52: cannot find symbol
    [javac] symbol  : variable ToStringStyle
    [javac] location: class com.scaleunlimited.wikipedia.SplitXmlOptions
    [javac]         return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
    [javac]                                                         ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlOptions.java:52: cannot find symbol
    [javac] symbol  : variable ReflectionToStringBuilder
    [javac] location: class com.scaleunlimited.wikipedia.SplitXmlOptions
    [javac]         return ReflectionToStringBuilder.toString(this, ToStringStyle.MULTI_LINE_STYLE);
    [javac]                ^
    [javac] /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/SplitXmlTool.java:128: cannot find symbol
    [javac] symbol  : variable IOUtils
    [javac] location: class com.scaleunlimited.wikipedia.SplitXmlTool
    [javac]         InputStream is = IOUtils.toInputStream(xml, "UTF-8");
    [javac]                          ^
    [javac] Note: /Users/jfk/work/data/aws_s3/wikipedia-lab/src/main/java/com/scaleunlimited/wikipedia/ProcessXmlTool.java uses or overrides a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 15 errors

BUILD FAILED
/Users/jfk/work/data/aws_s3/wikipedia-lab/build.xml:70: Compile failed; see the compiler error output for details.

Total time: 2 seconds

欢迎任何帮助。

Answer 1

删除 .m2/repository 目录然后执行 mvn clean install 帮助了我