maven - 构建 hadoop 2.3.0 - 发生 Ant BuildException : exec returned: 2

Question

我面对WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 在处理 hadoop 2.3.0 的东西时 在 ubuntu 64 位 .正如互联网所说，要构建 hadoop 源并将输出放到 库 .

我遵循的步骤在这里https://github.com/prayagupd/hadoop-prayaman/blob/master/prayaman.sh#L316

所以，我创建了以下脚本来构建 hadoop 2.3.0，

[完整脚本在 github ]

configureHadoopLib(){
   sudo apt-get install -y cmake
   sudo apt-get install -y build-essential
   sudo apt-get install g++ autoconf automake
   sudo apt-get install zlib1g-dev
   sudo apt-get install libssl-dev
   wget http://apache.mirror.anlx.net/hadoop/core/hadoop-2.3.0/hadoop-2.3.0-src.tar.gz
   tar -zxvf hadoop-2.3.0-src.tar.gz
   cd hadoop-2.3.0-src/
   #cd $HADOOP_HOME/src
   echo
   echo "#######################################################################"
   echo "[INFO] Packaging hadoop source"
   echo "#######################################################################"
   #mvn package -Pdist,native -Dskiptests -Dtar
   mvn -e package -Dmaven.javadoc.skip=true -Pdist,native -DskipTests -Dtar
   echo
   echo "#######################################################################"
   echo "[INFO] Packaging hadoop source - completed"
   echo "#######################################################################"

   #cp $HADOOP_HOME/src/hadoop-dist/target/hadoop-2.3.0-cdh5.0.1/lib/native/*  $HADOOP_HOME/lib/native/
   #cat ~/.bash_profile > export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
   #cat ~/.bash_profile > export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
}

我因以下错误而失败=> 无法在项目 hadoop-common 上执行目标 o.a.m.p:maven-antrun-plugin:1.7:run (make)，

[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:06.730s
[INFO] Finished at: Mon Aug 18 18:22:00 NPT 2014
[INFO] Final Memory: 67M/635M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...<exec dir="/home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/native" executable="make" failonerror="true">... @ 7:172 in /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2
around Ant part ...<exec dir="/home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/native" executable="make" failonerror="true">... @ 7:172 in /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 2
around Ant part ...<exec dir="/home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/native" executable="make" failonerror="true">... @ 7:172 in /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
    at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
    ... 19 more
Caused by: /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:7: exec returned: 2
    at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
    at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
    at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
    at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
    at org.apache.tools.ant.Task.perform(Task.java:348)
    at org.apache.tools.ant.Target.execute(Target.java:390)
    at org.apache.tools.ant.Target.performTasks(Target.java:411)
    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
    at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
    at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
    ... 21 more
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-common

#######################################################################
[INFO] Packaging hadoop source - completed
#######################################################################

协议(protocol)版本

$ protoc --version
libprotoc 2.5.0

mvn 版本是

$ mvn --version
Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.7.0, vendor: Oracle Corporation
Java home: /usr/local/jdk1.7.0/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.8.0-19-generic", arch: "amd64", family: "unix"

Ant 版本是

$ ant -version
Apache Ant(TM) version 1.8.2 compiled on May 18 2012

有趣的是，在两台不同的机器上，构建在两个不同的项目中失败，但出现相同的 ant 错误。

引用文献

http://www.elcct.com/installing-hadoop-2-3-0-on-ubuntu-13-10/

https://stackoverflow.com/a/17135947/432903

https://stackoverflow.com/a/23127967/432903

Hadoop 2.4 Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7

Answer 1

如果赶时间，可以从https://github.com/mehikmat/hadoop-install/tree/cdh5/native-libs下载使用hadoop原生库的预编译版本