java - 如何为 64 位 Linux 机器编译 Hadoop？

Question

我已经下载了 Hadoop (2.2.0) 的最新稳定二进制文件。就在我初始化 HDFS 时，我收到了这个警告:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

我知道我可以通过从源代码编译来解决这个问题，所以我从 Hadoop 下载了源代码包。我知道编译的基本过程，但在阅读 README 后感到困惑。快速谷歌显示我必须为此使用 maven，这是一个构建基于 java 的项目的工具。

所以我的问题是，如何使用 maven 从源代码编译 Hadoop？我应该进入每个目录并编译每个模块吗？分步指南将非常有帮助，我们将不胜感激。

Answer 1

提取源代码后，您会在如下位置找到一个 super pom。\hadoop-2.2.0-src.tar\hadoop-2.2.0-src\hadoop-2.2.0-src\pom.xml这将构建所有模块。您可以使用命令构建:mvn clean install

您应该注意到如下日志。

            [INFO] ------------------------------------------------------------------------
            [INFO] Reactor Build Order:
            [INFO]
            [INFO] Apache Hadoop Main
            [INFO] Apache Hadoop Project POM
            [INFO] Apache Hadoop Annotations
            [INFO] Apache Hadoop Project Dist POM
            [INFO] Apache Hadoop Assemblies
            [INFO] Apache Hadoop Maven Plugins
            [INFO] Apache Hadoop Auth
            [INFO] Apache Hadoop Auth Examples
            [INFO] Apache Hadoop Common
            [INFO] Apache Hadoop NFS
            [INFO] Apache Hadoop Common Project
            [INFO] Apache Hadoop HDFS
            [INFO] Apache Hadoop HttpFS
            [INFO] Apache Hadoop HDFS BookKeeper Journal
            [INFO] Apache Hadoop HDFS-NFS
            [INFO] Apache Hadoop HDFS Project
            [INFO] hadoop-yarn
            [INFO] hadoop-yarn-api
            [INFO] hadoop-yarn-common
            [INFO] hadoop-yarn-server
            [INFO] hadoop-yarn-server-common
            [INFO] hadoop-yarn-server-nodemanager
            [INFO] hadoop-yarn-server-web-proxy
            [INFO] hadoop-yarn-server-resourcemanager
            [INFO] hadoop-yarn-server-tests
            [INFO] hadoop-yarn-client
            [INFO] hadoop-yarn-applications
            [INFO] hadoop-yarn-applications-distributedshell
            [INFO] hadoop-mapreduce-client
            [INFO] hadoop-mapreduce-client-core
            [INFO] hadoop-yarn-applications-unmanaged-am-launcher
            [INFO] hadoop-yarn-site
            [INFO] hadoop-yarn-project
            [INFO] hadoop-mapreduce-client-common
            [INFO] hadoop-mapreduce-client-shuffle
            [INFO] hadoop-mapreduce-client-app
            [INFO] hadoop-mapreduce-client-hs
            [INFO] hadoop-mapreduce-client-jobclient
            [INFO] hadoop-mapreduce-client-hs-plugins
            [INFO] Apache Hadoop MapReduce Examples
            [INFO] hadoop-mapreduce

还有更多……

但是，如果您只想使用 Hadoop，这是一个漫长的过程。您应该能够使用现有的库。可能是一些配置问题。

其他选项是 Cloudera。我已经在 RedHat Linux 上安装了它。

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Quick-Start/cdh4qs_topic_3.html

祝你好运。