Hadoop:无法找到或加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Question

在安装 Hadoop 后，我试图启动一个相当简单的 WordCount(我非常关注 this tutorial)，但我得到了这个:

2018-04-05 16:51:00,192 INFO mapreduce.Job: Job job_1522936330711_0007 failed with state FAILED due to: Application application_1522936330711_0007 failed 2 times due to AM Container for appattempt_1522936330711_0007_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2018-04-05 16:50:59.449]Exception from container-launch.
Container id: container_1522936330711_0007_02_000001
Exit code: 1

[2018-04-05 16:50:59.452]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error : Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster


[2018-04-05 16:50:59.452]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error : Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster


For more detailed output, check the application tracking page: http://iMac-de-chaire-2.local:8088/cluster/app/application_1522936330711_0007 Then click on links to logs of each attempt.
. Failing the application.
2018-04-05 16:51:00,211 INFO mapreduce.Job: Counters: 0

当我尝试访问日志时，它告诉我“无法读取应用程序 application_1522940117199_0001 的尝试。”

我有以下文件:

hadoop-env.sh :

export HADOOP_HOME=/usr/local/hadoop/

export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

yarn-site.xml :

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>

<property>
    <name>yarn.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>
</configuration>

mapred-site.xml :

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
<property>
    <name>mapreduce.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>
</configuration>

我在 macosx high-sierra 上，这是我第一次尝试设置 Hadoop(我遵循了 this tutorial)。

Answer 1

我在 macosx high-sierra 上也遇到了同样的问题。要解决此问题，请将这些属性添加到 mapred-site.xml:

<property>
 <name>yarn.app.mapreduce.am.env</name>
 <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
 <name>mapreduce.map.env</name>
 <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
 <name>mapreduce.reduce.env</name>
 <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>