gpt4 book ai didi

xml - 无法启动MapReduce任务。找不到或加载主类org.apache.hadoop.mapreduce.v2.app.MRAppMaster

转载 作者:行者123 更新时间:2023-12-02 21:04:27 35 4
gpt4 key购买 nike

我是Hadoop的初学者,我尝试使用MR将一些数据导入HBase,但是失败了。控制台显示错误信息:

Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Here is the screenshot of the whole error log page

我通过google和stackoverflow找到了一些解决方案,大多数解决方案都说它与mapred-site.xml和yarn-site.xml的配置有关,但是,我发现我没有任何问题。 mapreduce.application.classpath有问题吗?

mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>
/usr/local/hadoop/etc/hadoop,
/usr/local/hadoop/share/hadoop/common/*,
/usr/local/hadoop/share/hadoop/common/lib/*,
/usr/local/hadoop/share/hadoop/hdfs/*,
/usr/local/hadoop/share/hadoop/hdfs/lib/*,
/usr/local/hadoop/share/hadoop/mapreduce/*,
/usr/local/hadoop/share/hadoop/mapreduce/lib/*,
/usr/local/hadoop/share/hadoop/yarn/*,
/usr/local/hadoop/share/hadoop/yarn/lib/*
</value>
</property>
<property>
<name>mapreduce.app-submisson.cross-platform</name>>
<value>true</value>>
</property>>
</configuration>

yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
<property>
<name>yarn.application.classpath</name>
<value>
/usr/local/hadoop/etc/hadoop,
/usr/local/hadoop/share/hadoop/common/*,
/usr/local/hadoop/share/hadoop/common/lib/*,
/usr/local/hadoop/share/hadoop/hdfs/*,
/usr/local/hadoop/share/hadoop/hdfs/lib/*,
/usr/local/hadoop/share/hadoop/mapreduce/*,
/usr/local/hadoop/share/hadoop/mapreduce/lib/*,
/usr/local/hadoop/share/hadoop/yarn/*,
/usr/local/hadoop/share/hadoop/yarn/lib/*
</value>
</property>
</configuration>

最佳答案

应该用冒号(:)而不是逗号(,)来分隔类路径,将mapreduce.application.classpathyarn.application.classpath修改为/usr/local/hadoop/etc/hadoop:/usr/local/hadoop/share/hadoop/common/*:...,然后看看是否可以解决问题。

关于xml - 无法启动MapReduce任务。找不到或加载主类org.apache.hadoop.mapreduce.v2.app.MRAppMaster,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/42358276/

35 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com