gpt4 book ai didi

hadoop 2.2,Windows 7 上的字数统计示例失败

转载 作者:可可西里 更新时间:2023-11-01 15:38:44 24 4
gpt4 key购买 nike

在执行 hadoop jar 命令以运行默认字数统计程序后,我开始关注 stderr。

java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService and  Could not find the
main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster

我相信我不必显式设置 hadoop 类路径。因为我已经设置了环境。变量如下:还是我在这里遗漏了什么?

%HADOOP_HOME% =c:\hadoop
HADOOP_COMMON_HOME =%HADOOP_HOME%
HADOOP_CONF_DIR=%HADOOP_HOME%\etc\hadoop
HADOOP_HDFS_HOME =%HADOOP_HOME%
HADOOP_MAPRED_HOME =%HADOOP_HOME%

我正在执行的命令是

hadoop jar c:/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /input /output

The output @ sdk command prompt is

13/12/20 16:26:00 INFO mapreduce.Job: Job job_1387536911324_0001 failed with sta
te FAILED due to: Application application_1387536911324_0001 failed 2 times due
to AM Container for appattempt_1387536911324_0001_000002 exited with exitCode:
1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.la
unchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.C
ontainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.C
ontainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:918)
at java.lang.Thread.run(Thread.java:662)

1 file(s) moved.

ANd the stderr file is

java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
Exception in thread "main"

最佳答案

我正在运行 Linux,我遇到了完全相同的问题。通过将此添加到 yarn-site.xml 解决了这个问题:

        <property>
<description>Classpath for typical applications.</description>
<name>yarn.application.classpath</name>
<value>
$HADOOP_CONF_DIR,
$HADOOP_COMMON_HOME/share/hadoop/common/*,
$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,
$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,
$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,
$HADOOP_YARN_HOME/share/hadoop/yarn/*,
$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*
</value>
</property>

关于hadoop 2.2,Windows 7 上的字数统计示例失败,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/20699632/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com