hadoop - 运行Mapreduce问题

Question

我正在尝试在群集hadoop 2.7.1(一个主节点和四个从属节点)上运行wordcount jar，但是MapReduce作业在以下位置被阻止:

$ hadoop jar wc.jar WordCount /input /output_hocine 
17/03/13 09:41:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
17/03/13 09:41:43 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
17/03/13 09:41:43 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 
17/03/13 09:41:44 INFO input.FileInputFormat: Total input paths to process : 3 
17/03/13 09:41:44 INFO mapreduce.JobSubmitter: number of splits:3 
17/03/13 09:41:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1489393376058_0003 
17/03/13 09:41:44 INFO impl.YarnClientImpl: Submitted application application_1489393376058_0003 
17/03/13 09:41:44 INFO mapreduce.Job: The url to track the job: http://ibnbadis21:8088/proxy/application_1489393376058_0003/ 
17/03/13 09:41:44 INFO mapreduce.Job: Running job: job_1489393376058_0003

通过导航器，此图显示了通过导航器的输出:



这是配置文件的内容:

Core-site.xml:

<configuration>
<!--    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://ibnbadis21:9000</value>
    </property>-->
    <property>
        <name>fs.default.name</name>
        <value>hdfs://ibnbadis21:9000</value>
    </property>
   <property>
      <name>dfs.permissions</name>
      <value>false</value>
   </property>
 </configuration>

yarn-site.xml:

<?xml version="1.0"?> <configuration>
   <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
</configuration>

mapred-site.xml:

<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. -->
<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
        <property>
                <name>mapreduce.jobhistory.address</name>
                <value>ibnbadis21:10020</value>
        </property>
        <property>
                <name>mapreduce.jobhistory.webapp.address</name>
                <value>ibnbadis21:19888</value>
        </property>
        <property>
                <name>yarn.app.mapreduce.am.staging-dir</name>
                <value>/user/app</value>
        </property>
</configuration>

hdfs-site.xml:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
 <property>
        <name>dfs.replication</name>
        <value>2</value>
 </property>
 <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/usr/local/hadoop/hadoop_data/hdfs/namenode</value>
 </property>
 <property> <name>dfs.namenode.checkpoint.dir</name>
        <value>file:/usr/local/hadoop_data/hdfs/namesecondary</value>
 </property>
 <property> <name>dfs.datanode.data.dir</name>
        <value>file:/usr/local/hadoop_data/hdfs/datanode</value>
 </property>
</configuration>

谁能告诉我如何解决这个问题？

Answer 1

Connecting to ResourceManager at /0.0.0.0:8032

0.0.0.0(默认)不是有效的主机名。

所以，在 yarn-site.xml中添加它

<property>
    <name>yarn.resourcemanager.hostname</name>
    <value> YOUR VALUE HERE </value> <!-- Needs Fully Qualified Domain Name -->
</property>

您可能没有设置很多值。

请参阅 Hadoop | Configuring the Hadoop Daemons

顺便说一句， fs.defaultFS是要使用的正确属性。