Hadoop NameNode 不启动

Question

我目前正在尝试在 Amazon EC2 实例(多节点集群)上运行 Hadoop 2.6.0。我启动了两个 Ubuntu 14.04 实例。其中一个是主人，另一个是奴隶。以下是我的配置:

大师

-核心站点.xml

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://hadoopmaster:9000</value>
        </property>
</configuration>

-hdfs-site.xml

<configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>file:/home/ubuntu/hadoop-2.6.0/hadoop_data/hdfs/namenode</value>
        </property>
</configuration>

-yarn-site.xml

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
</property>
<property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>hadoopmaster:8025</value>
</property>
<property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>hadoopmaster:8030</value>
</property>
<property>
        <name>yarn.resourcemanager.address</name>
        <value>hadoopmaster:8050</value>
</property>

-mapred-site.xml

<configuration>
        <property>
                <name>mapred.job.tracker</name>
                <value>hadoopmaster:54311</value>
        </property>
</configuration>

-大师

hadoopmaster

-奴隶

hadoopslave1

奴隶

-hdfs-site.xml

<configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>file:/home/ubuntu/hadoop-2.6.0/hadoop_data/hdfs/datanode</value>
        </property>
</configuration>

其他同master。

当我运行 hdfs namenode -format 时，它看起来很好并以状态 0 退出。当我运行 start-all.sh 时，它给出以下输出:

ubuntu@hadoopmaster:~/hadoop-2.6.0$ sbin/start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [hadoopmaster]
hadoopmaster: starting namenode, logging to /home/ubuntu/hadoop-2.6.0/logs/hadoop-ubuntu-namenode-hadoopmaster.out
hadoopslave1: starting datanode, logging to /home/ubuntu/hadoop-2.6.0/logs/hadoop-ubuntu-datanode-hadoopslave1.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/ubuntu/hadoop-2.6.0/logs/hadoop-ubuntu-secondarynamenode-hadoopmaster.out
starting yarn daemons
starting resourcemanager, logging to /home/ubuntu/hadoop-2.6.0/logs/yarn-ubuntu-resourcemanager-hadoopmaster.out
hadoopslave1: starting nodemanager, logging to /home/ubuntu/hadoop-2.6.0/logs/yarn-ubuntu-nodemanager-hadoopslave1.out

听起来不错，没有报错。但是，当我在主站点上运行 jps 时，它会给出以下输出:

ubuntu@hadoopmaster:~/hadoop-2.6.0$ jps
3640 ResourceManager
3501 SecondaryNameNode
3701 Jps

NameNode 不见了!当我在从站点上运行 jps 时，我得到了以下信息

ubuntu@hadoopslave1:~/hadoop-2.6.0$ jps
1686 DataNode
1870 Jps
1817 NodeManager

NameNode 的日志文件如下:

2015-06-11 04:16:18,987 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = hadoopmaster/54.172.40.127
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 2.6.0

和 .out 文件:

ulimit -a for user ubuntu
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 13357
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 13357
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

我重复了几次，结果都是一样的。 NameNode 总是丢失。谁能给我一些关于这个问题的建议？非常感谢!

Answer 1

我猜你已经找到了解决方案，但这适用于遇到同样问题的其他人(比如我)。

首先使用 stop-yarn.sh 和 stop-dfs.sh(按此顺序)关闭您的 hadoop 集群。现在您所要做的就是转到 hadoop 临时目录。如果用户没有配置，那么它将在 /usr/local/hadoop/tmp/ 中。

否则从属性 hadoop.tmp.dir 的 core-site.xml 中找到它。然后简单地输入:

rm -rf *

现在启动集群，中提琴，Namenode 已启动。