hadoop - 无法检查 hadoop 上的节点 [连接被拒绝]

Question

如果我键入 http://localhost:50070 或 http://localhost:9000 来查看节点，我的浏览器不会显示任何内容，我认为它可以' t 连接到服务器。我用这个命令测试了我的 hadoop:

hadoop jar hadoop-*test*.jar TestDFSIO -write -nrFiles 10 -fileSize 1000

但也没有用，它尝试连接到服务器，这是输出:

12/06/06 17:25:24 INFO mapred.FileInputFormat: nrFiles = 10
12/06/06 17:25:24 INFO mapred.FileInputFormat: fileSize (MB) = 1000
12/06/06 17:25:24 INFO mapred.FileInputFormat: bufferSize = 1000000
12/06/06 17:25:25 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 time(s).
12/06/06 17:25:26 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 1 time(s).
12/06/06 17:25:27 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 2 time(s).
12/06/06 17:25:28 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 3 time(s).
12/06/06 17:25:29 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 4 time(s).
12/06/06 17:25:30 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 5 time(s).
12/06/06 17:25:31 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 6 time(s).
12/06/06 17:25:32 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 7 time(s).
12/06/06 17:25:33 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 8 time(s).
12/06/06 17:25:34 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 9 time(s).
java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on connection exception: java.net.ConnectException: Connection refused

我像这样更改了一些文件:在 conf/core-site.xml 中:

<configuration>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
  </property>

在 conf/hdfs-site.xml 中:

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>
</configuration>

在 conf/mapred-site.xml 中:

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:9001</value>
  </property>
</configuration>

感谢您的关注。如果我运行这个命令:

cat /etc/hosts

我明白了:

127.0.0.1   localhost
127.0.1.1   ubuntu.ubuntu-domain    ubuntu

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

如果我运行这个:

ps axww | grep hadoop

我看到这个结果:

2170 pts/0    S+     0:00 grep --color=auto hadoop

但没有效果。你有什么想法，我该如何解决我的问题？

Answer 1

在启动 hadoop 服务之前，您需要注意几件事情。

检查返回的内容:

hostname --fqdn 

在您的情况下，这应该是本地主机。还要在/etc/hosts 中注释掉 IPV6。

在启动 HDFS 之前，您是否格式化了名称节点。

hadoop namenode -format

您是如何安装 Hadoop 的。日志文件的位置将取决于此。如果您使用过cloudera的发行版，通常它位于“/var/log/hadoop/”位置。

如果您是新手，我建议您使用 Cloudera SCM 安装 Hadoop，这非常简单。我已经发布了my approach in installing Hadoop使用 Cloudera 的发行版。

还有

确保 DFS 位置具有写入权限。它通常位于@ /usr/local/hadoop_store/hdfs

这是一个常见的原因。