gpt4 book ai didi

hadoop - HDFS无法正常工作?

转载 作者:行者123 更新时间:2023-12-02 22:01:21 26 4
gpt4 key购买 nike

我想在运行Ubuntu操作系统的三个虚拟机上设置多节点hadoop。

我正确设置了所有内容:我将其中任何一个都切换到另一个。我配置hadoop文件。

enter code here
*This is core.site-xml*

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://Master1:54310</value>
</property>
</configuration>

This is hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>dfs.name.dir</name>
<value>/hadoop/name/</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/hadoop/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>

This is mapred.site-xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>Master1:54311</value> // this is my hostname for master node
</property>
</configuration>

![enter image description here][1]

命令行:
 inside hadoop I typed on command line:
bin/hadoop namenode -format
bin/hadoop start-all.sh
jps
bin/hadoop dfsadmin -report
enter code here

下图中的输出:
预先谢谢你,我期待你的帮助

最佳答案

更新hdfs-site.xml中的以下属性,然后尝试

<property>
<name>dfs.name.dir</name>
<value>file:/home/user/hadoop/name/</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>file:/home/user/hadoop/data</value>
</property>

我检查了日志,应该是路径错误。将其更改为如上所述的适当路径,或创建两个文件夹/ home / user / hadoop / name和/ home / user / hadoop / data并在hdfs-site.xml文件中设置其路径。

日志
2015-06-24 02:57:19,997 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory in dfs.data.dir: can not create directory: /hadoop/data
2015-06-24 02:57:19,997 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in dfs.data.dir are invalid.
2015-06-24 02:57:19,998 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

关于hadoop - HDFS无法正常工作?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/30987358/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com