gpt4 book ai didi

hadoop - 辅助名称节点是否也更新存储在 NFS 中的元数据?

转载 作者:可可西里 更新时间:2023-11-01 16:36:21 25 4
gpt4 key购买 nike

我正在阅读“Hadoop:权威指南”。 Hadoop 2.x之前作者是这样解释容错的

Without the namenode, the filesystem cannot be used. In fact, if the machine running the namenode were obliterated, all the files on the filesystem would be lost since there would be no way of knowing how to reconstruct the files from the blocks on the datanodes. For this reason, it is important to make the namenode resilient to failure, and Hadoop provides two mechanisms for this. The first way is to back up the files that make up the persistent state of the filesystem metadata. Hadoop can be configured so that the namenode writes its persistent state to multiple filesystems. These writes are synchronous and atomic. The usual configuration choice is to write to local disk as well as a remote NFS mount. It is also possible to run a secondary namenode, which despite its name does not act as a namenode. Its main role is to periodically merge the namespace image with the edit log to prevent the edit log from becoming too large. The secondary namenode usually runs on a separate physical machine because it requires plenty of CPU and as much memory as the namenode to perform the merge. It keeps a copy of the merged name‐ space image, which can be used in the event of the namenode failing. However, the state of the secondary namenode lags that of the primary, so in the event of total failure of the primary, data loss is almost certain. The usual course of action in this case is to copy the namenode’s metadata files that are on NFS to the secondary and run it as the new primary

我的理解是 NFS 总是与主名称节点同步。我的问题是,在次要名称节点更新了主要名称节点的元数据后,存储在 NFS 中的元数据如何与主要名称节点同步?如果主节点在 NFS 同步之前完全失败会怎样?

最佳答案

该文档并没有说“主”或“次要”NameNode 必须与 NFS 同步,而是说如果您已将 Namenode 备份配置为 NFS(我相信您必须自己做一些事情,因为它说这是“配置选择”),您可以将它们恢复到新服务器并将其指定为新的Namenode。注意“尽管它的名字(辅助名称节点)不充当名称节点”,并且“辅助名称节点的状态滞后于主名称节点”,因此它永远不会获得没有的数据'如果还没有到达主节点,它将检查已经存在的内容。

引用的部分暗指 having a Standby Namenode ,它的用途与次要的不同,备用应该同步

引用自该链接,

Note that, in an HA cluster, the Standby NameNode also performs checkpoints of the namespace state, and thus it is not necessary to run a Secondary NameNode, CheckpointNode, or BackupNode in an HA cluster. In fact, to do so would be an error

关于hadoop - 辅助名称节点是否也更新存储在 NFS 中的元数据?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/51945766/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com