hadoop - 使用 Hadoop NameNode、Checkpoint Node 和 Backup Node 的优缺点是什么？

Question

我目前正在为一个内部项目评估 Hadoop 1.0.2。

Hadoop 文档是这么说的

The Secondary NameNode has been deprecated. Instead, consider using the Checkpoint Node or Backup Node

有关于这三个选项是什么以及它们做什么的信息，但我找不到关于这三个选项中哪一个是推荐的信息 在什么情况下。

Answer 1

基本上，检查点节点是二级名称节点的新实现，备份点是名称节点热备份途中的临时版本(此外，它目前可以通过分离读写来提供小的性能提升- 读入名称节点，写入备份节点

来自 Backupnode documentation正如 Konstantin Shvachko 所解释的:

This patch introduces two new types of name-nodes: a Checkpoint node and a Backup node.

• The role of the Checkpoint node to checkpoint name-node meta-data by merging image and edits files.
• The Backup node extends functionality of the Checkpointer by that it can receive online updates of the file system meta-data, apply them to its memory state and persist them on disks just like the name-node does. Thus at any time the Backup node contains an up-to-date image of the namespace both in memory and on local disk(s). This also results in much more efficient checkpointing because backup node does not need to transfer files from the active name-node and does not need to replay (merge) edits.
• The Term Standby node is reserved for further extension of the backup node functionality, when cluster will be able to switch over to the new name-node if the active dies. This is mentioned in the "Warm standby provision" section of the design document.

Typical use cases:

1. Run Checkpoint node only to create checkpoints. This should be used instead of the current SecondaryNameNode, which is deprecated by the patch. I reused a lot of the SecondaryNameNode code so this effort was not wasted, it just evolved.
2. Run Backup node to support online streaming of edits and efficient checkpointing. This particularly targets eliminating NFS as a remote storage for edits.
3. Run NameNode without persistent storage at all and delegate all "persisting" functionality to the Backup node. The trick here is to start name-node with -importCheckpoint option and then run the Backup node.