Hadoop HDFS - 缺少副本和复制 block 不足之间的区别

Question

我知道复制不足的 block 和错误复制的 block 都是由于相对于复制因子集的数据节点数较少而发生的。

但是它们有什么区别呢？

在可用数据节点为 1 的情况下将复制因子重新设置为 1，复制不足的 block 和丢失的副本错误都被清除。通过执行命令 hdfs fsck/ 确保这一点

Answer 1

摘自 Tom White 的“Hadoop:权威指南”:

Over-replicated blocksThese are blocks that exceed their target replication for the file they belong to.Normally, over-replication is not a problem, and HDFS will automatically delete excessreplicas.

Under-replicated blocksThese are blocks that do not meet their target replication for the file they belong to.HDFS will automatically create new replicas of under-replicated blocks until they meetthe target replication. You can get information about the blocks being replicated (orwaiting to be replicated) using hdfs dfsadmin -metasave .

Misreplicated blocksThese are blocks that do not satisfy the block replica placement policy (see ReplicaPlacement). For example, for a replication level of three in a multirack cluster, if allthree replicas of a block are on the same rack, then the block is misreplicated becausethe replicas should be spread across at least two racks for resilience. HDFS willautomatically re-replicate misreplicated blocks so that they satisfy the rack placementpolicy.

Corrupt blocksThese are blocks whose replicas are all corrupt. Blocks with at least one noncorruptreplica are not reported as corrupt; the namenode will replicate the noncorrupt replicauntil the target replication is met.

Missing replicasThese are blocks with no replicas anywhere in the cluster.

希望这能回答您的问题。