gpt4 book ai didi

hadoop - 一个数据节点如何知道写入管道中的下一个数据节点?

转载 作者:行者123 更新时间:2023-12-02 19:54:57 25 4
gpt4 key购买 nike

摘自“怀特,汤姆; Hadoop:明确指南;第3章,Hadoop分布式文件系统,文件写入剖析”:

...The DataStreamer streams the packets to the first datanode in the pipeline, which stores each packet and forwards it to the second datanode in the pipeline. Similarly, the second datanode stores the packet and forwards it to the third (and last) datanode in the...



它没有提到数据节点如何知道哪个是下一个要发送数据包的数据节点。

最佳答案

Namenode知道所有的datanode和机架位置。数据节点彼此之间一无所知。

客户端在写入过程中首先联系Namenode,然后发送数据节点地址以进行副本写入

相关问题-
Hadoop Replication Model - DataStreamer/Namenode

关于使用哪个地址,

HDFS’s placement policy is to put one replica on the local machine if the writer is on a datanode, otherwise on a random datanode in the same rack as that of the writer, another replica on a node in a different (remote) rack, and the last on a different node in the same remote rack



https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Replica_Placement:_The_First_Baby_Steps

关于hadoop - 一个数据节点如何知道写入管道中的下一个数据节点?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/57981804/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com