gpt4 book ai didi

hadoop - 在Hadoop中设计软限制和硬限制的目的是什么?

转载 作者:行者123 更新时间:2023-12-02 20:15:49 25 4
gpt4 key购买 nike

The lease manager maintains a soft limit (1 minute) and hard limit (1 hour) for the expiration time (these limits are currently non-configurable), and all leases maintained by the lease manager abide by the same soft and hard limits. Before the soft limit expires, the client holding the lease of a file has exclusive write access to the file. If the soft limit expires and the client has not renewed the lease or closed the file (the lease of a file is released when the file is closed), another client can forcibly take over the lease. If the hard limit expires and the client has not renewed the lease, HDFS assumes that the client has quit and will automatically close the file on behalf of the client, thereby recovering the lease.


为什么需要此功能?
在什么情况下其他客户会尝试接管租赁?

最佳答案

如果当前正在写入文件的客户端不正常或已死,则另一个客户端应仅尝试接管租约。任何时候,只有1个客户端可以打开一个文件以写入HDFS。
我不确定此功能的历史,但是我怀疑1分钟的限制是为了防止客户端由于短暂的暂停(例如GC暂停或类似的操作)而失去租约。
这里有1小时的超时时间,因此namenode会关闭客户端已经长时间死机的文件。
最近发生了更改,硬超时已变为可配置的,并且默认情况下减少到20分钟。

关于hadoop - 在Hadoop中设计软限制和硬限制的目的是什么?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63929491/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com