hadoop - cloudera垃圾检查点间隔配置

Question

Cloudera 允许我配置 fs.trash.interval。

但它不允许我配置 fs.trash.checkpoint.interval。

那么 hdfs 什么时候创建检查点呢？

这里有一个类似的问题没有回应: When does Hadoop Framework creates a checkpoint (expunge) to its "current" directory in trash?

Answer 1

Apache Hadoop文档包括左侧导航中指向各种 *-default.xml 文件的链接。这些文件包含所有配置属性的默认设置。

如果您点击 *-default.xml 链接，该网站会在漂亮的表格中显示它们。这是来自 core-site.xml 的原始 XML 版本讨论垃圾属性。

<property>
  <name>fs.trash.interval</name>
  <value>0</value>
  <description>Number of minutes after which the checkpoint
  gets deleted.  If zero, the trash feature is disabled.
  This option may be configured both on the server and the
  client. If trash is disabled server side then the client
  side configuration is checked. If trash is enabled on the
  server side then the value configured on the server is
  used and the client configuration value is ignored.
  </description>
</property>

<property>
  <name>fs.trash.checkpoint.interval</name>
  <value>0</value>
  <description>Number of minutes between trash checkpoints.
  Should be smaller or equal to fs.trash.interval. If zero,
  the value is set to the value of fs.trash.interval.
  Every time the checkpointer runs it creates a new checkpoint 
  out of current and removes checkpoints created more than 
  fs.trash.interval minutes ago.
  </description>
</property>

根据这个描述，如果你没有改变fs.trash.checkpoint.interval，那么它使用与fs.trash.interval相同的值，并且这就是它创建垃圾检查点的频率。

fs.trash.checkpoint.interval 配置属性是在 Apache Hadoop 2.x 发行版中引入的。旧版本不支持此配置属性，您可以认为该行为等同于让 fs.trash.checkpoint.interval 等于 fs.trash.interval。