apache-spark - 没有后缀的 spark.network.timeout 属性的单位是什么？秒还是毫秒？

Question

如果 spark.network.timeout 没有像这样定义任何后缀(s 或 ms):

spark.network.timeout 600

那是秒还是毫秒？

https://spark.apache.org/docs/latest/configuration.html 上的默认值
也用后缀“120s”定义

Answer 1

鉴于 lines在 HeartbeatReceiver似乎spark.network.timeout以秒为单位。

// "spark.network.timeout" uses "seconds", while `spark.storage.blockManagerSlaveTimeoutMs` uses
// "milliseconds"
private val slaveTimeoutMs =
  sc.conf.getTimeAsMs("spark.storage.blockManagerSlaveTimeoutMs", "120s")
private val executorTimeoutMs =
  sc.conf.getTimeAsSeconds("spark.network.timeout", s"${slaveTimeoutMs}ms") * 1000

Spark官方文档在 Spark Properties说:

Properties that specify some time duration should be configured with a unit of time.

Properties that specify a byte size should be configured with a unit of size.

很难找到，但希望你的问题和我的回答变得更容易。