apache-kafka - Kafka 中的 commit-log 是什么意思？

Question

原谅我只是学习Kafka .我遇到了一个叫commit-log的词很多次在阅读Kafka的资料时.但仍然不知道它到底是什么。提到的链接如下。

https://kafka.apache.org/documentation/#uses_commitlog

Kafka can serve as a kind of external commit-log for a distributed system. The log helps replicate data between nodes and acts as a re-syncing mechanism for failed nodes to restore their data.

https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

One of the most useful things I learned in all this was that many of the things we were building had a very simple concept at their heart: the log. Sometimes called write-ahead logs or commit logs or transaction logs,

https://kafka.apache.org/protocol.html#protocol_partitioning

Kafka is a partitioned system so not all servers have the complete data set. Instead recall that topics are split into a pre-defined number of partitions, P, and each partition is replicated with some replication factor, N. Topic partitions themselves are just ordered "commit logs" numbered 0, 1, ..., P.

提交日志是什么意思？与 DBMS 的概念有什么不同吗？怎么理解呢？谢谢。

Answer 1

从概念上讲，Kafka 提供的“提交日志”和 DBMS 使用的提交日志/事务日志/预写日志之间没有区别:它们都是关于记录对某事所做的更改，以便以后可以重播。

在 DBMS 的情况下，如果 DB 没有完全关闭并且有必要确保 DB 以一致的状态恢复服务，则会发生此重播。重要的是，在数据库中，此提交日志是数据库的实现细节，而不是数据库客户端的关注点。

在 Kafka 应用程序中，此提交日志是一流的概念。如果愿意，主题的订阅者可以为自己重建应用程序的状态(实际上是“重放日志”)。他们还可以对主题中的特定事件使用react，并了解特定状态是如何到达的，这对于传统的 DBMS 来说都不容易。