apache-kafka - kafka ack=all 和 min-isr

Question

摘要

Kafka 的文档和代码注释建议，当生产者设置 acks设置为 all那么只有在 时才会向生产者发送 ack所有同步副本都 catch 了 ，但代码( Partition.Scala ， checkEnoughReplicasReachOffset )似乎表明只要 就发送 ack最小同步副本已 catch .

详情

卡夫卡文档有这个:

acks=all This means the leader will wait for the full set of in-sync replicas to acknowledge the record. source

另外，查看 Kafka 源代码 - partition.scala checkEnoughReplicasReachOffset()有以下评论(强调我的):

Note that this method will only be called if requiredAcks = -1 and we are waiting for all replicas in ISR to be fully caught up to the (local) leader's offset corresponding to this produce request before we acknowledge the produce request.

最后， this answer关于 Stack Overflow(再次强调我的)

Also the min in-sync replica setting specifies the minimum number of replicas that need to be in-sync for the partition to remain available for writes. When a producer specifies ack (-1 / all config) it will still wait for acks from all in sync replicas at that moment (independent of the setting for min in-sync replicas).

但是当我查看 Partition.Scala 中的代码时(注意 minIsr < curInSyncReplicas.size):

def checkEnoughReplicasReachOffset(requiredOffset: Long): (Boolean, Errors) = {
  ...
  val minIsr = leaderReplica.log.get.config.minInSyncReplicas
  if (leaderReplica.highWatermark.messageOffset >= requiredOffset) {          
    if (minIsr <= curInSyncReplicas.size)
      (true, Errors.NONE)

调用它的代码返回 ack:

if (error != Errors.NONE || hasEnough) {
  status.acksPending = false
  status.responseStatus.error = error
}

因此，一旦同步副本集大于最小同步副本，代码看起来就会返回 ack。但是，文档和评论表明只有在所有同步副本都 catch 后才会发送 ack。我错过了什么？至少，上面的评论 checkEnoughReplicasReachOffset看起来它应该改变。

Answer 1

感谢 jira-dev 邮件列表中的 Ismael。

The key point is the line:

if(leaderReplica.highWatermark.messageOffset >= requiredOffset) {

The high watermark only moves when all the replicas in ISR have that particular offset.