apache-kafka - 为什么在zookeeper和kafka中交换不是一个好主意？

Question

我已阅读有关的说明

do not use swap

无论是Zookeeper还是kafka。我知道kafka依靠分页缓存来将顺序日志的某些部分保留在内存中，即使它们已写入磁盘也是如此。

但是无法理解交换会如何损害zk和kafka。

Answer 1

交换可能会导致性能以及稳定性问题。在您的示例中，您不希望Linux内核“错误/偶然地”交换您的Kafka或ZooKeeper进程。

另外，swapping may be particularly bad for JVM processes(例如Kafka和ZooKeeper)引用:

[The] JVM generally won't do a full GC cycle until it has run out of its allowed heap, so most of your heap is likely occupied by not-yet-collected garbage. Since these pages aren't being touched (because they are garbage and thus unreferenced), the OS happily swaps them out. When GC finally runs, you have a ridiculous swap storm, pulling in all these pages only to then discover that they are in fact filled with garbage and should be discarded; this can easily make your GC cycle take many minutes!

因此，建议通过将 vm.swappiness设置为 0来禁用交换，尽管对于某些操作系统(如RHEL 6.5)，它实际上应该是 1(因为在这些OS上更改了 0值的语义)。请注意，某些交换可能仍会发生。

以下链接可以进一步阐明您的问题。他们解释了为什么分别对Hadoop和Elasticsearch禁用交换，并且出于同样的原因，您应该对Kafka和ZooKeeper禁用交换:

Hadoop:Adam Kawa的Two memory-related issues on the Apache Hadoop cluster (memory swapping and the OOM killer)；在撰写本文时，他在Spotify的Hadoop基础架构团队中工作。

Elasticsearch:Why to disable swapping for machines running Elasticsearch。