Python - cql - Cassandra 1.2 - 读取时 rpc 超时

Question

我有一个使用 Cassandra 1.2 集群的 Python 应用程序。该集群有 7 个使用虚拟节点的物理节点，其中 1 个键空间的复制因子为 3，另一个键空间的复制因子为 1。该应用程序使用 cql 库连接到 Cassandra 并运行查询。问题是我在尝试在数据库上运行选择时开始出现错误，并且出现此错误:

Request did not complete within rpc_timeout

当我检查集群的状态时，我可以看到我的一个节点的 cpu 使用率超过 100%，并检查 Cassandra system.log 我可以看到这个一直弹出:

 INFO [ScheduledTasks:1] 2013-06-07 02:02:01,640 StorageService.java (line 3565) Unable to reduce heap usage since there are no dirty column families
 INFO [ScheduledTasks:1] 2013-06-07 02:02:02,642 GCInspector.java (line 119) GC for ConcurrentMarkSweep: 630 ms for 1 collections, 948849672 used; max is 958398464
 WARN [ScheduledTasks:1] 2013-06-07 02:02:02,643 GCInspector.java (line 142) Heap is 0.9900367202591844 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
 INFO [ScheduledTasks:1] 2013-06-07 02:02:02,685 StorageService.java (line 3565) Unable to reduce heap usage since there are no dirty column families
 INFO [ScheduledTasks:1] 2013-06-07 02:02:04,224 GCInspector.java (line 119) GC for ConcurrentMarkSweep: 1222 ms for 2 collections, 931216176 used; max is 958398464
 WARN [ScheduledTasks:1] 2013-06-07 02:02:04,224 GCInspector.java (line 142) Heap is 0.9716378009554072 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
 INFO [ScheduledTasks:1] 2013-06-07 02:02:04,225 StorageService.java (line 3565) Unable to reduce heap usage since there are no dirty column families
 INFO [ScheduledTasks:1] 2013-06-07 02:02:05,226 GCInspector.java (line 119) GC for ConcurrentMarkSweep: 709 ms for 1 collections, 942735576 used; max is 958398464
 WARN [ScheduledTasks:1] 2013-06-07 02:02:05,227 GCInspector.java (line 142) Heap is 0.9836572275641711 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
 INFO [ScheduledTasks:1] 2013-06-07 02:02:05,229 StorageService.java (line 3565) Unable to reduce heap usage since there are no dirty column families
 INFO [ScheduledTasks:1] 2013-06-07 02:02:06,946 GCInspector.java (line 119) GC for ConcurrentMarkSweep: 1271 ms for 2 collections, 939532792 used; max is 958398464
 WARN [ScheduledTasks:1] 2013-06-07 02:02:06,946 GCInspector.java (line 142) Heap is 0.980315419203343 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically

关于如何解决这个问题有什么想法吗？

提前致谢!

Answer 1

看起来 Cassandra JVM 堆大小可能太小，只有 1Gb:

max is 958398464

我建议将堆增加到至少 2Gb，假设您的节点上有可用内存。

请参阅 cassandra-env.sh 以查看 JVM 堆分配是如何计算的或如何将其手动设置为特定值。