concurrency - DynamoDB 中原子计数器的可靠性

Question

我正在考虑使用Amazon DynamoDB在我的申请中，我有一个关于其 atomic counters 的问题可靠性。

我正在构建一个分布式应用程序，需要同时和一致递增/递减 Dynamo 属性中存储的计数器。我想知道 Dynamo 的原子计数器在并发水平极高的高并发环境中有多可靠(例如，平均 20k 并发命中率 - 明白了这一点，这将是近 520 亿的增量/每月递减)。

计数器应该 super 可靠，并且永远不会错过任何一次点击。有人在如此关键的环境中测试过 DynamoDB 吗？

谢谢

Answer 1

DynamoDB 通过将 key 拆分到多个服务器来获取其扩展属性。这类似于 Cassandra 和 HBase 等其他分布式数据库的扩展方式。虽然您可以增加 DynamoDB 的吞吐量，只需将数据移动到多个服务器，但现在每个服务器都可以处理总并发连接数/服务器数量。看看at their FAQ有关如何实现最大吞吐量的说明:

Q: Will I always be able to achieve my level of provisioned throughput?

Amazon DynamoDB assumes a relatively random access pattern across all primary keys. You should set up your data model so that your requests result in a fairly even distribution of traffic across primary keys. If you have a highly uneven or skewed access pattern, you may not be able to achieve your level of provisioned throughput.

When storing data, Amazon DynamoDB divides a table into multiple partitions and distributes the data based on the hash key element of the primary key. The provisioned throughput associated with a table is also divided among the partitions; each partition's throughput is managed independently based on the quota allotted to it. There is no sharing of provisioned throughput across partitions. Consequently, a table in Amazon DynamoDB is best able to meet the provisioned throughput levels if the workload is spread fairly uniformly across the hash key values. Distributing requests across hash key values distributes the requests across partitions, which helps achieve your full provisioned throughput level.

If you have an uneven workload pattern across primary keys and are unable to achieve your provisioned throughput level, you may be able to meet your throughput needs by increasing your provisioned throughput level further, which will give more throughput to each partition. However, it is recommended that you considering modifying your request pattern or your data model in order to achieve a relatively random access pattern across primary keys.

这意味着拥有一个直接递增的 key 将无法扩展，因为该 key 必须位于一台服务器上。还有其他方法可以处理此问题，例如在内存聚合中将增量刷新到 DynamoDB(尽管这可能会产生可靠性问题)或分片计数器，其中增量分布在多个键上，并通过拉动分片中的所有键来读回计数器(http://whynosql.com/scaling-distributed-counters/)。