mongodb - pymongo 中的默认 batchSize 是多少？

Question

我正在使用 pymongo 在一次查询中获取大约 2M 文档，每个文档仅包含三个字符串字段。查询只是一个简单的 find()，没有任何 limit() 或 batchSize()。

在遍历光标时，我注意到脚本在处理了大约 25k 个文档后等待了大约 30~40 秒。

所以我想知道 mongo 是否会在一批中返回所有 2M 结果？ pymongo 中默认的 batchSize() 是什么？

Answer 1

MongoDB 中的游标默认返回最多 101 个文档或足以让您达到 1 MB。在弹出到 4MB 之后通过光标进行迭代的调用。返回的文档数量取决于您的文档大小:

Cursor Batches

The MongoDB server returns the query results in batches. Batch size will not exceed the maximum BSON document size. For most queries, the first batch returns 101 documents or just enough documents to exceed 1 megabyte. Subsequent batch size is 4 megabytes. To override the default size of the batch, see batchSize() and limit().

For queries that include a sort operation without an index, the server must load all the documents in memory to perform the sort and will return all documents in the first batch.

As you iterate through the cursor and reach the end of the returned batch, if there are more results, cursor.next() will perform a getmore operation to retrieve the next batch.

http://docs.mongodb.org/manual/core/cursors/

您可以在光标上使用 pymongo 中的 batch_size() 方法来覆盖默认值 - 但是它不会超过 16 MB(最大 BSON 文档大小):

batch_size(batch_size)

Limits the number of documents returned in one batch. Each batch requires a round trip to the server. It can be adjusted to optimize performance and limit data transfer.

Note

batch_size can not override MongoDB’s internal limits on the amount of data it will return to the client in a single batch (i.e if you set batch size to 1,000,000,000, MongoDB will currently only return 4-16MB of results per batch).

Raises TypeError if batch_size is not an integer. Raises ValueError if batch_size is less than 0. Raises InvalidOperation if this Cursor has already been used. The last batch_size applied to this cursor takes precedence. Parameters :

batch_size: The size of each batch of results requested.

http://api.mongodb.org/python/current/api/pymongo/cursor.html