java - 当文档不在工作集中时，MongoDB 使用更新插入和唯一索引原子性进行更新

Question

总而言之，当文档不属于工作集(不在驻留内存中)时，我们在对现有文档进行并发更新时遇到了这种奇怪的行为。

更多详情:

给定一个具有唯一索引的集合，并在给定现有文档上运行并发更新(3 个线程)并将 upsert 设置为 true 时，1 到 2 个线程引发以下异常:

Processing failed (Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$key_1  dup key: { : 1008 }'):

根据文档，我希望所有三个更新都能成功，因为我尝试更新的文档已经存在。相反，它看起来像是在尝试对少数或所有更新请求执行插入操作，并且由于唯一索引而失败的很少。

对文档重复相同的并发更新不会引发任何异常。此外，在文档上使用 find() 将其放入工作集，然后在该文档上运行并发更新也会按预期运行。此外，使用具有相同查询和设置的 findAndModify 不会出现相同的问题。

这是按预期工作还是我遗漏了什么？

设置:

-mongodb java驱动3.0.1

运行MongoDB版本“2.6.3”的-3节点副本集

查询:

BasicDBObject query = new BasicDBObject();  
query.put("docId", 123L);
collection.update (query, object, true, false);

索引:

name: docId_1
unique: true
key: {"docId":1}
background: true

已于 5 月 28 日更新，包含用于重现问题的示例代码。按如下方式在本地运行 MongoDB(请注意，测试将写入约 4 GB 的数据):./mongodb-osx-x86_64-2.6.10/bin/mongod --dbpath/tmp/mongo运行以下代码，重启数据库，注释掉“fillUpCollection(testMongoDB.col1, value, 0, 300);”，然后再次运行代码。根据机器的不同，您可能需要调整一些数字才能看到异常。

package test;

import com.mongodb.BasicDBObject;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import com.mongodb.Mongo;
import com.mongodb.MongoClient;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class TestMongoDB {
    public static final String DOC_ID = "docId";
    public static final String VALUE = "value";
    public static final String DB_NAME = "db1";
    public static final String UNIQUE = "unique";
    public static final String BACKGROUND = "background";
    private DBCollection col1;
    private DBCollection col2;

    private static DBCollection getCollection(Mongo mongo, String collectionName) {
        DBCollection col =  mongo.getDB(DB_NAME).getCollection(collectionName);
        BasicDBObject index = new BasicDBObject();
        index.append(DOC_ID, 1);
        DBObject indexOptions = new BasicDBObject();
        indexOptions.put(UNIQUE, true);
        indexOptions.put(BACKGROUND, true);
        col.createIndex(index, indexOptions);
        return col;
    }

    private static void storeDoc(String docId, DBObject doc, DBCollection dbCollection) throws IOException {
        BasicDBObject query = new BasicDBObject();
        query.put(DOC_ID, docId);
        dbCollection.update(query, doc, true, false);
        //dbCollection.findAndModify(query, null, null, false, doc, false, true);
    }

    public static void main(String[] args) throws Exception{
        final String value = new String(new char[1000000]).replace('\0', 'a');
        Mongo mongo = new MongoClient("localhost:27017");
        final TestMongoDB testMongoDB = new TestMongoDB();
        testMongoDB.col1 = getCollection(mongo, "col1");
        testMongoDB.col2 = getCollection(mongo, "col2");

        fillUpCollection(testMongoDB.col1, value, 0, 300);
        //restart Database, comment out previous line, and run again
        fillUpCollection(testMongoDB.col2, value, 0, 2000);
        updateExistingDocuments(testMongoDB, value);
    }

    private static void updateExistingDocuments(TestMongoDB testMongoDB, String value) {
        List<String> docIds = new ArrayList<String>();
        for(int i = 0; i < 10; i++) {
            docIds.add(new Random().nextInt(300) + "");
        }
        multiThreadUpdate(testMongoDB.col1, value, docIds);
    }


    private static void multiThreadUpdate(final DBCollection col, final String value, final List<String> docIds) {
        Runnable worker = new Runnable() {
            @Override
            public void run() {
                try {
                    System.out.println("Started Thread");
                    for(String id : docIds) {
                        storeDoc(id, getDbObject(value, id), col);
                    }
                } catch (Exception e) {
                    System.out.println(e);
                } finally {
                    System.out.println("Completed");
                }
            }
        };

        for(int i = 0; i < 8; i++) {
            new Thread(worker).start();
        }
    }

    private static DBObject getDbObject(String value, String docId) {
        final DBObject object2 = new BasicDBObject();
        object2.put(DOC_ID, docId);
        object2.put(VALUE, value);
        return object2;
    }

    private static void fillUpCollection(DBCollection col, String value, int from, int to) throws IOException {
        for(int i = from ; i <= to; i++) {
            storeDoc(i + "", getDbObject(value, i + ""), col);
        }
    }
}

第二次运行的示例输出:

Started Thread
Started Thread
Started Thread
Started Thread
Started Thread
Started Thread
Started Thread
Started Thread
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "290" }'
Completed
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "170" }'
Completed
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "241" }'
Completed
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "127" }'
Completed
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "120" }'
Completed
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "91" }'
Completed
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'insertDocument :: caused by :: 11000 E11000 duplicate key error index: db1.col1.$docId_1  dup key: { : "136" }'
Completed
Completed

Answer 1

这看起来像是 MongoDB 的一个已知问题，至少在 2.6 版之前是这样。他们推荐的修复方法是让您的代码在出现错误时重试更新插入。 https://jira.mongodb.org/browse/SERVER-14322