mongodb - MongoWaitQueueFullException : The wait queue for acquiring a connection to server is full

Question

有时在插入一小堆不同的文档(同步)时，我会收到以下异常(请参阅下面的完整堆栈跟踪):

MongoDB.Driver.MongoWaitQueueFullException: The wait queue for acquiring a connection to server xyz.mongolab.com:54128 is full.

我在所有存储库之间使用单例 MongoDatabase 实例(因此是单个连接)。本质上，我正在做这样的事情(每个集合中不超过 20 个文档):

Context.Collection<ClientDocument>("clients").InsertMany(clients);
Context.Collection<VendorDocument>("vendors").InsertMany(vendors);
Context.Collection<SaleDocument>("sales").InsertOne(sale);

下面是单例上下文:

public class MongoContext
{
    public IMongoDatabase Database { get; }

    public MongoContext(IOptions<MongoSettings> settings)
    {
        var url = MongoUrl.Create(settings.Value.EndpointUri);

        var client = new MongoClient(new MongoClientSettings()
        {
            Server = url.Server
        });

        Database = client.GetDatabase(url.DatabaseName);
    }

    public IMongoCollection<TDocument> Collection<TDocument>(string collection)
        where TDocument : IDocument
    {
        return Database.GetCollection<TDocument>(collection);
    }
}

在 MongoDB 的 Jira (https://jira.mongodb.org/browse/CSHARP-1144) 上提交了类似的内容，但这些案例处理的是大量的批量插入(并且通常是异步的)。

我认为没有必要通过这么小的插入来增加 MaxConnectionPoolSize 或 WaitQueueSize。

这可能是什么原因？

我正在使用托管在 mLabs 中的 MongoDB 3.0.7。我们的应用程序托管在 Azure 中(作为 Web 应用程序)，我使用的是 C# 2.2.3 SDK。

MongoDB.Driver.MongoWaitQueueFullException: The wait queue for acquiring a connection to server xyz.mongolab.com:54128 is full. at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.AcquireConnectionHelper.CheckingOutConnection() at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.AcquireConnection(CancellationToken cancellationToken) at MongoDB.Driver.Core.Servers.ClusterableServer.GetChannel(CancellationToken cancellationToken) at MongoDB.Driver.Core.Bindings.ServerChannelSource.GetChannel(CancellationToken cancellationToken) at MongoDB.Driver.Core.Bindings.ChannelSourceHandle.GetChannel(CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation.Execute(IWriteBinding binding, CancellationToken cancellationToken) at MongoDB.Driver.OperationExecutor.ExecuteWriteOperation[TResult](IWriteBinding binding, IWriteOperation'1 operation, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl'1.ExecuteWriteOperation[TResult](IWriteOperation`1 operation, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl'1.BulkWrite(IEnumerable'1 requests, BulkWriteOptions options, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionBase'1.InsertOne(TDocument document, InsertOneOptions options, CancellationToken cancellationToken)

编辑:

如果我将 MaxConnectionPoolSize 设置为 500 并将 WaitQueueSize 设置为 2000，则会出现以下异常:

MongoConnectionException: An exception occurred while opening a connection to the server. ---> System.Net.Sockets.SocketException: An attempt was made to access a socket in a way forbidden by its access permissions 191.235.xxx.xxx:54128

实例化MongoClient:

var client = new MongoClient(new MongoClientSettings()
{
    Server = url.Server,
    Credentials = credentials,
    MaxConnectionPoolSize = 500,
    WaitQueueSize = 2000
});

我最初提出了这个问题 here .这导致我试图弄清楚为什么我有这么多的联系。这导致了this post (质疑 Insert/InsertBulk 是否可能是一个原因)。无论如何，我仍然需要修复原来的 MongoWaitQueueFullException 问题。

Answer 1

解决问题的长期方法可能是引入节流机制，以确保不超过最大连接数。幸运的是，使用 Semaphore 很容易实现。

public class ConnectionThrottlingPipeline : IConnectionThrottlingPipeline
{
    private readonly Semaphore openConnectionSemaphore;

    public ConnectionThrottlingPipeline( IMongoClient client )
    {
        //Only grabbing half the available connections to hedge against collisions.
        //If you send every operation through here
        //you should be able to use the entire connection pool.
        openConnectionSemaphore = new Semaphore( client.Settings.MaxConnectionPoolSize / 2,
            client.Settings.MaxConnectionPoolSize / 2 );
    }

    public async Task<T> AddRequest<T>( Task<T> task )
    {
        openConnectionSemaphore.WaitOne();
        try
        {
            var result = await task;
            return result;
        }
        finally
        {
            openConnectionSemaphore.Release();
        }
    }
}

如果您通过此限制管道将所有请求发送到数据库，则永远不应达到限制。

在您的情况下，通过管道发送操作可能看起来像这样(您必须做出的重大改变是使您的数据库调用异步):

await connectionThrottlingPipeline.AddRequest( 
   Context.Collection<ClientDocument>("clients").InsertManyAsync(clients))