python - Python中Redis池中的默认连接数

Question

python - 3.7

Redis - 2.10.6

---

我正在为 Redis 使用创建一个连接池

redis_pool = redis.ConnectionPool(host=REDIS_URL, port=REDIS_PORT, decode_responses=True) 

我没有指定 max_connections。在查看 redis.ConnectionPool() 的源代码时，

def __init__(self, connection_class=Connection, max_connections=None,
             **connection_kwargs):
    """
    Create a connection pool. If max_connections is set, then this
    object raises redis.ConnectionError when the pool's limit is reached.

    By default, TCP connections are created unless connection_class is
    specified. Use redis.UnixDomainSocketConnection for unix sockets.

    Any additional keyword arguments are passed to the constructor of
    connection_class.
    """
    max_connections = max_connections or 2 ** 31
    if not isinstance(max_connections, (int, long)) or max_connections < 0:
        raise ValueError('"max_connections" must be a positive integer')

    self.connection_class = connection_class
    self.connection_kwargs = connection_kwargs
    self.max_connections = max_connections

    self.reset()

我看到 max_connections 设置为 2 ** 31 即2,147,483,648(如果未设置)。这对我来说很奇怪。

Redis 在池中维护的默认连接数是多少？最大值在200万左右。因此，这意味着我们必须为此传递我们自己的实用值(value)。

Answer 1

Redis 端不存在池，这个类实际上只是 Python 端的 self.connection_class 实例的奇特集合。

不过同意你的看法，99+% 的情况下 2**31 这个数字是不必要的大。不过不要认为这太令人担忧，因为初始化池不会创建任何连接(或为它们保留空间)。 max_connections 仅限制 _available_connections 数组，该数组在需要连接但池中没有可立即使用的空闲连接时增长。

下面是一些带有一些注释的 ConnectionPool 类。

https://github.com/andymccurdy/redis-py/blob/master/redis/connection.py#L967

def reset(self):
    self.pid = os.getpid()
    self._created_connections = 0
    self._available_connections = []  # <- starts empty
    self._in_use_connections = set()
    self._check_lock = threading.Lock()

https://github.com/andymccurdy/redis-py/blob/master/redis/connection.py#L983

def get_connection(self, command_name, *keys, **options):
    "Get a connection from the pool"
    self._checkpid()
    try:
        connection = self._available_connections.pop()
    except IndexError:
        connection = self.make_connection()  # <- make a new conn only if _available_connections is tapped
    self._in_use_connections.add(connection)
    try:
        # ensure this connection is connected to Redis
        connection.connect()
        # connections that the pool provides should be ready to send
        # a command. if not, the connection was either returned to the
        # pool before all data has been read or the socket has been
        # closed. either way, reconnect and verify everything is good.
        if not connection.is_ready_for_command():
            connection.disconnect()
            connection.connect()
            if not connection.is_ready_for_command():
                raise ConnectionError('Connection not ready')
    except:  # noqa: E722
        # release the connection back to the pool so that we don't leak it
        self.release(connection)
        raise

    return connection

https://github.com/andymccurdy/redis-py/blob/master/redis/connection.py#L1019

 def make_connection(self):
    "Create a new connection"
    if self._created_connections >= self.max_connections:  # <- where the bounding happens
        raise ConnectionError("Too many connections")
    self._created_connections += 1
    return self.connection_class(**self.connection_kwargs)

无论如何，我敢打赌选择特定值是为了将开发人员完全耗尽池的可能性降低到接近 0。请注意，连接对象非常轻量级，因此不太可能包含数千或数百万个连接对象使您的应用程序停止运行。实际上它不应该有什么不同:大多数 Redis 调用返回得如此之快，无论如何你都很难不小心同时启动数百万个调用。 (如果您是故意这样做的，您可能知道的足够多，可以根据您的确切需求调整一切。;-)