python - Redis Python Client 打开很多连接

Question

我正在使用以下代码连接到 Redis 服务器。我看到巨大的连接处于 TIME_WAIT 状态。有什么问题吗？

root@ubuntu:~$ netstat | grep :6479 | grep TIME_WAIT |wc -l
9061
root@ubuntu:~$ netstat | grep :6479 | grep ESTABLISHED |wc -l
7

我想在使用以下代码对 Redis 服务器完成操作后关闭连接。但是我遇到了这个错误。

@staticmethod
def disconnectRedisConnection(r_server):
    if r_server is not None and r_server:
        r_server.connection.disconnect()

我收到以下错误，

r_server.connection.disconnect()
AttributeError: 'Redis' object has no attribute 'connection'

对巨大的 TIME_WAIT 连接/Redis 操作完成后关闭连接有什么想法吗？代码:

import threading
from time import sleep
import time, datetime
import traceback
import CACHE_CONST
import json
import os

import MySQLdb
import redis

# Static methods to interact with the Redis cache server
class CacheUtil(object):

    # Log Errors
    @staticmethod
    def log_message(msg):
        log_file = None
        log_file = open (os.path.abspath(CACHE_CONST.REDIS_LOG_FILE), "a")
        print(msg)
        if (log_file):
            message = time.strftime("%d-%m-%Y %H:%M:%S")    
            message += " :: " + str(msg)
            log_file.write(message + "\n")

    @staticmethod
    def saveToCache(hashName, hashValue):
        r_server = CacheUtil.getRedisConnection()
        r_server.hmset(hashName, hashValue)
        CacheUtil.disconnectRedisConnection(r_server)

    @staticmethod
    def getTrackerDetailsByID(trackerId):
        trackerDetail = None
        r_server = None
        hashName = "tDetails:" + str(trackerId)
        try:
            if trackerId is not None:
                print("getTrackerDetailsByID ")
                r_server = CacheUtil.getRedisConnection()
                trackerDetail = r_server.hgetall(hashName)
            else:
                CacheUtil.log_message("getDetailsByID failed with empty trackerId ")
        except:
            CacheUtil.log_message("getDetailsByID failed, ll fetch from DB " + str(traceback.format_exc()))
        finally:
            CacheUtil.disconnectRedisConnection(r_server)
        return trackerDetail

    @staticmethod
    def getRedisConnection():
        print("Get Redis Connection on Util ")
        r_server = redis.Redis(host=CACHE_CONST.REDIS_SERVER_URL, port=CACHE_CONST.REDIS_SERVER_PORT, db=0, password=CACHE_CONST.REDIS_PASS_PHRASE, socket_connect_timeout=2, socket_timeout=2)
        return r_server;

    @staticmethod
    def disconnectRedisConnection(r_server):
        if r_server is not None and r_server:
            r_server.connection.disconnect()

Answer 1

实际上，当您调用 redis.Redis() 时，它会为您创建“客户端”，它有一个连接池，而不仅仅是一个连接。

每次你发送像 redis.set() 或其他的命令时，它会从它的连接池中检索一个连接，并使用这个连接发送这个命令并等待回复。请求完成后，它将连接放回连接池以供重用。所以你不需要自己管理连接。看看这个https://github.com/andymccurdy/redis-py了解更多信息。

就像这样:

import threading
from time import sleep
import time, datetime
import traceback
import CACHE_CONST
import json
import os

import MySQLdb
import redis
r_server = redis.Redis(host=CACHE_CONST.REDIS_SERVER_URL, port=CACHE_CONST.REDIS_SERVER_PORT, db=0, password=CACHE_CONST.REDIS_PASS_PHRASE, socket_connect_timeout=2, socket_timeout=2)

# Static methods to interact with the Redis cache server
class CacheUtil(object):

    # Log Errors
    @staticmethod
    def log_message(msg):
        log_file = None
        log_file = open (os.path.abspath(CACHE_CONST.REDIS_LOG_FILE), "a")
        print(msg)
        if (log_file):
            message = time.strftime("%d-%m-%Y %H:%M:%S")    
            message += " :: " + str(msg)
            log_file.write(message + "\n")

    @staticmethod
    def saveToCache(hashName, hashValue):
        r_server.hmset(hashName, hashValue)

    @staticmethod
    def getTrackerDetailsByID(trackerId):
        hashName = "tDetails:" + str(trackerId)
        try:
            if trackerId is not None:
                print("getTrackerDetailsByID ")
                trackerDetail = r_server.hgetall(hashName)
            else:
                CacheUtil.log_message("getDetailsByID failed with empty trackerId ")
        except:
            CacheUtil.log_message("getDetailsByID failed, ll fetch from DB " + str(traceback.format_exc()))
        return trackerDetail

更新

每次使用redis实例发送命令，都会调用这个方法:

    # COMMAND EXECUTION AND PROTOCOL PARSING
def execute_command(self, *args, **options):
    "Execute a command and return a parsed response"
    pool = self.connection_pool
    command_name = args[0]
    connection = pool.get_connection(command_name, **options)
    try:
        connection.send_command(*args)
        return self.parse_response(connection, command_name, **options)
    except (ConnectionError, TimeoutError) as e:
        connection.disconnect()
        if not connection.retry_on_timeout and isinstance(e, TimeoutError):
            raise
        connection.send_command(*args)
        return self.parse_response(connection, command_name, **options)
    finally:
        pool.release(connection)

对巨大的 TIME_WAIT 连接/Redis 操作完成后关闭连接的任何想法

这是一个关于 TCP 连接终止过程的图像。当client(Initiator)为server(Reciever)FIN发送ACK，进入Time_WAIT状态。

引自TCP/IP illustrated vol 1的话:

When TCP performs an active close and sends the final ACK, that connection must stay in the TIME_WAIT state for twice the MSL. This lets TCP resend the final ACK in case it is lost. The final ACK is resent not because the TCP retransmits ACKs (they do not consume sequence numbers and are not retransmitted by TCP), but because the other side will retransmit its FIN (which does consume a sequence number). Indeed, TCP will always retransmit FINs until it receives a final ACK.

所以会处于TIME_WAIT状态四分钟，之后会自动关闭连接。因为你经常打开新的tcp连接并关闭它，那么很多关闭的连接将处于TIME_WAIT状态。

还有一篇关于the purpose about TIME_WAIT的更详细的文章