Python unpickle 一个对象里面有一个类实例

Question

我正在使用 python 请求库并尝试保持 session 。

由于我的主机上有多个 IP，我创建了以下方法以使 session 绑定(bind)到特定 IP。

class SourceAddressAdapter(HTTPAdapter):
    def __init__(self, source_address, **kwargs):
        self.source_address = source_address
        super(SourceAddressAdapter, self).__init__(**kwargs)

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                   maxsize=maxsize,
                                   block=block,
                                   source_address=self.source_address)

下面的代码片段用于调用这个类:

r = requests.Session()
r.mount('http://', SourceAddressAdapter((self.ip,0)))
r.mount('https://', SourceAddressAdapter((self.ip,0)))

在这个适配器上挂载http和https协议(protocol)后，我使用pickle将对象持久化到redis中，如下所示:

session = pickle.dumps(r)
redis.hset('sessions',id,session)

当我试图解开 session 对象时出现问题:

s=redis.hget('sessions', id)
pickle.loads(s)


Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1217, in load_build
    setstate(state)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 114, in __setstate__
    block=self._pool_block)
  File "network_driver.py", line 158, in init_poolmanager
    source_address=self.source_address)
AttributeError: 'SourceAddressAdapter' object has no attribute 'source_address'

它提示 SourceAddressAdapter 没有 source_address 属性。在我将此类 SourceAddressAdapter 添加到我的 session 之前，序列化运行良好。

所以我猜这是自定义类 pickling/unpickling 的问题。

---

更新:

它在我将 __getstate__ 和 __setstate__ 方法添加到 SourceAddressAdapter 后起作用

def __getstate__(self):
    # it calls HTTPAdapter's __getstate__()
    state = super(SourceAddressAdapter, self).__getstate__() 
    state['source_address'] = self.source_address
    return state

def __setstate__(self,state):
    self.source_address = state['source_address']
    # Call HTTPAdapter's __setstate__ function to pack the attributes in parent class 
    super(SourceAddressAdapter, self).__setstate__(state)

Answer 1

我认为问题在于 HTTPAdapter 类定义了一个 __setstate__方法。这个函数在 unpickling 时调用，并将实例恢复到 pickled 状态。但是，HTTPAdapter 对您的 source_address 属性一无所知，因此该属性不会恢复(或者甚至可能一开始就不会被 pickle )。

要解决此问题，您需要覆盖 __setstate__ 函数，有点像这样:

def __setstate__(self, state):
    self.source_address= state['source_address'] # do this before calling __setstate__
    HTTPAdapter.__setstate__(self, state)

并且，如前所述，您可能还必须覆盖 __getstate__ 函数，以便对 source_address 进行 pickle。