python-requests - 由于 urllib3 v1.8.3 已添加 "socket_options"功能，如何在 python-requests 库中指定 "socket_options"？

Question

urllib3 V1.8.3 增加了“socket_options”特性。对我来说，我想在请求库中指定 (socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)。但我不知道怎么做。
Here 是 urllib3 的版本变更日志。
以下是 urllib3 中新功能的示例:

    def test_socket_options(self):
    """Test that connections accept socket options."""
    # This test needs to be here in order to be run. socket.create_connection actually tries to
    # connect to the host provided so we need a dummyserver to be running.
    pool = HTTPConnectionPool(self.host, self.port, socket_options=[
        (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
    ])
    s = pool._new_conn()._new_conn()  # Get the socket
    using_keepalive = s.getsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE) > 0
    self.assertTrue(using_keepalive)
    s.close()

如果1.8.3版本合并到新的请求库中，如何指定参数。
任何答案将不胜感激。

这是我的代码:

import requests
import socket

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager

class SourceAddressAdapter(HTTPAdapter):
    def __init__(self, source_address, **kwargs):
        self.source_address = source_address
        super(SourceAddressAdapter, self).__init__(**kwargs)
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       socket_options=[(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)],
                                       source_address=self.source_address,
                                       )
http_session = requests.Session()
http_session.mount('http://', SourceAddressAdapter(('0.0.0.0', 1238)))
print http_session.get('http://10.0.10.7')

这是错误:

    Traceback (most recent call last):
  File "./tt.py", line 27, in <module>
    print http_session.get('http://10.0.10.7')
  File "/usr/local/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/sessions.py", line 468, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests-2.3.0-py2.7.egg/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='10.0.10.7', port=80): Max retries exceeded with url: / (Caused by <class 'socket.error'>: [Errno 98] Address already in use)

================================================== ==============================

我曾尝试使用 urllib3 v1.9 来设置 (socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)，但失败了。 time_wait 状态的源端口不能复用的原因可能是“(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)”应该在“socket.bind”之前配置。因此，即使我在“HTTPConnectionPool”中设置了 REUSEADDR 参数，也无法使用处于 time_wait 状态的 src 端口。好难过..

这是我的尝试:

#!/usr/bin/python
import socket
from urllib3 import (
    encode_multipart_formdata,
    HTTPConnectionPool,
    )

pool = HTTPConnectionPool('10.0.10.7', 80, 
                          socket_options=[(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)],
                          source_address=('10.0.10.11',1239))
r=pool.request('GET','/')

这是错误:

    Traceback (most recent call last):
  File "./aa.py", line 10, in <module>
    r=pool.request('GET','/')
  File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 68, in request
  File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 81, in request_encode_url
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 579, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 579, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 579, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 559, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/util/retry.py", line 265, in increment
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.0.10.7', port=80): Max retries exceeded with url: / (Caused by ProtocolError('Connection aborted.', error(98, 'Address already in use')))

Answer 1

假设您已将 urllib3 正确合并到请求中并且没有任何问题，您将使用传输适配器:

from requests.adapters import HTTPAdapter

class SockOpsAdapter(HTTPAdapter):
    def __init__(self, options, **kwargs):
        self.options = options

        super(SourceAddressAdapter, self).__init__(**kwargs)

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       socket_options=self.options)

然后你像这样使用它:

import requests
from requests.packages.urllib3.connection import HTTPConnection

options = HTTPConnection.default_socket_options + [
              (socket.SOL_SOCKET, socket.SO_REUSEADDR, 1),
          ]

s = requests.Session()
s.mount('http://', SockOpsAdapter(options))
s.mount('https://', SockOpsAdapter(options))