Python:requests.exceptions.ConnectionError。 url 超出了最大重试次数

Question

这是脚本:

import requests
import json
import urlparse
from requests.adapters import HTTPAdapter

s = requests.Session()
s.mount('http://', HTTPAdapter(max_retries=1))

with open('proxies.txt') as proxies:
    for line in proxies:
        proxy=json.loads(line)

    with open('urls.txt') as urls:
        for line in urls:

            url=line.rstrip()
            data=requests.get(url, proxies=proxy)
            data1=data.content
            print data1
            print {'http': line}

如您所见，它试图通过代理列表访问 url 列表。这是 urls.txt 文件:

http://api.exip.org/?call=ip

这里是 proxies.txt 文件:

{"http":"http://107.17.92.18:8080"}

我在 www.hidemyass.com 获得了这个代理。它可能是一个糟糕的代理吗？我已经尝试了几次，这就是结果。注意:如果您尝试复制此内容，您可能需要在 hidemyass.com 上将代理更新为最近的代理。他们似乎最终停止工作。这是完整的错误和回溯:

Traceback (most recent call last):
  File "test.py", line 17, in <module>
    data=requests.get(url, proxies=proxy)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 454, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 144, in resolve_redirects
    allow_redirects=False,
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 327, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host=u'219.231.143.96', port=18186): Max retries exceeded with url: http://www.google.com/ (Caused by <class 'httplib.BadStatusLine'>: '')

Answer 1

查看您提供的堆栈跟踪，您的错误是由 httplib.BadStatusLine 异常引起的，根据 docs , 是:

Raised if a server responds with a HTTP status code that we don’t understand.

换句话说，代理服务器返回的东西(如果返回的话)不能被执行实际请求的 httplib 解析。

根据我在(编写)http 代理方面的经验，我可以说某些实现可能不会太严格地遵循规范(http 上的 rfc 规范实际上并不容易阅读)，或者使用 hack 来修复在其实现中存在缺陷的旧浏览器。

所以，回答这个问题:

Could it be a bad proxy?

...我会说 - 这是可能的。唯一确定的方法是查看代理服务器返回的内容。

尝试使用调试器或抓包嗅探器(例如 Wireshark 或 Network Monitor )对其进行调试，以分析网络中发生的情况。了解代理服务器返回的确切信息应该可以为您提供解决此问题的关键。