Python requests-html 与 Tor

Question

要求是匿名报废或者调用一定次数后换ip。我用 https://github.com/kennethreitz/requests-html模块来解析 HTML，但出现以下错误，

socks.SOCKS5Error: 0x01: General SOCKS server failure

代码

import socks
import socket
import requests_html

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, addr='127.0.0.1', port=int('9150'))
socket.socket = socks.socksocket

session = requests_html.HTMLSession()
r = session.get('http://icanhazip.com')
r.html.render(sleep=5)
print(r.html.text)

但它与 requests 模块一起工作得很好，

import socks
import socket
import requests

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, addr='127.0.0.1', port=int('9150'))
socket.socket = socks.socksocket

print(requests.get("http://icanhazip.com").text)

如果您能帮助解决 requests-html 模块的问题，我们将不胜感激。

Answer 1

尝试:

session = requests_html.HTMLSession(browser_args=["--no-sandbox","--proxy-server=127.0.0.1:9150"])

取决于您的代理如何设置以使用 tor，但这对我有用!