python - 如何使用 python3.4 通过 'https' 登录页面登录？

Question

我正在尝试使用 python3.4 请求、bs4 模块登录特定页面。
当我尝试使用以下代码登录时，出现 ssl 错误:

import requests

userid = "myid"
password = "mypassword"

loginURL = "https://example.com/member/login"
params = {'userid': userid, 'password':password}
res = requests.post(loginURL, params=params, verify=True)

错误:

choejeong-us-MacBook-Pro:Desktop kris$ python a.py
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 782, in _validate_conn
    conn.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 250, in connect
    ssl_version=resolved_ssl_version)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 285, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 365, in wrap_socket
    _context=self)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 583, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 810, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 588, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "a.py", line 10, in <module>
    res = requests.post(loginURL, params=params, verify=True)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 109, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 433, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

如何使用 ssl 页面登录网站？

我看了一些关于 python 请求模块和网络抓取的书籍，没有处理这个问题的书籍。

p.s 我想知道 python 是否真的足够强大以处理“所有”网页？我的意思是，我可以抓取网络中的所有站点吗？还是有一些限制？

Answer 1

回溯的最后一行告诉你原因:

requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

请求 无法验证服务器的 SSL 证书。您可以使用 verify 选项禁用证书验证:

res = requests.post(loginURL, params=params, verify=False)

这样做会带来安全隐患，因此您是否应该这样做取决于您的应用程序要求。