python - 如何解决 python dropbox 上传脚本中的超时错误？

Question

我目前正在做一个项目，用树莓派 pi3 进行延时拍摄(效果很好，拍摄照片不是问题)。

但是，我使用另一个脚本将这些照片上传到保管箱帐户，并且一切正常，直到我今天检查它，看到超时错误。

有谁知道这个问题如何解决吗？

这是错误:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 453, in wrap_socket
    cnx.do_handshake()
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1426, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1149, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connection.py", line 344, in connect
    ssl_context=context)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 344, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 456, in wrap_socket
    raise timeout('select timed out')
socket.timeout: select timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='content.dropboxapi.com', port=443): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/Desktop/time/picture/drop.py", line 16, in <module>
    response = db.files_upload(f.read(), dname)
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/base.py", line 2293, in files_upload
    f,
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 274, in request
    timeout=timeout)
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 365, in request_json_string_with_retry
    timeout=timeout)
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 449, in request_json_string
    timeout=timeout,
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='content.dropboxapi.com', port=443): Read timed out. (read timeout=30)
>>> 

这是在 Dropbox 上上传的代码的样子:

import dropbox
from time import sleep


db = dropbox.Dropbox('apikeyplaceholder')


for i in range(4500): ##4500

    name = 'image{0:04d}.jpg'

    fname = '/home/pi/Desktop/time/picture/'+name.format(i)
    dname = '/'+name.format(i)

    f = open(fname, 'rb')
    response = db.files_upload(f.read(), dname)
    print ('uploaded:', response)

    sleep(585)

f.close()

<小时/>

新错误:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 317, in _send_until_done
    return self.connection.send(data)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1256, in send
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1151, in _raise_ssl_error
    raise WantWriteError()
OpenSSL.SSL.WantWriteError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 936, in _send_output
    self.send(message_body)
  File "/usr/lib/python3.5/http/client.py", line 908, in send
    self.sock.sendall(data)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 328, in sendall
    sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE])
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 320, in _send_until_done
    raise timeout()
socket.timeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 936, in _send_output
    self.send(message_body)
  File "/usr/lib/python3.5/http/client.py", line 908, in send
    self.sock.sendall(data)
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 328, in sendall
    sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE])
  File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 320, in _send_until_done
    raise timeout()
urllib3.exceptions.ProtocolError: ('Connection aborted.', timeout())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/Desktop/time/bild/drop.py", line 20, in <module>
    response = upload(f, dname)
  File "/usr/local/lib/python3.5/dist-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/home/pi/Desktop/time/bild/drop.py", line 11, in upload
    return db.files_upload(f.read(), dname)
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/base.py", line 2293, in files_upload
    f,
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 274, in request
    timeout=timeout)
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 365, in request_json_string_with_retry
    timeout=timeout)
  File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 449, in request_json_string
    timeout=timeout,
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', timeout())

Answer 1

在实现处理超时的代码之前，您首先需要考虑如何处理它。常见的策略如下:

• 尝试重新上传一定次数，可能会加上 exponential backoff (例如使用 this library )
• 如果失败，请跳过当前图像或完全中止该过程。

根据您的设置，您可能希望将上传代码放入单独的线程中，以便上传过程中的延迟不会干扰拍摄照片。

使用上面链接的 backoff 库的简单示例(未经测试):

from time import sleep

import backoff
import dropbox
import requests

db = dropbox.Dropbox('apikeyplaceholder')

@backoff.on_exception(backoff.expo, requests.exceptions.ReadTimeout)
def upload(f, name):
    return db.files_upload(f.read(), dname)

NAME_PATTERN = '/image{0:04d}.jpg'
for i in range(4500):
    name = NAME_PATTERN.format(i)
    fname = '/home/pi/Desktop/time/picture' + name
    with open(fname, 'rb') as f:
        response = upload(f, name)
    print ('uploaded:', response)
    sleep(585)