python - "IncompleteRead"使用 Python 检索 Twitter 数据时出错

Question

在运行此程序以使用 Python 2.7.8 检索 Twitter 数据时:

#imports
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener

#setting up the keys
consumer_key = '…………...'
consumer_secret = '………...'
access_token = '…………...'
access_secret = '……………..'

class TweetListener(StreamListener):
# A listener handles tweets are the received from the stream.
#This is a basic listener that just prints received tweets to standard output

def on_data(self, data):
    print (data)
    return True

def on_error(self, status):
    print (status)

#printing all the tweets to the standard output
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)



stream = Stream(auth, TweetListener())

t = u"سوريا"
stream.filter(track=[t])

运行此程序 5 小时后，我收到此错误消息:

Traceback (most recent call last):
  File "/Users/Mona/Desktop/twitter.py", line 32, in <module>
    stream.filter(track=[t])
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 316, in filter
    self._start(async)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 237, in _start
    self._run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 173, in _run
    self._read_loop(resp)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 225, in _read_loop
    next_status_obj = resp.read( int(delimited_string) )
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 543, in read
    return self._read_chunked(amt)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 612, in _read_chunked
    value.append(self._safe_read(chunk_left))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 660, in _safe_read
    raise IncompleteRead(''.join(s), amt)
IncompleteRead: IncompleteRead(0 bytes read, 976 more expected)
>>> 

其实我不知道如何处理这个问题!!!

Answer 1

您应该使用 stall_warnings 检查您是否未能足够快地处理推文参数。

stream.filter(track=[t], stall_warnings=True)

这些消息由 Tweepy 处理(查看实现 here )并且会在您落后时通知您。落后意味着您无法像 Twitter API 向您发送的那样快速处理推文。来自 Twitter 文档:

Setting this parameter to the string true will cause periodic messages to be delivered if the client is in danger of being disconnected. These messages are only sent when the client is falling behind, and will occur at a maximum rate of about once every 5 minutes.

理论上，您应该收到 disconnect message在这种情况下来自 API。然而，情况并非总是如此:

The streaming API will attempt to deliver a message indicating why a stream was closed. Note that if the disconnect was due to network issues or a client reading too slowly, it is possible that this message will not be received.

IncompleteRead 也可能是由于临时网络问题造成的，并且可能永远不会再发生。如果它在大约 5 小时后重复发生，那么落后是一个不错的选择。