twitter - requests.exceptions.ChunkedEncodingError : ('Connection broken: IncompleteRead(0 bytes read, 512 more expected)' , IncompleteRead

Question

我想编写一个程序来从 Twitter 获取推文，然后进行情感分析。我编写了以下代码，即使在导入所有必要的库后也出现错误。我对数据科学比较陌生，所以请帮助我。
我无法理解此错误的原因:

class TwitterClient(object):


def __init__(self):

    # keys and tokens from the Twitter Dev Console
    consumer_key = 'XXXXXXXXX'
    consumer_secret = 'XXXXXXXXX'
    access_token = 'XXXXXXXXX'
    access_token_secret = 'XXXXXXXXX'
    api = Api(consumer_key, consumer_secret, access_token, access_token_secret)

    def preprocess(tweet, ascii=True, ignore_rt_char=True, ignore_url=True, ignore_mention=True, ignore_hashtag=True,letter_only=True, remove_stopwords=True, min_tweet_len=3):
        sword = stopwords.words('english')

        if ascii:  # maybe remove lines with ANY non-ascii character
            for c in tweet:
                if not (0 < ord(c) < 127):
                    return ''

        tokens = tweet.lower().split()  # to lower, split
        res = []

        for token in tokens:
            if remove_stopwords and token in sword: # ignore stopword
                continue
            if ignore_rt_char and token == 'rt': # ignore 'retweet' symbol
                continue
            if ignore_url and token.startswith('https:'): # ignore url
                continue
            if ignore_mention and token.startswith('@'): # ignore mentions
                continue
            if ignore_hashtag and token.startswith('#'): # ignore hashtags
                continue
            if letter_only: # ignore digits
                if not token.isalpha():
                    continue
            elif token.isdigit(): # otherwise unify digits
                token = '<num>'

            res += token, # append token

        if min_tweet_len and len(res) < min_tweet_len: # ignore tweets few than n tokens
            return ''
        else:
            return ' '.join(res)

    for line in api.GetStreamSample():            
        if 'text' in line and line['lang'] == u'en': # step 1
            text = line['text'].encode('utf-8').replace('\n', ' ') # step 2
            p_t = preprocess(text)

    # attempt authentication
    try:
        # create OAuthHandler object
        self.auth = OAuthHandler(consumer_key, consumer_secret)
        # set access token and secret
        self.auth.set_access_token(access_token, access_token_secret)
        # create tweepy API object to fetch tweets
        self.api = tweepy.API(self.auth)
    except:
        print("Error: Authentication Failed")

假设所有必需的库都已导入。错误在第 69 行。

for line in api.GetStreamSample():            
    if 'text' in line and line['lang'] == u'en': # step 1
        text = line['text'].encode('utf-8').replace('\n', ' ') # step 2
        p_t = preprocess(text)

我尝试在互联网上检查错误的原因，但找不到任何解决方案。

错误是:

requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read, 512 more expected)', IncompleteRead(0 bytes read, 512 more expected))

我正在使用 Python 2.7 并请求最新版本 2.14。

Answer 1

如果在发出请求时将 stream 设置为 True，则除非您消耗所有数据或调用 Response.close，否则 Requests 无法将连接释放回池。这可能会导致连接效率低下。如果您在使用 stream=True 时发现自己部分读取了请求主体(或根本不读取它们)，则应该在 with 语句中发出请求以确保它始终关闭:

with requests.get('http://httpbin.org/get', stream=True) as r:
    # Do things with the response here.