python - 连接重置错误 : An existing connection was forcibly closed by the remote host

Question

我正在编写一个脚本来下载一组文件。我成功地完成了这个并且它工作得很好。现在我尝试添加下载进度的动态打印输出。

对于小型下载(顺便说一下，它是 .mp4 文件)，例如 5MB，进度效果很好并且文件成功关闭，从而生成完整且有效的下载 .mp4 文件。对于较大的文件，例如 250MB 及以上，它无法成功运行，我收到以下错误:

这是我的代码:

import urllib.request
import shutil
import os
import sys
import io

script_dir = os.path.dirname('C:/Users/Kenny/Desktop/')
rel_path = 'stupid_folder/video.mp4'
abs_file_path = os.path.join(script_dir, rel_path)
url = 'https://archive.org/download/SF145/SF145_512kb.mp4'
# Download the file from `url` and save it locally under `file_name`:

with urllib.request.urlopen(url) as response, open(abs_file_path, 'wb') as out_file:

    eventID = 123456

    resp = urllib.request.urlopen(url)
    length = resp.getheader('content-length')
    if length:
        length = int(length)
        blocksize = max(4096, length//100)
    else:
        blocksize = 1000000 # just made something up

    # print(length, blocksize)

    buf = io.BytesIO()
    size = 0
    while True:
        buf1 = resp.read(blocksize)
        if not buf1:
            break
        buf.write(buf1)
        size += len(buf1)
        if length:
            print('\r[{:.1f}%] Downloading: {}'.format(size/length*100, eventID), end='')#print('\rDownloading: {:.1f}%'.format(size/length*100), end='')
    print()

    shutil.copyfileobj(response, out_file)

这对小文件非常有效，但对大文件我会出错。现在，如果我注释掉 progress 指示器代码，我不会收到更大文件的错误:

with urllib.request.urlopen(url) as response, open(abs_file_path, 'wb') as out_file:

    # eventID = 123456
    # 
    # resp = urllib.request.urlopen(url)
    # length = resp.getheader('content-length')
    # if length:
    #     length = int(length)
    #     blocksize = max(4096, length//100)
    # else:
    #     blocksize = 1000000 # just made something up
    # 
    # # print(length, blocksize)
    # 
    # buf = io.BytesIO()
    # size = 0
    # while True:
    #     buf1 = resp.read(blocksize)
    #     if not buf1:
    #         break
    #     buf.write(buf1)
    #     size += len(buf1)
    #     if length:
    #         print('\r[{:.1f}%] Downloading: {}'.format(size/length*100, eventID), end='')#print('\rDownloading: {:.1f}%'.format(size/length*100), end='')
    # print()

    shutil.copyfileobj(response, out_file)

有人有什么想法吗？这是我项目的最后一部分，我真的很希望能够看到进展。再一次，这是 Python 3.5。感谢您提供的任何帮助!

Answer 1

您打开您的网址两次，一次作为response，一次作为resp。有了进度条，你就在消耗数据，所以当使用 copyfileobj 复制文件时，数据是空的(这可能是不准确的，因为它适用于小文件，但你正在做事情在这里两次，这可能是你问题的根源)

要获取进度条和有效文件，请执行以下操作:

with urllib.request.urlopen(url) as response, open(abs_file_path, 'wb') as out_file:

    eventID = 123456

    length = response.getheader('content-length')
    if length:
        length = int(length)
        blocksize = max(4096, length//100)
    else:
        blocksize = 1000000 # just made something up


    size = 0
    while True:
        buf1 = response.read(blocksize)
        if not buf1:
            break
        out_file.write(buf1)
        size += len(buf1)
        if length:
            print('\r[{:.1f}%] Downloading: {}'.format(size/length*100, eventID), end='')#print('\rDownloading: {:.1f}%'.format(size/length*100), end='')
    print()

对您的代码进行的简化:

• 只有一个urlopen，作为response
• 没有BytesIO，直接写入out_file