Python 2.7 - 为大文件下载输出命令行百分比条时遇到问题(我想要像 : 0%. ..25%...50%...75%...100% 这样的输出)？

Question

如果这看起来是一个基本问题，我深表歉意，在过去的一个小时里，我一直在努力弄清楚如何做到这一点，但没有任何进展。我在 Python 中使用这种方法来下载大 (10GB+) 文件:

def downloadChunks(url, destination):
    """Helper to download large files
       the file will be downloaded
       in chunks and print out how much remains
    modified slightly from: https://gist.github.com/gourneau/1430932
    """

    baseFile = os.path.basename(url)

    #move the file to a more uniq path
    os.umask(0002)
    try:
        file = os.path.join(destination,baseFile)

        req = urllib2.urlopen(url)
        total_size = int(req.info().getheader('Content-Length').strip())
        downloaded = 0
        CHUNK = 256 * 10240 # 256 * 10240 == 2560 KB
        with open(file, 'wb') as fp:
            while True:
                chunk = req.read(CHUNK)
                downloaded += len(chunk)
                #print math.floor( (downloaded / total_size) * 100 ),
        if math.floor( (downloaded / total_size) * 100 ) % 100000000 ==0:
            #print "downloaded ="+str(downloaded)+" bytes"
            #print "total_size ="+str(total_size)+" bytes ("+str( (float(downloaded) / float (total_size)) )+"%)"
            #print ""

                if not chunk: break
                fp.write(chunk)

    except urllib2.HTTPError, e:
        print "HTTP Error:",e.code , url
        return False
    except urllib2.URLError, e:
        print "URL Error:",e.reason , url
        return False

    return file

问题:

当这个方法运行时，我基本上想要这样的输出:0%...25%...50%...75%...100% 或 10%...20%... 等等(因为它们是大文件，所以我希望可以通过常数调整粒度。)

我知道使用 print(message,end="") 但我一直在使用算法时遇到问题。

我尝试过的:

你可以看到我过去的一些实验被注释掉了，他们给出了这个输出，这对于一个日志文件来说显然太长了。

downloaded =2621440 bytes
total_size =611886812 bytes (0.00428419104414%)

downloaded =5242880 bytes
total_size =611886812 bytes (0.00856838208829%)

downloaded =7864320 bytes
total_size =611886812 bytes (0.0128525731324%)

downloaded =10485760 bytes
total_size =611886812 bytes (0.0171367641766%)

downloaded =13107200 bytes
total_size =611886812 bytes (0.0214209552207%)

downloaded =15728640 bytes
total_size =611886812 bytes (0.0257051462649%)

downloaded =18350080 bytes
total_size =611886812 bytes (0.029989337309%)

downloaded =20971520 bytes
total_size =611886812 bytes (0.0342735283531%)

downloaded =23592960 bytes
total_size =611886812 bytes (0.0385577193973%)

downloaded =26214400 bytes
total_size =611886812 bytes (0.0428419104414%)

downloaded =28835840 bytes
total_size =611886812 bytes (0.0471261014856%)

downloaded =31457280 bytes
total_size =611886812 bytes (0.0514102925297%)

downloaded =34078720 bytes
total_size =611886812 bytes (0.0556944835739%)

downloaded =36700160 bytes
total_size =611886812 bytes (0.059978674618%)

只是想知道是否有人可以帮助我做到这一点？非常感谢。

编辑:

我也写了这个小程序，它给出了错误的输出:

程序:

#!/usr/bin/python
import os
import math
scale = 100.0
CHUNK_SIZE=77   
FILE_SIZE_TOTAL =611886812
count =0.0

printnum = 25
numIncrememnts = math.floor( scale /printnum)
incrementsize = FILE_SIZE_TOTAL / numIncrememnts

output =0


while True:
        if math.floor(count) % math.floor(incrementsize) ==0:
                print str(output)+"%"
                output= output+printnum
        count = count+CHUNK_SIZE
        if count >= FILE_SIZE_TOTAL:
                break

输出:

khennessy@organization-9758:~/Desktop/VM_deployer$ ./test3.py 
0%
khennessy@organization-9758:~/Desktop/VM_deployer$ 

当 CHUNK_SIZE ==1 时它工作正常。这是它的输出:

khennessy@organization-9758:~/Desktop/VM_deployer$ ./test3.py 
0%
25%
50%
75%
khennessy@organization-9758:~/Desktop/VM_deployer$ 

编辑 - 我的最终答案:

代码:

def downloadChunks(url, destination):
    """Helper to download large files
       the file will be downloaded
       in chunks and print out how much remains
    modified slightly from: https://gist.github.com/gourneau/1430932
    """

    baseFile = os.path.basename(url)

    #move the file to a more uniq path
    os.umask(0002)
    try:
        file = os.path.join(destination,baseFile)

        req = urllib2.urlopen(url)
        total_size = int(req.info().getheader('Content-Length').strip())
        downloaded = 0
        CHUNK = 256 * 10240 # 256 * 10240 == 2560 KB
    val = -1.0
    savedval=0.0
        with open(file, 'wb') as fp:
            while True:
                chunk = req.read(CHUNK)
                downloaded += len(chunk)
        savedval = val      
        val = 5 * (20 * downloaded / total_size)
        if val != savedval:
            printInfo(str(val)+"% downloaded. ("+str(downloaded)+" bytes / "+str(total_size)+" bytes)") 
                if not chunk: break
                fp.write(chunk)

    except urllib2.HTTPError, e:
        print "HTTP Error:",e.code , url
        return False
    except urllib2.URLError, e:
        print "URL Error:",e.reason , url
        return False

    return file

输出:

[INFO] - 0% downloaded. (2621440 bytes / 611886812 bytes)
[INFO] - 5% downloaded. (31457280 bytes / 611886812 bytes)
[INFO] - 10% downloaded. (62914560 bytes / 611886812 bytes)

感谢 JuniorCompressor!

Answer 1

您似乎没有将进度乘以 100 来获得百分比。

你也可以像这样四舍五入这个百分比:

print "total_size =%d bytes (%.0f%%)" % (total_size, 100. * downloaded / total_size)

如果您希望百分比是 25 的倍数，您可以使用以下代码:

print "total_size =%d bytes (%d%%)" % (total_size, 25 * (4 * downloaded / total_size))

4 * downloaded/total_size 因为整数除法可以有 0, 1, 2, 3, 4 作为可能的值，具体取决于进度所属的范围 ( 0%-25%、25%-50%、50%-75%、75%-100%)。

同样，您使用 10 * (10 * downloaded/total_size) 得到 10 的倍数。