python - Python 线程池的 CPU 负载

Question

典型的 Python 线程池将具有如下结构:

def run(self):
    while True:
        z=self.some_task_queue.get() 
        do_work(z)

所以看来任务队列是有持续监控的。这种持续监控任务队列的 CPU 密集程度如何？

能不能介绍一些比较好 sleep (几毫秒)时间来降低 CPU 负载？通过这种方式，可以在一段时间内停止对任务队列的监视所有线程都很忙，减少了 CPU 负载。

Answer 1

在我的机器上 .get() 上有 1000 个线程被阻塞时，CPU 负载为 0.0%:

#!/usr/bin/env python
from __future__ import print_function
import os
import time
from threading import Thread
from Queue import Queue

try: import psutil # pip install psutil
except ImportError:
    psutil = None

def f(queue):
    while True:
        item = queue.get() # block until an item is available
        print("got %s" % (item,))
        break # end thread

# create threads
q = Queue()
threads = [Thread(target=f, args=(q,)) for _ in xrange(1000)]

# starts them
for t in threads:
    t.daemon = True # die with the program
    t.start()


# show cpu load while the threads are blocked on `queue.get()`
if psutil is None:
    print('Observe cpu load yourself (or install psutil and rerun the script)')
    time.sleep(10) # observe cpu load
else:
    p = psutil.Process(os.getpid())
    for _ in xrange(10):
        print("cpu %s%%" % (p.get_cpu_percent(interval=0),))
        time.sleep(1)


# finish threads
for i in range(len(threads)):
    q.put_nowait(i) #note: queue is unlimited so there is no reason to wait

for t in threads: t.join() # wait for completion
print('done')