python - python multiprocessing 出现奇怪的进程克隆

Question

我遇到了一个非常奇怪的 Python 行为。看起来当我启动使用多处理的并行程序时，在主进程中又生成了 2 个(生产者、消费者)，我看到有 4 个进程在运行。我觉得应该只有3个:main，Producer，Consumer。但是过了一段时间，第四个过程出现了。

我已经制作了一个最小的代码示例来重现该问题。它创建了两个使用递归计算斐波那契数的过程:

from multiprocessing import Process, Queue
import os, sys
import time
import signal

def fib(n):
    if n == 1 or n == 2:
        return 1
    result = fib(n-1) + fib(n-2)
    return result

def worker(queue, amount):
    pid = os.getpid()
    def workerProcess(a, b):
        print a, b
        print 'This is Writer(', pid, ')'
    signal.signal(signal.SIGUSR1, workerProcess)

    print 'Worker', os.getpid()
    for i in range(0, amount):
        queue.put(fib(35 - i % 4))
    queue.put('end')
    print 'Worker finished'

def writer(queue):
    pid = os.getpid()
    def writerProcess(a, b):
        print a, b
        print 'This is Writer(', pid, ')'
    signal.signal(signal.SIGUSR1, writerProcess)

    print 'Writer', os.getpid()
    working = True
    while working:
        if not queue.empty():
            value = queue.get()
            if value != 'end':
                fib(32 + value % 4)
            else:
                working = False
        else:
            time.sleep(1)
    print 'Writer finished'

def daemon():
    print 'Daemon', os.getpid()
    while True:
        time.sleep(1)

def useProcesses(amount):
    q = Queue()
    writer_process = Process(target=writer, args=(q,))
    worker_process = Process(target=worker, args=(q, amount))
    writer_process.daemon = True
    worker_process.daemon = True
    worker_process.start()
    writer_process.start()

def run(amount):
    print 'Main', os.getpid()
    pid = os.getpid()
    def killThisProcess(a, b):
        print a, b
        print 'Main killed by signal(', pid, ')'
        sys.exit(0)
    signal.signal(signal.SIGTERM, killThisProcess)
    useProcesses(amount)
    print 'Ready to exit main'
    while True:
        time.sleep(1)

def main():
    run(1000)

if __name__=='__main__':
    main()

我在输出中看到的是:

$ python python_daemon.py 
Main 13257
Ready to exit main
Worker 13258
Writer 13259

但在 htop 中我看到以下内容: 看起来 PID 为 13322 的进程实际上是一个线程。 问题是它是什么？谁产的？为什么？如果我将 SIGUSR1 发送到此 PID，我会在输出中看到:

10 <frame object at 0x7f05c14ed5d8>
This is Writer( 13258 )

此问题与:Python multiprocessing: more processes than requested 略有相关

Answer 1

线程属于队列对象。

它在内部使用一个线程来通过管道分派(dispatch)数据。

来自docs :

class multiprocessing.Queue([maxsize])

Returns a process shared queue implemented using a pipe and a few locks/semaphores. When a process first puts an item on the queue a feeder thread is started which transfers objects from a buffer into the pipe.