python - 方法内的多处理回调不起作用

Question

我正在尝试在方法内运行多重处理。但是回调似乎无法正常运行。

这是我需要实现的一个示例。

from multiprocessing import Pool
from time import sleep
import random

def sum(task, a, b): 
    sleepTime = random.randint(1, 4)
    print(task, " requires ", sleepTime, " seconds to finish")
    sleep(sleepTime)
    return a+b 

def printResult(result):
    print(result)

myPool = Pool(5)

result1 = myPool.apply_async(sum, args=("task1", 10, 20,), callback = printResult)
result2 = myPool.apply_async(sum, args=("task2", 20, 30,), callback = printResult)
result3 = myPool.apply_async(sum, args=("task3", 30, 40,), callback = printResult)
result4 = myPool.apply_async(sum, args=("task4", 40, 50,), callback = printResult)
result5 = myPool.apply_async(sum, args=("task5", 50, 60,), callback = printResult)

print("Submitted tasks to pool")

myPool.close()
myPool.join()

返回效果很好

Submitted tasks to pool
task1  requires  3  seconds to finish
task2  requires  4  seconds to finish
task3  requires  1  seconds to finish
task4  requires  4  seconds to finish
task5  requires  1  seconds to finish
110
70
30
50
90

但是，当我将多处理包装在方法内并调用该方法时，它似乎没有按预期运行。

例如

def run_mp():
    from multiprocessing import Pool
    from time import sleep
    import random

    def sum(task, a, b): 
        sleepTime = random.randint(1, 4)
        print(task, " requires ", sleepTime, " seconds to finish")
        sleep(sleepTime)
        return a+b 

    def printResult(result):
        print(result)

    myPool = Pool(5)

    result1 = myPool.apply_async(sum, args=("task1", 10, 20,), callback = printResult)
    result2 = myPool.apply_async(sum, args=("task2", 20, 30,), callback = printResult)
    result3 = myPool.apply_async(sum, args=("task3", 30, 40,), callback = printResult)
    result4 = myPool.apply_async(sum, args=("task4", 40, 50,), callback = printResult)
    result5 = myPool.apply_async(sum, args=("task5", 50, 60,), callback = printResult)

    print("Submitted tasks to pool")

    myPool.close()
    myPool.join()

当调用run_mp()时，它只返回以下内容。

Submitted tasks to pool

这不仅仅是屏幕上标准输出打印的问题。在我处理的实际问题中似乎没有正确调用回调函数。

Answer 1

简单的答案。使用多处理来pickle一个类方法是不可能的，或者至少是不容易的。有一个 multiprocessing 的分支，称为 multiprocess (我是其作者)，它使用 dill 序列化器......并且能够准确地实现您正在寻找的行为类型。

只需将 from multiprocessing 替换为 from multiprocess ，您的代码就会按预期运行。