python-3.x - 为什么python为共享锁抛出 "multiprocessing.managers.RemoteError"？

Question

我在 Ubuntu 18.04 上使用 python 3.6.7

运行以下脚本后，每个进程都有自己的共享锁:

from multiprocessing import Process, Manager


def foo(l1):
    with l1:
        print('lol')


if __name__ == '__main__':
    processes = []
    with Manager() as manager:
        for cluster in range(10):
            lock1 = manager.Lock()
            calc_args = (lock1, )
            processes.append(Process(target=foo,
                                     args=calc_args))
        for p in processes:
            p.start()

        for p in processes:
            p.join()

我有一个奇怪的异常(exception):

Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "temp.py", line 5, in foo
    with l1:
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 991, in __enter__
    return self._callmethod('acquire')
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 235, in serve_client
    self.id_to_local_proxy_obj[ident]
KeyError: '7f49974624e0'

任何想法或建议如何解决这个问题？

谢谢

Answer 1

出于某种原因，您必须保留 原版引用您从 SyncManager 获得的任何内容，请参见下文:

from multiprocessing import Manager, Process, current_process
from multiprocessing.managers import AcquirerProxy, SyncManager

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager: SyncManager = Manager()

    locks = []
    for i in range(3):
        # Always keep the reference in some variable
        locks.append(manager.Lock())

    processes = []
    for i in range(3):
        p = Process(target=foo, args=[locks[i]])
        processes.append(p)

    # If you clear the list, which you lose the reference, it won't work
    # locks.clear()

    for p in processes:
        p.start()
    for p in processes:
        p.join()

很抱歉在聚会上迟到了，但我希望它有所帮助!

---------- 原回复如下: ----------

嘿，这个问题在 Python 3.8.2 中仍然存在

我设法重现了错误:

from multiprocessing import Process, Manager, current_process
from multiprocessing.managers import AcquirerProxy

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager = Manager()

    process1 = Process(target=foo, args=[manager.Lock()])

    process1.start()
    process1.join()

但是如果我拿出 manager.Lock() ，它工作正常!

from multiprocessing import Process, Manager, current_process
from multiprocessing.managers import AcquirerProxy

def foo(lock: AcquirerProxy):
    lock.acquire()
    print('pid={}'.format(current_process().pid))

if __name__ == '__main__':
    manager = Manager()

    lock1 = manager.Lock()           # Here
    process1 = Process(target=foo, args=[lock1])

    process1.start()
    process1.join()

真的很困惑为什么取出锁会有所不同