gpt4 book ai didi

python - 如何在进程之间共享一个类?

转载 作者:太空狗 更新时间:2023-10-29 18:01:30 25 4
gpt4 key购买 nike

我想要一个全局对象,该对象由所有进程以最小锁定共享和更新。

import multiprocessing

class Counter(object):
def __init__(self):
self.value = 0

def update(self, value):
self.value += value


def update(counter_proxy, thread_id):
counter_proxy.value.update(1)
print counter_proxy.value.value, 't%s' % thread_id, \
multiprocessing.current_process().name
return counter_proxy.value.value

def main():
manager = multiprocessing.Manager()
counter = manager.Value(Counter, Counter())
pool = multiprocessing.Pool(multiprocessing.cpu_count())
for i in range(10):
pool.apply(func = update, args = (counter, i))
pool.close()
pool.join()

print 'Should be 10 but is %s.' % counter.value.value

if __name__ == '__main__':
main()

结果是这样的——不是 10,而是零。看起来对象的共享值没有更新。如何锁定和更新此类值?

0 t0 PoolWorker-2
0 t1 PoolWorker-3
0 t2 PoolWorker-5
0 t3 PoolWorker-8
0 t4 PoolWorker-9
0 t5 PoolWorker-2
0 t6 PoolWorker-7
0 t7 PoolWorker-4
0 t8 PoolWorker-6
0 t9 PoolWorker-3
Should be 10 but is 0.

@dano 目前最好的解决方案 - 我将自定义管理器与类代理混合。

import multiprocessing
from multiprocessing.managers import BaseManager, NamespaceProxy


class Counter(object):
def __init__(self):
self.value = 0

def update(self, value):
self.value += value


def update(counter_proxy, thread_id):
counter_proxy.update(1)

class CounterManager(BaseManager):
pass

class CounterProxy(NamespaceProxy):
_exposed_ = ('__getattribute__', '__setattr__', '__delattr__', 'update')

def update(self, value):
callmethod = object.__getattribute__(self, '_callmethod')
return callmethod(self.update.__name__, (value,))

CounterManager.register('Counter', Counter, CounterProxy)

def main():
manager = CounterManager()
manager.start()

counter = manager.Counter()
pool = multiprocessing.Pool(multiprocessing.cpu_count())
for i in range(10):
pool.apply(func = update, args = (counter, i))
pool.close()
pool.join()

print 'Should be 10 but is %s.' % counter.value

if __name__ == '__main__':
main()

最佳答案

multiprocessing.Value 并非设计用于自定义类,它应该类似于 multiprocessing.sharedctypes.Value .相反,您需要创建一个 custom manager并用它注册你的类(class)。如果您不直接访问 value,而是通过方法修改/访问它,您的生活也会更轻松,这些方法将由为您的类创建的默认 Proxy 导出默认。常规属性(如 Counter.value)不是,因此如果不进行额外的自定义,则无法访问它们。这是一个工作示例:

import multiprocessing
from multiprocessing.managers import BaseManager

class MyManager(BaseManager): pass

def Manager():
m = MyManager()
m.start()
return m

class Counter(object):
def __init__(self):
self._value = 0

def update(self, value):
self._value += value

def get_value(self):
return self._value

MyManager.register('Counter', Counter)

def update(counter_proxy, thread_id):
counter_proxy.update(1)
print counter_proxy.get_value(), 't%s' % thread_id, \
multiprocessing.current_process().name
return counter_proxy

def main():
manager = Manager()
counter = manager.Counter()
pool = multiprocessing.Pool(multiprocessing.cpu_count())
for i in range(10):
pool.apply(func=update, args=(counter, i))
pool.close()
pool.join()

print 'Should be 10 but is %s.' % counter.get_value()

if __name__ == '__main__':
main()

输出:

1 t0 PoolWorker-2
2 t1 PoolWorker-8
3 t2 PoolWorker-4
4 t3 PoolWorker-5
5 t4 PoolWorker-6
6 t5 PoolWorker-7
7 t6 PoolWorker-3
8 t7 PoolWorker-9
9 t8 PoolWorker-2
10 t9 PoolWorker-8
Should be 10 but is 10.

关于python - 如何在进程之间共享一个类?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/28612412/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com