gpt4 book ai didi

python - 是否可以在 Python 的线程内生成进程?

转载 作者:可可西里 更新时间:2023-11-01 11:15:31 24 4
gpt4 key购买 nike

我正在编写一个生成进程并在特定条件下重新启动进程的程序。比如子进程不再给母进程发送数据,一段时间后,我想让母进程终止子进程,重新启动。我以为我可以使用线程从子进程接收数据并重新启动子进程,但它并不像我想的那样工作。

import numpy as np
import multiprocessing as mp
import threading
import time
from apscheduler.schedulers.background import BackgroundScheduler

pipe_in, pipe_out = mp.Pipe()

class Mother():
def __init__(self):
self.pipe_out = pipe_out

self.proc = mp.Process(target = self.test_func, args=(pipe_in, ))
self.proc.start()

self.thread = threading.Thread(target=self.thread_reciever, args=(self.pipe_out, ))
self.thread.start()

def thread_reciever(self, pipe_out):
while True:
value = pipe_out.recv()

print(value)
if value == 5:
self.proc.terminate()
time.sleep(2)
self.proc = mp.Process(target = self.test_func)
self.proc.start()

def test_func(self, pipe_in):
for i in range(10):
pipe_in.send(i)
time.sleep(1)


if __name__ == '__main__':
r = Mother()

它打印出这个错误。

D:\>d:\python36-32\python.exe temp06.py
0
1
2
3
4
5
Exception in thread Thread-1:
Traceback (most recent call last):
File "d:\python36-32\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "d:\python36-32\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "temp06.py", line 28, in thread_reciever
self.proc.start()
File "d:\python36-32\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "d:\python36-32\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "d:\python36-32\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "d:\python36-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
reduction.dump(process_obj, to_child)
File "d:\python36-32\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects


D:\>Traceback (most recent call last):
File "<string>", line 1, in <module>
File "d:\python36-32\lib\multiprocessing\spawn.py", line 99, in spawn_main
new_handle = reduction.steal_handle(parent_pid, pipe_handle)
File "d:\python36-32\lib\multiprocessing\reduction.py", line 82, in steal_handle
_winapi.PROCESS_DUP_HANDLE, False, source_pid)
OSError: [WinError 87]

如何在线程内启动和终止进程? (我正在使用一个线程,因为它可以同步接收来自不同进程的数据)或者还有其他方法可以完成这项工作吗?

作为全局函数的 test_func

import numpy as np
import multiprocessing as mp
import threading
import time
from apscheduler.schedulers.background import BackgroundScheduler

pipe_in, pipe_out = mp.Pipe()

def test_func( pipe_in):
for i in range(10):
pipe_in.send(i)
time.sleep(1)

class Mother():
def __init__(self):
self.pipe_out = pipe_out
mp.freeze_support()
self.proc = mp.Process(target = test_func, args=(pipe_in, ))
self.proc.start()

self.thread = threading.Thread(target=self.thread_reciever, args=(self.pipe_out, ))
self.thread.start()

def thread_reciever(self, pipe_out):
while True:
value = pipe_out.recv()

print(value)
if value == 5:
self.proc.terminate()
time.sleep(2)
mp.freeze_support()
self.proc = mp.Process(target = test_func, args=(pipe_in,))
self.proc.start()


if __name__ == '__main__':

r = Mother()

输出

D:\> d:\python36-32\python.exe temp06.py
0
1
2
3
4
5
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "d:\python36-32\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "d:\python36-32\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'test_func' on <module '__main__' (built-in)>

最佳答案

在windows下,由于没有fork系统调用,python启动一个新的解释器实例,使用pickle/unpickle重构执行上下文,但是 thread.Lock 不可 picklable。在 pickling self.test_func 时,self.thread 引用了一个 thread.Lock 对象,使其不可 picklable。

您可以简单地将 test_func 更改为普通的全局函数,而无需线程对象引用:

self.proc = mp.Process(target = test_func, args=(pipe_in,))
...
def test_func(pipe_in):
for i in range(10):
pipe_in.send(i)
time.sleep(1)

关于python - 是否可以在 Python 的线程内生成进程?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/53183365/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com