python - 取消已挂起的 ProcessPoolExecutor future

Question

我有一个 python 函数正在调用我无法控制或更新的 C 库。不幸的是，C 库存在间歇性错误，有时会挂起。为了防止我的应用程序也挂起，我尝试隔离 ThreadPoolExecutor 或 ProcessPoolExecutor 中的函数调用，以便只有该线程或进程崩溃。

但是，以下代码挂起，因为执行器无法关闭，因为进程仍在运行!

是否可以取消具有挂起的 future 的执行者？

import time
from concurrent.futures import ThreadPoolExecutor, wait

if __name__ == "__main__":
    def hang_forever(*args):
        print("Starting hang_forever")
        time.sleep(10.0)
        print("Finishing hang_forever")

    print("Starting executor")
    with ThreadPoolExecutor() as executor:
        future = executor.submit(hang_forever)
        print("Submitted future")
        done, not_done = wait([future], timeout=1.0)
        print("Done", done, "Not done", not_done)
        # with never exits because future has hung!
        if len(not_done) > 0:
            raise IOError("Timeout")

Answer 1

docs说在所有待处理的 future 执行完毕之前不可能关闭执行器:

Regardless of the value of wait, the entire Python program will not exit until all pending futures are done executing.

调用 future.cancel() 不会有帮助，因为它也会挂起。幸运的是，您可以直接使用 multiprocessing.Process 而不是使用 ProcessPoolExecutor 来解决您的问题:

import time
from multiprocessing import Process


def hang_forever():

    while True:
        print('hang forever...')
        time.sleep(1)


def main():
    proc = Process(target=hang_forever)

    print('start the process')
    proc.start()

    time.sleep(1)

    timeout = 3
    print(f'trying to join the process in {timeout} sec...')
    proc.join(timeout)

    if proc.is_alive():
        print('timeout is exceeded, terminate the process!')
        proc.terminate()
        proc.join()

    print('done')


if __name__ == '__main__':
    main()

输出:

start the process
hang forever...
trying to join the process in 3 sec...
hang forever...
hang forever...
hang forever...
hang forever...
timeout is exceeded, terminate the process!
done