python-3.x - ValueError ("I/O operation on closed pipe") 将 asyncio 与子进程一起使用时

Question

我的代码的简化版本如下所示:

def run_async(spawn_coro, timeout):
    async def _read_print_line(process):
        line = await process.stdout.readline()
        line = line.decode('utf-8', errors='replace')
        sys.stdout.write(line)

    process = await spawn_coro
    try:
        while not process.stdout.at_eof():
            # We should cancel if no output has been received for a certain timeout
            await asyncio.wait_for(_read_print_line(process), timeout)
    except asyncio.TimeoutError:
        print('Timed out executing "{}".  No output received for {} seconds'.format(cmd, timeout))
        process.kill()
    await process.wait()

processes = [asyncio.create_subprocess_shell(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) for cmd in cmds]

loop = asyncio.ProactorEventLoop()   # This is windows
asyncio.set_event_loop(loop)
all_jobs = asyncio.gather(
    *(run_async(process, timeout) for process in processes)
)
loop.run_until_complete(all_jobs)
loop.close()

当进程退出时，我得到一堆这样的错误输出:

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001CD9B582AF8>
Traceback (most recent call last):
  File "C:\Users\divis\AppData\Local\Programs\Python\Python37\lib\asyncio\proactor_events.py", line 93, in __del__
    warnings.warn(f"unclosed transport {self!r}", ResourceWarning,
  File "C:\Users\divis\AppData\Local\Programs\Python\Python37\lib\asyncio\proactor_events.py", line 57, in __repr__
    info.append(f'fd={self._sock.fileno()}')
  File "C:\Users\divis\AppData\Local\Programs\Python\Python37\lib\asyncio\windows_utils.py", line 102, in fileno
    raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x000001CD9B571DC8>
Traceback (most recent call last):
    warnings.warn(f"unclosed transport {self!r}", ResourceWarning,
  File "C:\Users\divis\AppData\Local\Programs\Python\Python37\lib\asyncio\base_subprocess.py", line 78, in __repr__
    info.append(f'stdout={stdout.pipe}')
  File "C:\Users\divis\AppData\Local\Programs\Python\Python37\lib\asyncio\proactor_events.py", line 57, in __repr__
    info.append(f'fd={self._sock.fileno()}')
  File "C:\Users\divis\AppData\Local\Programs\Python\Python37\lib\asyncio\windows_utils.py", line 102, in fileno
    raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe

如果我删除 loop.close()，则进程不会出现错误而是挂起。在关闭循环之前，我正在正确地等待每个子进程，所以我不确定这里的问题是什么。

Answer 1

我知道这个问题不久前有人问过，但我遇到了同样的问题，我能够这样解决它:

try:
    process = asyncio.create_subprocess_exec(cmd)
    ...
    await process.wait()
finally:
    process._transport.close()

查看代码，这似乎关闭了子进程的管道文件句柄。我不知道为什么 asyncio 不将其作为返回的 Process 对象的一部分公开。