python - 将子进程输出显示到标准输出并将其重定向

Question

我正在通过 Python 的子进程模块运行脚本。目前我使用:

p = subprocess.Popen('/path/to/script', stdout=subprocess.PIPE, stderr=subprocess.PIPE)
result = p.communicate()

然后我将结果打印到标准输出。这一切都很好，但由于脚本需要很长时间才能完成，我也希望从脚本实时输出到标准输出。我通过管道输出的原因是因为我想解析它。

Answer 1

将子进程的标准输出保存到变量以供进一步处理，并保存到 display it while the child process is running as it arrives :

#!/usr/bin/env python3
from io import StringIO
from subprocess import Popen, PIPE

with Popen('/path/to/script', stdout=PIPE, bufsize=1,
           universal_newlines=True) as p, StringIO() as buf:
    for line in p.stdout:
        print(line, end='')
        buf.write(line)
    output = buf.getvalue()
rc = p.returncode

同时保存子进程的标准输出和标准错误更复杂，因为您应该 consume both streams concurrently to avoid a deadlock :

stdout_buf, stderr_buf = StringIO(), StringIO()
rc =  teed_call('/path/to/script', stdout=stdout_buf, stderr=stderr_buf,
                universal_newlines=True)
output = stdout_buf.getvalue()
...

哪里teed_call() is define here .

---

更新:这里是a simpler asyncio version .

---

^旧版本:

这是一个基于child_process.py example from tulip 的单线程解决方案:

import asyncio
import sys
from asyncio.subprocess import PIPE

@asyncio.coroutine
def read_and_display(*cmd):
    """Read cmd's stdout, stderr while displaying them as they arrive."""
    # start process
    process = yield from asyncio.create_subprocess_exec(*cmd,
            stdout=PIPE, stderr=PIPE)

    # read child's stdout/stderr concurrently
    stdout, stderr = [], [] # stderr, stdout buffers
    tasks = {
        asyncio.Task(process.stdout.readline()): (
            stdout, process.stdout, sys.stdout.buffer),
        asyncio.Task(process.stderr.readline()): (
            stderr, process.stderr, sys.stderr.buffer)}
    while tasks:
        done, pending = yield from asyncio.wait(tasks,
                return_when=asyncio.FIRST_COMPLETED)
        assert done
        for future in done:
            buf, stream, display = tasks.pop(future)
            line = future.result()
            if line: # not EOF
                buf.append(line)    # save for later
                display.write(line) # display in terminal
                # schedule to read the next line
                tasks[asyncio.Task(stream.readline())] = buf, stream, display

    # wait for the process to exit
    rc = yield from process.wait()
    return rc, b''.join(stdout), b''.join(stderr)

脚本运行 '/path/to/script 命令并同时逐行读取其标准输出和标准错误。这些行相应地打印到父级的标准输出/标准错误，并保存为字节串以供将来处理。要运行 read_and_display() 协程，我们需要一个事件循环:

import os

if os.name == 'nt':
    loop = asyncio.ProactorEventLoop() # for subprocess' pipes on Windows
    asyncio.set_event_loop(loop)
else:
    loop = asyncio.get_event_loop()
try:
    rc, *output = loop.run_until_complete(read_and_display("/path/to/script"))
    if rc:
        sys.exit("child failed with '{}' exit code".format(rc))
finally:
    loop.close()