Python分别从子进程stdout和stderr读取，同时保留顺序

Question

我有一个 python 子进程，我正在尝试从中读取输出和错误流。目前我有它的工作，但我只能在读完 stdout 之后才能从 stderr 中读取。这是它的样子:

process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout_iterator = iter(process.stdout.readline, b"")
stderr_iterator = iter(process.stderr.readline, b"")

for line in stdout_iterator:
    # Do stuff with line
    print line

for line in stderr_iterator:
    # Do stuff with line
    print line

如您所见，在 stdout 循环完成之前，stderr for 循环无法启动。我怎样才能修改它以便能够以正确的行进来的顺序从两者中读取？

澄清一下:我仍然需要能够判断一行是来自 stdout 还是 stderr 因为它们在我的代码。

Answer 1

如果子进程在 stderr 上产生足够的输出(在我的 Linux 机器上约为 100KB)，您问题中的代码可能会死锁。

有一个 communicate() 方法允许分别从 stdout 和 stderr 读取:

from subprocess import Popen, PIPE

process = Popen(command, stdout=PIPE, stderr=PIPE)
output, err = process.communicate()

如果您需要在子进程仍在运行时读取流，那么可移植的解决方案是使用线程(未测试):

from subprocess import Popen, PIPE
from threading import Thread
from Queue import Queue # Python 2

def reader(pipe, queue):
    try:
        with pipe:
            for line in iter(pipe.readline, b''):
                queue.put((pipe, line))
    finally:
        queue.put(None)

process = Popen(command, stdout=PIPE, stderr=PIPE, bufsize=1)
q = Queue()
Thread(target=reader, args=[process.stdout, q]).start()
Thread(target=reader, args=[process.stderr, q]).start()
for _ in range(2):
    for source, line in iter(q.get, None):
        print "%s: %s" % (source, line),

见:

• Python: read streaming input from subprocess.communicate()
• Non-blocking read on a subprocess.PIPE in python
• Python subprocess get children's output to file and terminal?