Python 子进程 readlines() 挂起

Question

我尝试完成的任务是流式传输 ruby​​ 文件并打印输出。 (注意:我不想一次打印所有内容)

ma​​in.py

from subprocess import Popen, PIPE, STDOUT

import pty
import os

file_path = '/Users/luciano/Desktop/ruby_sleep.rb'

command = ' '.join(["ruby", file_path])

master, slave = pty.openpty()
proc = Popen(command, bufsize=0, shell=True, stdout=slave, stderr=slave, close_fds=True)     
stdout = os.fdopen(master, 'r', 0)

while proc.poll() is None:
    data = stdout.readline()
    if data != "":
        print(data)
    else:
        break

print("This is never reached!")

ruby_sleep.rb

puts "hello"

sleep 2

puts "goodbye!"

问题

流式传输文件工作正常。 hello/goodbye 输出以 2 秒的延迟打印。就像脚本应该工作一样。问题是 readline() 最终挂起并且永远不会退出。我从来没有到达最后一张。

我知道这里有很多这样的问题 stackoverflow 但没有一个让我解决了问题。我不太喜欢整个子流程，所以请给我一个更实际/具体的答案。

问候

编辑

修复意外代码。 (与实际错误无关)

Answer 1

由于 Q: Why not just use a pipe (popen())? 中列出的原因，我假设您使用 pty (到目前为止，所有其他答案都忽略了您的“注意:我不想一次打印所有内容”)。

pty 仅适用于 Linux as said in the docs :

Because pseudo-terminal handling is highly platform dependent, there is code to do it only for Linux. (The Linux code is supposed to work on other platforms, but hasn’t been tested yet.)

尚不清楚它在其他操作系统上的效果如何。

你可以试试pexpect:

import sys
import pexpect

pexpect.run("ruby ruby_sleep.rb", logfile=sys.stdout)

或stdbuf在非交互模式下启用行缓冲:

from subprocess import Popen, PIPE, STDOUT

proc = Popen(['stdbuf', '-oL', 'ruby', 'ruby_sleep.rb'],
             bufsize=1, stdout=PIPE, stderr=STDOUT, close_fds=True)
for line in iter(proc.stdout.readline, b''):
    print line,
proc.stdout.close()
proc.wait()

或使用基于 @Antti Haapala's answer 的 stdlib 中的 pty :

#!/usr/bin/env python
import errno
import os
import pty
from subprocess import Popen, STDOUT

master_fd, slave_fd = pty.openpty()  # provide tty to enable
                                     # line-buffering on ruby's side
proc = Popen(['ruby', 'ruby_sleep.rb'],
             stdin=slave_fd, stdout=slave_fd, stderr=STDOUT, close_fds=True)
os.close(slave_fd)
try:
    while 1:
        try:
            data = os.read(master_fd, 512)
        except OSError as e:
            if e.errno != errno.EIO:
                raise
            break # EIO means EOF on some systems
        else:
            if not data: # EOF
                break
            print('got ' + repr(data))
finally:
    os.close(master_fd)
    if proc.poll() is None:
        proc.kill()
    proc.wait()
print("This is reached!")

所有三个代码示例都会立即打印“hello”(只要看到第一个 EOL)。

---

^{在此处保留旧的更复杂的代码示例，因为它可能会在 SO 的其他帖子中被引用和讨论}

或者使用基于@Antti Haapala's answer的pty :

import os
import pty
import select
from subprocess import Popen, STDOUT

master_fd, slave_fd = pty.openpty()  # provide tty to enable
                                     # line-buffering on ruby's side
proc = Popen(['ruby', 'ruby_sleep.rb'],
             stdout=slave_fd, stderr=STDOUT, close_fds=True)
timeout = .04 # seconds
while 1:
    ready, _, _ = select.select([master_fd], [], [], timeout)
    if ready:
        data = os.read(master_fd, 512)
        if not data:
            break
        print("got " + repr(data))
    elif proc.poll() is not None: # select timeout
        assert not select.select([master_fd], [], [], 0)[0] # detect race condition
        break # proc exited
os.close(slave_fd) # can't do it sooner: it leads to errno.EIO error
os.close(master_fd)
proc.wait()

print("This is reached!")