python - 将子进程输出缓冲到文件 : io. 不支持的操作:fileno

Question

我想将 grep 的输出缓冲到缓冲区，然后用 pandas 读取它，以避免将巨大的原始文件加载到内存中:

import subprocess
import io
import pandas as pd

firstfile = "~/references/rs_hg19.snps.uniq.bed"
outf = io.StringIO("")
p0 = subprocess.Popen(('grep', '-P', "chr22\\t", firstfile), stdout=subprocess.PIPE)
p1 = subprocess.call(["head", "-n5", ], stdin=p0.stdout, stdout= outf)
p0.wait()
print(pd.read_table(outf)
    )

我收到错误:

Traceback (most recent call last)
  File "test.py", line 9, in <module>
    p1 = subprocess.call(["head", "-n1", ], stdin=p0.stdout, stdout= outf)
  File "/opt/rh/python33/root/usr/lib64/python3.3/subprocess.py", line 520, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/opt/rh/python33/root/usr/lib64/python3.3/subprocess.py", line 786, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/opt/rh/python33/root/usr/lib64/python3.3/subprocess.py", line 1294, in _get_handles
    c2pwrite = stdout.fileno()
io.UnsupportedOperation: fileno

即使我直接运行它，not from any IDE .

有什么想法/建议吗？

系统1:CentOS6.7、Python3.3

系统2:MacOSX10.10.5、Python3.5

Answer 1

您可以通过让子进程将标准输出读取为字节字符串来完成您想要的操作。将第二个调用替换为

p1 = subprocess.Popen(["head", "-n5" ], stdin=p0.stdout, stdout=subprocess.PIPE)
data = p1.communicate()[0]
string = data.decode('utf-8')