python - islice 一次读取 N 行的问题

Question

我正在尝试使用“from itertools import islice”，以便使用 liblas 模块一次从 *.las 文件中读取多行。 (我的目标是逐 block 阅读)

以下问题:Python how to read N number of lines at a time

islice() can be used to get the next n items of an iterator. Thus, list(islice(f, n)) will return a list of the next n lines of the file f. Using this inside a loop will give you the file in chunks of n lines. At the end of the file, the list might be shorter, and finally the call will return an empty list.

我使用了以下代码:

from numpy import nonzero
from liblas import file as lasfile
from itertools import islice


chunkSize = 1000000

f = lasfile.File(inFile,None,'r') # open LAS
while True:
    chunk = list(islice(f,chunkSize))
    if not chunk:
        break
    # do other stuff

但是我有这个问题:

len(f)
2866390

chunk = list(islice(f, 1000000))
len(chunk)
**1000000**
chunk = list(islice(f, 1000000))
len(chunk)
**1000000**
chunk = list(islice(f, 1000000))
len(chunk)
**866390**
chunk = list(islice(f, 1000000))
len(chunk)
**1000000**

当文件f到达末尾时，islice重新开始读取文件。

感谢您的任何建议和帮助。非常感谢

Answer 1

编写一个生成器一次生成 n 行似乎很容易:

def n_line_iterator(fobj,n):
    if n < 1:
       raise ValueError("Must supply a positive number of lines to read")

    out = []
    num = 0
    for line in fobj:
       if num == n:
          yield out  #yield 1 chunk
          num = 0
          out = []
       out.append(line)
       num += 1
    yield out  #need to yield the rest of the lines