python - 在不加载到内存的情况下查找保存的 numpy 数组(.npy 或 .npz)的形状

Question

我有一个巨大的压缩 numpy 数组保存到磁盘(内存约 20gb，压缩后更少)。我需要知道这个数组的形状，但我没有足够的内存来加载它。如何在不将其加载到内存的情况下找到 numpy 数组的形状？

Answer 1

这样做:

import numpy as np
import zipfile

def npz_headers(npz):
    """Takes a path to an .npz file, which is a Zip archive of .npy files.
    Generates a sequence of (name, shape, np.dtype).
    """
    with zipfile.ZipFile(npz) as archive:
        for name in archive.namelist():
            if not name.endswith('.npy'):
                continue

            npy = archive.open(name)
            version = np.lib.format.read_magic(npy)
            shape, fortran, dtype = np.lib.format._read_array_header(npy, version)
            yield name[:-4], shape, dtype