python - numpy.savez 之后 np.load 的断言错误

Question

我有 5 个 numpy 数组 a,b,c,d 和 e 都定义为:

array([1, 2, 3, 4, 5, 6, 7, 8, 9])

我像这样保存这些数组:

np.savez_compressed('tmp/test',a=a,b=b,c=c,d=d,e=e)

这会导致创建文件 test.npz。

但是我在尝试加载数据时遇到问题(遵循示例 here ):

>>> f=np.load('tmp/test.npz')
>>> f.files
['a', 'c', 'b', 'e', 'd']
>>> f['a']

给出一大串以以下结尾的错误:

File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
return self.com_dictorsetmaker(nodelist[1])
File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
assert nodelist[0] == symbol.dictorsetmaker
AssertionError

我考虑过使用 pickle 代替。然而，这导致文件大小是 .npz 文件的四倍，所以我想使用 savez 或 savez_compressed。

有没有人知道我做错了什么，或者有替代方法的建议？

这是一个会产生错误的脚本:

def saver():
    import numpy as np
    a= np.arange(1,10)
    b=a
    c=a
    d=a
    e=a
    np.savez_compressed('tmp/test',a=a,b=b,c=c,d=d,e=e)
    f=np.load('tmp/test.npz')
    print f.files
    print f['a']

这是完整的回溯:

Traceback (most recent call last):
  File "<pyshell#16>", line 1, in <module>
    saver.saver()
  File "C:\Python27\saver.py", line 14, in saver
    print f['a']
  File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 241, in __getitem__
    return format.read_array(value)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 440, in read_array
    shape, fortran_order, dtype = read_array_header_1_0(fp)
  File "C:\Python27\lib\site-packages\numpy\lib\format.py", line 336, in read_array_header_1_0
    d = safe_eval(header)
  File "C:\Python27\lib\site-packages\numpy\lib\utils.py", line 1156, in safe_eval
    ast = compiler.parse(source, mode="eval")
  File "C:\Python27\lib\compiler\transformer.py", line 53, in parse
    return Transformer().parseexpr(buf)
  File "C:\Python27\lib\compiler\transformer.py", line 132, in parseexpr
    return self.transform(parser.expr(text))
  File "C:\Python27\lib\compiler\transformer.py", line 124, in transform
    return self.compile_node(tree)
  File "C:\Python27\lib\compiler\transformer.py", line 159, in compile_node
    return self.eval_input(node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 194, in eval_input
    return Expression(self.com_node(nodelist[0]))
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 578, in testlist
    return self.com_binary(Tuple, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 596, in test
    then = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 610, in or_test
    return self.com_binary(Or, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 615, in and_test
    return self.com_binary(And, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 619, in not_test
    result = self.com_node(nodelist[-1])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 626, in comparison
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 659, in expr
    return self.com_binary(Bitor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 663, in xor_expr
    return self.com_binary(Bitxor, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 667, in and_expr
    return self.com_binary(Bitand, nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 671, in shift_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 683, in arith_expr
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 695, in term
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 715, in factor
    node = self.lookup_node(nodelist[-1])(nodelist[-1][1:])
  File "C:\Python27\lib\compiler\transformer.py", line 727, in power
    node = self.com_node(nodelist[0])
  File "C:\Python27\lib\compiler\transformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:\Python27\lib\compiler\transformer.py", line 739, in atom
    return self._atom_dispatch[nodelist[0][0]](nodelist)
  File "C:\Python27\lib\compiler\transformer.py", line 754, in atom_lbrace
    return self.com_dictorsetmaker(nodelist[1])
  File "C:\Python27\lib\compiler\transformer.py", line 1214, in com_dictorsetmaker
    assert nodelist[0] == symbol.dictorsetmaker
AssertionError

Answer 1

无法在 Linux 或 Mac(Python 2.7、numpy 1.6.1/1.7.1)上重现您的问题

但是，我注意到您使用相对路径来保存文件 tmp/test.npz。那是故意的吗？在我的记忆中，最近版本的 Windows 对尝试在某些目录(如“/Program Files/”)中创建的新文件应用程序有一些特殊处理 - 它会将它们移开但在某些情况下仍然告诉应用程序它们在那里。这里似乎不太可能，但是您可以尝试保存文件的绝对路径吗？

顺便说一句:作为 ZIP 存档(由 savez savez_compressed 创建)的替代方案，您可以尝试使用“LZMAFile”作为文件对象进行 pickle。它提供了非常好的压缩率(但它可能很慢并且在压缩/保存文件时需要更多的内存和时间)；

它被用作任何其他文件对象包装器，类似的东西(加载腌制数据):

from lzma import LZMAFile
import cPickle as pickle

if fileName.endswith('.xz'):
    dataFile = LZMAFile(fileName,'r')
else:
    dataFile = file(fileName, 'ro')     
data = pickle.load(dataFile)