python - 尝试打开 h5py 文件，返回 errorno = 17，错误信息 = 'file exists'

Question

import numpy as np import h5py

with h5py.File("testfile.hdf5", "w-") as f: arr = np.ones((5,2)) f["my dataset"] = arr dset = f["my dataset"]

此代码第一次运行正确，但第二次运行时，返回以下错误:

%run "C:\Users\James\Google Drive\Python Scripts\Python and HDF5\Chapter3.py" --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) C:\Users\James\Google Drive\Python Scripts\Python and HDF5\Chapter3.py in () 6 with h5py.File("testfile.hdf5") as f: 7 arr = np.ones((5,2)) ----> 8 f["my dataset"] = arr 9 dset = f["my dataset"] 10

h5py_objects.pyx in h5py._objects.with_phil.wrapper (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py_objects.c:2696)()

h5py_objects.pyx in h5py._objects.with_phil.wrapper (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py_objects.c:2654)()

C:\Users\James\AppData\Local\Enthought\Canopy\User\lib\site-packages\h5py_hl\group.py in setitem(self, name, obj) 291 else: 292 ds = self.create_dataset(None, data=obj, dtype=base.guess_dtype(obj)) --> 293 h5o.link(ds.id, self.id, name, lcpl=lcpl) 294 295 @with_phil

h5py_objects.pyx in h5py._objects.with_phil.wrapper (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py_objects.c:2696)()

h5py_objects.pyx in h5py._objects.with_phil.wrapper (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py_objects.c:2654)()

h5py\h5o.pyx in h5py.h5o.link (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py\h5o.c:3610)()

RuntimeError: Unable to create link (Name already exists)

%run "C:\Users\James\Google Drive\Python Scripts\Python and HDF5\Chapter3.py" --------------------------------------------------------------------------- IOError Traceback (most recent call last) C:\Users\James\Google Drive\Python Scripts\Python and HDF5\Chapter3.py in () 4 from timeit import timeit 5 ----> 6 with h5py.File("testfile.hdf5", "w-") as f: 7 arr = np.ones((5,2)) 8 f["my dataset"] = arr

C:\Users\James\AppData\Local\Enthought\Canopy\User\lib\site-packages\h5py_hl\files.py in init(self, name, mode, driver, libver, userblock_size, swmr, **kwds) 270 271 fapl = make_fapl(driver, libver, **kwds) --> 272 fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr) 273 274 if swmr_support:

C:\Users\James\AppData\Local\Enthought\Canopy\User\lib\site-packages\h5py_hl\files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr) 94 fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl) 95 elif mode in ['w-', 'x']: ---> 96 fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl) 97 elif mode == 'w': 98 fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)

h5py_objects.pyx in h5py._objects.with_phil.wrapper (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py_objects.c:2696)()

h5py_objects.pyx in h5py._objects.with_phil.wrapper (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py_objects.c:2654)()

h5py\h5f.pyx in h5py.h5f.create (C:\pisi\tmp\h5py-2.6.0-2\work\h5py-2.6.0\h5py\h5f.c:2109)()

IOError: Unable to create file (Unable to open file: name = 'testfile.hdf5', errno = 17, error message = 'file exists', flags = 15, o_flags = 502)

代码和错误是在 Canopy//Python 3.5 中运行的。我也在 Spyder 中运行它并收到相同的结果。我也尝试使用

with h5py.File("testfile.hdf5", "a") as f:

没有成功。

Answer 1

我在使用 HDF5Matrix 时遇到了完全相同的错误消息keras(v2.2.2) 中的类。但是，当我有多个训练过程都需要访问磁盘上相同的 HDF5 数据时，我未能找到一个成熟的解决方案来完全避免这个错误。只有一个进程可以成功访问此 HDF5 数据，而所有其他进程都会报告相同的错误，即使我将读取模式从默认的 r+ 修改为 r。我放弃并使用了可行的解决方案，即保留多个 HDF5 数据副本并为每个训练过程保留一个副本。