tensorflow - Keras:加载由多个 GPU 生成的检查点权重 HDF5

Question

检查点片段:

checkpointer = ModelCheckpoint(filepath=os.path.join(savedir, "mid/weights.{epoch:02d}.hd5"), monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False)
hist = model.fit_generator(
    gen.generate(batch_size = batch_size, nb_classes=nb_classes), samples_per_epoch=593920, nb_epoch=nb_epoch, verbose=1, callbacks=[checkpointer], validation_data = gen.vld_generate(VLD_PATH, batch_size = 64, nb_classes=nb_classes), nb_val_samples=10000
)

我在转储 mid 的多 GPU 主机上训练了我的模型HDF5 格式的文件。当我使用 keras.load_weights('mid') 在一台 GPU 机器上加载它们时，出现错误:

Using TensorFlow backend.
Traceback (most recent call last):
  File "server.py", line 171, in <module>
    model = load_model_and_weights('zhch.yml', '7_weights.52.hd5')
  File "server.py", line 16, in load_model_and_weights
    model.load_weights(os.path.join('model', weights_name))
  File "/home/lz/code/ProjectGo/meta/project/libpolicy-server/.virtualenv/lib/python3.5/site-packages/keras/engine/topology.py", line 2701, in load_weights
    self.load_weights_from_hdf5_group(f)
  File "/home/lz/code/ProjectGo/meta/project/libpolicy-server/.virtualenv/lib/python3.5/site-packages/keras/engine/topology.py", line 2753, in load_weights_from_hdf5_group
    str(len(flattened_layers)) + ' layers.')
ValueError: You are trying to load a weight file containing 1 layers into a model with 21 layers.

有没有办法在单个 GPU 机器上加载由多个 GPU 生成的检查点权重？似乎 Keras 没有讨论这个问题，因此任何帮助将不胜感激。

Answer 1

您可以像这样在单个 GPU 上加载模型:

from keras.models import load_model

multi_gpus_model = load_model('mid')
origin_model = multi_gpus_model.layers[-2]  # you can use multi_gpus_model.summary() to see the layer of the original model
origin_model.save_weights('single_gpu_model.hdf5')

'single_gpu_model.hdf5' 是可以加载到单 GPU 机器模型的文件。