python - 微调 keras .h5 模型到 c++ 中使用的 .pb 模型给出运行模型失败错误

Question

我已经微调了 VGG16 模型并以最佳精度保存了模型。我的keras代码是-

#Input size 
rows = 125
column = 125

channels = 3 #RGB

#Resizing

def read_images(src):
    #read every image and resize every image to same size
    img = Image.open(src)
    #print(img.size)
    #method = Image.NEAREST if img.size == (rows,column) else Image.ANTIALIAS
    im_rz = img.resize((rows,column), Image.ANTIALIAS)
    #im_rz = resize_image_to_square(img, new_size = (rows,column))
    #im_rz = ImageOps.fit(img, (rows,column), method = method)
    return im_rz

#Conversion of image to np

l = LabelEncoder()
y_all = l.fit_transform(y_all)
y_all = np_utils.to_categorical(y_all)

#print(l.classes_)
#x_train, x_valid, y_train, y_valid = test_train_split(x_all, y_all, test_size = 0.2, random_state = 42, stratify = y_all)

x_train, x_test, y_train, y_test = train_test_split(x_all, y_all, test_size = 0.2, random_state = 42, stratify = y_all)

batch_size = 32
nb_classes = 3
epochs = 300

x_train = np.array(x_train).astype('float32')
x_test = np.array(x_test).astype('float32')
y_train = np.array(y_train).astype('float32')
y_test = np.array(y_test).astype('float32')

x_train /= 255.
x_test /= 255.

#model

vgg = VGG16(weights='imagenet', include_top=False, input_shape=(125, 125, 3))

# Freeze the layers except the last 4 layers
for layer in vgg.layers[:-4]:
    layer.trainable = False

# Check the trainable status of the individual layers
for layer in vgg.layers:
    print(layer, layer.trainable)

model = Sequential()

# Add the vgg convolutional base model
model.add(vgg_conv)

# Add new layers
model.add(layers.Flatten())
model.add(layers.Dense(1024, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(3, activation='softmax'))

print(model.summary())


opt = keras.optimizers.RMSProp(lr=1e-4)

# #Training the model using optimizer

model.compile(loss='categorical_crossentropy',optimizer=opt,metrics=['accuracy']) 

#Save the best model obtained during training

#from keras.callbacks import ReduceLROnPlateau
#lr_reduce = ReduceLROnPlateau(monitor='val_acc', factor=0.1, epsilon=0.0001, patience=1, verbose=1)

filepath = "weights.h5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=0, save_best_only=True, mode='max')
callbacks_list = [checkpoint]

#data generation and fitting

data_generation = ImageDataGenerator(
    rotation_range = 45, #Makes sense to keep the rotation between 0 and 10, we don't want to rotate it a lot
    width_shift_range = 0.0,
    height_shift_range = 0.0,
    horizontal_flip=True, #Random rotations, might help
    vertical_flip=True,
    zoom_range = 0.15
    )

#data_generation.fit(x_train)



#Model fitting! 

#tbCallBack = keras.callbacks.TensorBoard(log_dir="./Graph", histogram_freq = 0, write_graph = True, write_images = True)
#callbacks_list.append(tbCallBack)

model_param=model.fit_generator(data_generation.flow(x_train, y_train,batch_size=batch_size),
                                steps_per_epoch=x_train.shape[0] // batch_size,
                                epochs=epochs,
                validation_data=(x_test, y_test),
                validation_steps=x_test.shape[0], 
                callbacks=callbacks_list)

这是模型的总结

Layer (type)                 Output Shape              Param #
=================================================================
vgg16 (Model)                (None, 7, 7, 512)         14714688
_________________________________________________________________
flatten_1 (Flatten)          (None, 25088)             0
_________________________________________________________________
dense_1 (Dense)              (None, 1024)              25691136
_________________________________________________________________
dropout_1 (Dropout)          (None, 1024)              0
_________________________________________________________________
dense_2 (Dense)              (None, 3)                 3075
=================================================================
Total params: 40,408,899
Trainable params: 32,773,635
Non-trainable params: 7,635,264
_________________________________________________________________

我已使用此脚本将其从 .h5 文件转换为 .pb 文件，以便在 C++ 中使用。 https://github.com/bitbionic/keras-to-tensorflow/blob/master/k2tf_convert.py

我使用了类似于此代码的 C++ 代码 - https://github.com/bitbionic/keras-to-tensorflow/blob/master/main.cpp

在 C++ 中执行代码时，出现以下错误 -

Running model failed: Invalid argument: Matrix size-incompatible: In[0]: [1,1152], In[1]: [4608,512]
         [[Node: dense_1/MatMul = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/cpu:0"](flatten_1/Reshape, dense_1/kernel/read)]]

如何消除此错误以及是否未正确创建 pb 文件？

Answer 1

问题可能是由于在将图像馈送到 VGG16 网络之前将图像大小调整为 125*125。它需要 224*224 的图像大小，否则在训练时图像大小可能会减小到零，这很可能正在发生。将图像大小保持为 224*224 消除了所有错误，并允许在更大的批量大小上训练模型。