python - 检查目标时出错 : Converting FC layers to Conv2D

Question

我试图用卷积层替换 VGG 16 网络末端的 FC 层。下面是我的代码:

model2= Sequential()
model2.add(Conv2D(4096, kernel_size=(8,8), activation="relu"))
model2.add(Conv2D(4096, kernel_size=(1,1), activation="relu"))
model2.add(Conv2D(16, kernel_size=(1,1), activation="softmax"))

model = applications.VGG16(weights='imagenet', include_top=False, input_shape=inputshape)

F2model = Model(inputs=model.input, outputs=model2(model.output))

for layer in F2model.layers[:25]:
   layer.trainable = False

F2model.compile(optimizer=optimizers.Adam(), loss="binary_crossentropy", metrics=["accuracy"])

batch_size = 128
trainsize = 36000
validsize = 12000


F2model.fit_generator(
    train_generator,
    steps_per_epoch=trainsize // batch_size,
    epochs=5,
    validation_data=valid_generator,
    validation_steps=validsize // batch_size,callbacks=[tensorboard_callback])

我用 FC 层训练常规网络并且运行良好，但是当我运行上面的代码时，出现以下错误:

ValueError                                Traceback (most recent call last)in <module>
  4         epochs=5,
  5         validation_data=valid_generator,
  ----> 6         validation_steps=validsize // batch_size,callbacks=[tensorboard_callback])


ValueError: Error when checking target: expected sequential_1 to have 4 dimensions, but got array with shape (32, 16)

此时我正在尝试找出这些尺寸(32,16)来自哪里。任何帮助，将不胜感激。谢谢

编辑1:完整回溯:

ValueError                                Traceback (most recent call last)
<ipython-input-15-2702f38208c0> in <module>
      4         epochs=5,
      5         validation_data=valid_generator,
----> 6         validation_steps=validsize // batch_size,callbacks=[tensorboard_callback])

~/anaconda3/lib/python3.7/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/anaconda3/lib/python3.7/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1730             use_multiprocessing=use_multiprocessing,
   1731             shuffle=shuffle,
-> 1732             initial_epoch=initial_epoch)
   1733 
   1734     @interfaces.legacy_generator_methods_support

~/anaconda3/lib/python3.7/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    218                                             sample_weight=sample_weight,
    219                                             class_weight=class_weight,
--> 220                                             reset_metrics=False)
    221 
    222                 outs = to_list(outs)

~/anaconda3/lib/python3.7/site-packages/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics)
   1506             x, y,
   1507             sample_weight=sample_weight,
-> 1508             class_weight=class_weight)
   1509         if self._uses_dynamic_learning_phase():
   1510             ins = x + y + sample_weights + [1]

~/anaconda3/lib/python3.7/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
    619                 feed_output_shapes,
    620                 check_batch_axis=False,  # Don't enforce the batch size.
--> 621                 exception_prefix='target')
    622 
    623             # Generate sample-wise weight values given the `sample_weight` and

~/anaconda3/lib/python3.7/site-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
    133                         ': expected ' + names[i] + ' to have ' +
    134                         str(len(shape)) + ' dimensions, but got array '
--> 135                         'with shape ' + str(data_shape))
    136                 if not check_batch_axis:
    137                     data_shape = data_shape[1:]

ValueError: Error when checking target: expected conv2d_3 to have 4 dimensions, but got array with shape (32, 16)

编辑2:输入信息:

train_generator=datagen.flow_from_dataframe(dataframe=traindf,directory="data_final",x_col="path",y_col="label",subset="training",batch_size=32,seed=42,shuffle=True,class_mode="categorical",target_size=(256,256))

valid_generator=datagen.flow_from_dataframe(dataframe=traindf,directory="data_final",x_col="path",y_col="label",subset="validation",batch_size=32,seed=42,shuffle=True,class_mode="categorical",target_size=(256,256))

if K.image_data_format()=="channels_first":
  inputshape=(3,imrows,imcols)
else:
  inputshape=(imrows,imcols,3)

Answer 1

keras 的函数式 API 更适合解决此类问题:

model = VGG16(weights='imagenet', include_top=False, input_shape=inputshape)
x = model.output
x = Conv2D(4096, kernel_size=(8, 8), activation="relu")(x)
x = Conv2D(4096, kernel_size=(1, 1), activation="relu")(x)
out = Conv2D(16, kernel_size=(1, 1), activation="softmax")(x)

F2model = Model(inputs=model.inputs, outputs=out)


for layer in F2model.layers[:25]:
    layer.trainable = False

此外，我发现您正在使用带有 softmax 激活的 binary_crossentropy，这可能会导致一些问题:
- 使用softmax和categorical_crossentropy
- 使用sigmoid和binary_crossentropy

并且要小心这个模型，使用 4096 的卷积将使你的参数数量真的非常大!!

(本例中为 1.65 亿)

编辑

看来您的问题仅来自您的标签数组:

• 你的最后一层是一个卷积层，所以它需要一个形状为(batch_size，height，width，channel)的4D数组，但你给它一个形状为的数组(批量大小，16)

• 因此，要么将最后一层更改为:

out = Dense(16, activation="softmax")(x)

• 或者更改标签数组以适应卷积层。