machine-learning - 验证准确性停滞，而训练准确性提高

Question

我对深度学习还很陌生，所以如果我遗漏了一些明显的东西，我很抱歉。

我目前正在使用我整理的数据集训练 CNN。训练时，训练准确率表现得非常正常，并且有所提高，达到 >99% 的准确率。我的验证准确率一开始约为 75%，并在 81% ± 1% 左右波动。训练后，模型在全新数据上表现非常好。

Epoch 1/100
187/187 [==============================] - 103s 550ms/step - loss: 1.1336 - acc: 0.5384 - val_loss: 0.8065 - val_acc: 0.7405
Epoch 2/100
187/187 [==============================] - 97s 519ms/step - loss: 0.8041 - acc: 0.7345 - val_loss: 0.7566 - val_acc: 0.7720
Epoch 3/100
187/187 [==============================] - 97s 519ms/step - loss: 0.7194 - acc: 0.7945 - val_loss: 0.7410 - val_acc: 0.7846
Epoch 4/100
187/187 [==============================] - 97s 517ms/step - loss: 0.6688 - acc: 0.8324 - val_loss: 0.7295 - val_acc: 0.7924
Epoch 5/100
187/187 [==============================] - 97s 518ms/step - loss: 0.6288 - acc: 0.8611 - val_loss: 0.7197 - val_acc: 0.7961
Epoch 6/100
187/187 [==============================] - 96s 515ms/step - loss: 0.5989 - acc: 0.8862 - val_loss: 0.7252 - val_acc: 0.7961
Epoch 7/100
187/187 [==============================] - 96s 514ms/step - loss: 0.5762 - acc: 0.8981 - val_loss: 0.7135 - val_acc: 0.8063
Epoch 8/100
187/187 [==============================] - 97s 518ms/step - loss: 0.5513 - acc: 0.9186 - val_loss: 0.7089 - val_acc: 0.8077
Epoch 9/100
187/187 [==============================] - 96s 513ms/step - loss: 0.5351 - acc: 0.9280 - val_loss: 0.7113 - val_acc: 0.8053
Epoch 10/100
187/187 [==============================] - 96s 514ms/step - loss: 0.5189 - acc: 0.9417 - val_loss: 0.7167 - val_acc: 0.8094
Epoch 11/100
187/187 [==============================] - 96s 515ms/step - loss: 0.5026 - acc: 0.9483 - val_loss: 0.7104 - val_acc: 0.8162
Epoch 12/100
187/187 [==============================] - 96s 516ms/step - loss: 0.4914 - acc: 0.9538 - val_loss: 0.7114 - val_acc: 0.8101
Epoch 13/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4809 - acc: 0.9583 - val_loss: 0.7099 - val_acc: 0.8141
Epoch 14/100
187/187 [==============================] - 96s 512ms/step - loss: 0.4681 - acc: 0.9656 - val_loss: 0.7149 - val_acc: 0.8182
Epoch 15/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4605 - acc: 0.9701 - val_loss: 0.7139 - val_acc: 0.8172
Epoch 16/100
187/187 [==============================] - 96s 514ms/step - loss: 0.4479 - acc: 0.9753 - val_loss: 0.7102 - val_acc: 0.8182
Epoch 17/100
187/187 [==============================] - 96s 513ms/step - loss: 0.4418 - acc: 0.9805 - val_loss: 0.7087 - val_acc: 0.8247
Epoch 18/100
187/187 [==============================] - 96s 512ms/step - loss: 0.4363 - acc: 0.9809 - val_loss: 0.7148 - val_acc: 0.8213
Epoch 19/100
187/187 [==============================] - 96s 516ms/step - loss: 0.4225 - acc: 0.9870 - val_loss: 0.7184 - val_acc: 0.8203
Epoch 20/100
187/187 [==============================] - 96s 513ms/step - loss: 0.4241 - acc: 0.9863 - val_loss: 0.7216 - val_acc: 0.8189
Epoch 21/100
187/187 [==============================] - 96s 513ms/step - loss: 0.4132 - acc: 0.9908 - val_loss: 0.7143 - val_acc: 0.8199
Epoch 22/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4050 - acc: 0.9936 - val_loss: 0.7109 - val_acc: 0.8233
Epoch 23/100
187/187 [==============================] - 96s 515ms/step - loss: 0.4040 - acc: 0.9928 - val_loss: 0.7118 - val_acc: 0.8203
Epoch 24/100
187/187 [==============================] - 96s 511ms/step - loss: 0.3989 - acc: 0.9930 - val_loss: 0.7194 - val_acc: 0.8165
Epoch 25/100
187/187 [==============================] - 97s 517ms/step - loss: 0.3933 - acc: 0.9946 - val_loss: 0.7163 - val_acc: 0.8155
Epoch 26/100
187/187 [==============================] - 97s 516ms/step - loss: 0.3884 - acc: 0.9957 - val_loss: 0.7225 - val_acc: 0.8148
Epoch 27/100
187/187 [==============================] - 95s 510ms/step - loss: 0.3876 - acc: 0.9959 - val_loss: 0.7224 - val_acc: 0.8179

情节本身看起来像是过度拟合，但我已经采取了很多措施来修复过度拟合，但似乎都不起作用。这是我的模型:

# transfer learning with ResNet50
base_model=ResNet50(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

# function to finetune model
def build_finetune_model(base_model, dropout, fc_layers, num_classes):
    # make base model untrainable
    for layer in base_model.layers:
        layer.trainable = False

    x = base_model.output
    x = Flatten()(x)

    # add dense layers
    for fc in fc_layers:
        # use regularizer
        x = Dense(fc, use_bias=False, kernel_regularizer=l2(0.003))(x)
        # add batch normalization
        x = BatchNormalization()(x)
        x = Activation('relu')(x)
        # add dropout
        x = Dropout(dropout)(x)

    # New softmax layer
    x = Dense(num_classes, use_bias=False)(x) 
    x = BatchNormalization()(x)
    predictions = Activation('softmax')(x)

    finetune_model = Model(inputs=base_model.input, outputs=predictions)
    return finetune_model


FC_LAYERS = [1024, 1024]
dropout = 0.5

model = build_finetune_model(base_model, dropout=dropout, fc_layers=FC_LAYERS,num_classes=len(categories))

我正在调整类(class)权重，并设置了非常低的学习率，希望能减慢学习速度。

model.compile(optimizer=Adam(lr=0.000005),loss='categorical_crossentropy',metrics=['accuracy'], weighted_metrics=class_weight)

令我感到非常困惑的是，验证准确度一开始就如此之高(明显高于训练准确度)，并且在整个训练过程中几乎没有提高。正如之前提到的，它似乎过度拟合，但我添加了 dropout、批量归一化和正则化器，它似乎不起作用。通过水平翻转、随机裁剪、随机亮度和旋转来增强数据也不会显着改变准确性。为我的训练数据关闭 ImageDataGenerator().flow_from_directory() 中的数据随机播放，使模型训练的训练准确度约为 25%，验证准确度<50%(编辑:准确度似乎如此低，因为在这种情况下学习率太低)。

同样，该模型在新的测试数据上表现得非常好。我希望提高验证准确性，并希望了解神经网络为何会出现这种行为。

Answer 1

您的模型过度拟合。您可能想在图像模型上使用数据增强。例如使用 ImageDataGenerator ( https://keras.io/preprocessing/image/ ) 随机移动、旋转和裁剪图像。

SGD 试图找到尽可能最简单的方法来最小化数据集上的损失函数；给定足够大的数据点集，它被迫提出一个通用的解决方案；但只要有可能，DNN 就会倾向于“记住”输入，因为这是减少损失的最简单方法。退出和正则化确实有帮助，但最终重要的是验证指标。当然，假设您的验证集正确平衡。