python - Keras VGG16 相同模型不同方法给出不同结果

Question

我基于VGG16模型构建keras CNN来对花卉进行分类，数据集是 here 。我构建了两个具有相同架构和参数总和但方法不同的模型。一种使用 Model (功能 API)，另一种使用 Sequential 。顺序给了我很好的结果(84% val_acc)，但模型给了我很差的结果(50% val_acc)。我希望有人能指出有什么区别。谢谢!

顺序

import tensorflow as tf
import keras
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential, Model
from keras.layers import Input, Dense, Flatten, Dropout, GlobalAveragePooling2D
from keras import backend as K
from keras import optimizers
from keras.callbacks import ModelCheckpoint
from keras.callbacks import TensorBoard
import numpy as np
import time

## image path
train_data_dir = 'dataset/training_set'
validation_data_dir = 'dataset/test_set'
## other
img_width, img_height = 299, 299
nb_train_samples = 100
nb_validation_samples = 800
top_epochs = 50
fit_epochs = 50
batch_size = 24
nb_classes = 5
nb_epoch = 10

#　start measurement
start = time.time()

# import vgg16 model
input_tensor = Input(shape=(img_width, img_height, 3))
vgg16 = keras.applications.VGG16(weights='imagenet', include_top=False, input_tensor=input_tensor)

# creating an FC layer
top_model = Sequential()
top_model.add(Flatten(input_shape=vgg16.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(nb_classes, activation='softmax'))
top_model.summary()
# bound VGG 16 and FC layer
vgg_model = Model(inputs=vgg16.input, outputs=top_model(vgg16.output))

print(vgg_model.layers[:15])
# prevent re-learning of the layer before the last convolution layer
for layer in vgg_model.layers[:15]:
    layer.trainable = False
vgg_model.summary()
# create model
vgg_model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.SGD(lr=1e-3, momentum=0.9),
              metrics=['accuracy']
)

# Setting learning data
train_datagen = ImageDataGenerator(rescale=1.0 / 255, zoom_range=0.2, horizontal_flip=True)
validation_datagen = ImageDataGenerator(rescale=1.0 / 255)

train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        color_mode='rgb',
        class_mode='categorical',
        batch_size=batch_size,
        shuffle=True
)

validation_generator = validation_datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_width, img_height),
        color_mode='rgb',
        class_mode='categorical',
        batch_size=batch_size,
        shuffle=True
)

history = vgg_model.fit_generator(
        train_generator,
        steps_per_epoch=nb_train_samples,
        epochs=nb_epoch,
        validation_data=validation_generator,
        validation_steps=nb_validation_samples
)

顺序网络

Layer (type)                 Output Shape              Param #   
=================================================================
input_10 (InputLayer)        (None, 299, 299, 3)       0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 299, 299, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 299, 299, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 149, 149, 64)      0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 149, 149, 128)     73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 149, 149, 128)     147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 74, 74, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 74, 74, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 74, 74, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 74, 74, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 37, 37, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 37, 37, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 37, 37, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 37, 37, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 18, 18, 512)       0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 18, 18, 512)       2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 18, 18, 512)       2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 18, 18, 512)       2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 9, 9, 512)         0         
_________________________________________________________________
sequential_6 (Sequential)    (None, 5)                 10618373  
=================================================================
Total params: 25,333,061
Trainable params: 17,697,797
Non-trainable params: 7,635,264

顺序 - 结果

Epoch 1/10
100/100 [==============================] - 50s 498ms/step - loss: 1.2821 - acc: 0.4912 - val_loss: 0.7209 - val_acc: 0.7327
Epoch 2/10
100/100 [==============================] - 48s 477ms/step - loss: 0.5827 - acc: 0.7787 - val_loss: 0.5326 - val_acc: 0.7816
Epoch 3/10
100/100 [==============================] - 47s 466ms/step - loss: 0.5355 - acc: 0.8101 - val_loss: 0.4951 - val_acc: 0.8150
Epoch 4/10
100/100 [==============================] - 46s 458ms/step - loss: 0.4020 - acc: 0.8612 - val_loss: 0.4458 - val_acc: 0.8413
Epoch 5/10
100/100 [==============================] - 49s 485ms/step - loss: 0.3465 - acc: 0.8767 - val_loss: 0.3904 - val_acc: 0.8496
Epoch 6/10
100/100 [==============================] - 46s 460ms/step - loss: 0.3330 - acc: 0.8747 - val_loss: 0.3961 - val_acc: 0.8568
Epoch 7/10
100/100 [==============================] - 45s 448ms/step - loss: 0.3188 - acc: 0.8896 - val_loss: 0.4462 - val_acc: 0.8389
Epoch 8/10
100/100 [==============================] - 47s 472ms/step - loss: 0.2302 - acc: 0.9208 - val_loss: 0.4048 - val_acc: 0.8568
Epoch 9/10
100/100 [==============================] - 45s 453ms/step - loss: 0.2172 - acc: 0.9192 - val_loss: 0.4101 - val_acc: 0.8795
Epoch 10/10
100/100 [==============================] - 45s 453ms/step - loss: 0.1867 - acc: 0.9321 - val_loss: 0.3337 - val_acc: 0.8878

型号

from keras.applications.vgg16 import VGG16
from keras.preprocessing.image import ImageDataGenerator
from keras.layers import Input, Flatten, Dense, Dropout
from keras.models import Model
from keras import optimizers
train_data_dir = 'dataset/training_set'
validation_data_dir = 'dataset/test_set'
## other
img_width, img_height = 299, 299
nb_train_samples = 100
nb_validation_samples = 800
top_epochs = 50
fit_epochs = 50
batch_size = 24
nb_classes = 5
nb_epoch = 10

#build CNN

model_vgg16_conv = VGG16(weights='imagenet', include_top=False)

input = Input(shape=(299,299, 3),name = 'image_input')

output_vgg16_conv = model_vgg16_conv(input)

for layer in model_vgg16_conv.layers[:15]:
    layer.trainable = False
model_vgg16_conv.summary()

x = Flatten(name='flatten')(output_vgg16_conv)
x = Dense(256, activation='softmax')(x)
x = Dropout(0.5)(x)
x = Dense(5, activation='softmax', name='predictions')(x)

vgg_model = Model(inputs=input, outputs=x)

vgg_model.summary()


#Image preprocessing and image augmentation with keras
vgg_model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.SGD(lr=1e-3, momentum=0.9),
              metrics=['accuracy']
)

# Setting learning data
train_datagen = ImageDataGenerator(rescale=1.0 / 255, zoom_range=0.2, horizontal_flip=True)
validation_datagen = ImageDataGenerator(rescale=1.0 / 255)

train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        color_mode='rgb',
        class_mode='categorical',
        batch_size=batch_size,
        shuffle=True
)

validation_generator = validation_datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_width, img_height),
        color_mode='rgb',
        class_mode='categorical',
        batch_size=batch_size,
        shuffle=True
)

history = vgg_model.fit_generator(
        train_generator,
        steps_per_epoch=nb_train_samples,
        epochs=nb_epoch,
        validation_data=validation_generator,
        validation_steps=nb_validation_samples
)

模型网络

    Layer (type)                 Output Shape              Param #   
    =================================================================
    image_input (InputLayer)     (None, 299, 299, 3)       0         
    _________________________________________________________________
    vgg16 (Model)                multiple                  14714688  
    _________________________________________________________________
    flatten (Flatten)            (None, 41472)             0         
    _________________________________________________________________
    dense_16 (Dense)             (None, 256)               10617088  
    _________________________________________________________________
    dropout_10 (Dropout)         (None, 256)               0         
    _________________________________________________________________
    predictions (Dense)          (None, 5)                 1285      
    =================================================================
    Total params: 25,333,061
    Trainable params: 17,697,797
    Non-trainable params: 7,635,264

模型结果

Epoch 1/10
100/100 [==============================] - 48s 484ms/step - loss: 1.6028 - acc: 0.2379 - val_loss: 1.5978 - val_acc: 0.1814
Epoch 2/10
100/100 [==============================] - 47s 470ms/step - loss: 1.5758 - acc: 0.3098 - val_loss: 1.5577 - val_acc: 0.3258
Epoch 3/10
100/100 [==============================] - 45s 455ms/step - loss: 1.5352 - acc: 0.3386 - val_loss: 1.5273 - val_acc: 0.3496
Epoch 4/10
100/100 [==============================] - 45s 453ms/step - loss: 1.4991 - acc: 0.3425 - val_loss: 1.4890 - val_acc: 0.3914
Epoch 5/10
100/100 [==============================] - 47s 472ms/step - loss: 1.4600 - acc: 0.3826 - val_loss: 1.4406 - val_acc: 0.4523
Epoch 6/10
100/100 [==============================] - 46s 456ms/step - loss: 1.4252 - acc: 0.4021 - val_loss: 1.4337 - val_acc: 0.4165
Epoch 7/10
100/100 [==============================] - 45s 453ms/step - loss: 1.3944 - acc: 0.4037 - val_loss: 1.3720 - val_acc: 0.4964
Epoch 8/10
100/100 [==============================] - 48s 479ms/step - loss: 1.3787 - acc: 0.4193 - val_loss: 1.3615 - val_acc: 0.4988
Epoch 9/10
100/100 [==============================] - 46s 464ms/step - loss: 1.3590 - acc: 0.4067 - val_loss: 1.3272 - val_acc: 0.4952
Epoch 10/10
100/100 [==============================] - 45s 449ms/step - loss: 1.3419 - acc: 0.4244 - val_loss: 1.3038 - val_acc: 0.5060

Answer 1

Dense 中的 softmax 单元是 sigmoid 函数的集合。它的工作方式类似于多类分类器，其工作原理是每个类一个分类器。 Sigmoid 非常适合识别 1 或 0 等二进制输出。因此，softmax 对于输出层非常有用，但不如中间层那么好。

深入的解释是，relu 单元上的反向传播保留了中间特征，而 softmax 在这方面做得不那么好，但做得更好在输出层。

这就是区别

top_model = Sequential()

top_model.add(Flatten(input_shape=vgg16.output_shape[1:]))

top_model.add(Dense(256, activation='relu'))

top_model.add(Dropout(0.5))

top_model.add(Dense(nb_classes,activation='softmax'))

top_model.summary()

鉴于

x = Flatten(name='flatten')(output_vgg16_conv)

x = Dense(256, activation='softmax')(x)

x = Dropout(0.5)(x)

x = Dense(5, activation='softmax', name='predictions')(x)

很高兴看到您正在通过迁移学习重新训练 imagenet! :)

请告诉我们这是否解决了问题，或者如果还需要其他内容，请发表评论!