machine-learning - Keras 卷积网络在 CIFAR-10 数据集上得分较低

Question

我正在尝试在 Keras 中的 CIFAR-10 数据集上训练 CNN，但只能获得 10% 左右的准确率，基本上是随机的。我正在训练超过 50 个 epoch，批量大小为 32，学习率为 0.01。我有什么特别做错的地方吗？

import os
import numpy as np
import pandas as pd
from PIL import Image

from keras.models import Model
from keras.layers import Input, Dense, Conv2D, MaxPool2D, Dropout, Flatten
from keras.optimizers import SGD
from keras.utils import np_utils


# trainingData = np.array([np.array(Image.open("train/" + f)) for f in os.listdir("train")]) #shape: 50k, 32, 32, 3
# testingData = np.array([np.array(Image.open("test/" + f)) for f in os.listdir("test")]) #shape: same as training
#
# trainingLabels = np.array(pd.read_csv("trainLabels.csv"))[:,1] #categorical labels ["dog", "cat", "etc"....]
# listOfLabels = sorted(list(set(trainingLabels)))
# trainingOutput = np.array([np.array([1.0 if label == ind else 0.0 for ind in listOfLabels]) for label in trainingLabels]) #converted to output
#                                                                                                                            #for example: training output for dog =
#                                                                                                                                  #[1.0, 0.0, 0.0, ...]
# np.save("trainingInput.np", trainingData)
# np.save("testingInput.np", testingData)
# np.save("trainingOutput.np", trainingOutput)


trainingInput = np.load("trainingInput.npy") #shape = 50k, 32, 32, 3
testingInput = np.load("testingInput.npy") #shape = 10k, 32, 32, 3

listOfLabels = sorted(list(set(np.array(pd.read_csv("trainLabels.csv"))[:,1]))) #categorical list of labels as strings
trainingOutput = np.load("trainingOutput.npy") #shape = 50k, 10
                                                    #looks like [0.0, 1.0, 0.0 ... 0.0, 0.0]

print(listOfLabels)

print("Data loaded\n______________\n")


inp = Input(shape=(32, 32, 3))
conva1 = Conv2D(64, (3, 3), padding='same', activation='relu')(inp)
conva2 = Conv2D(64, (3, 3), padding='same', activation='relu')(conva1)
poola = MaxPool2D(pool_size=(3, 3))(conva2)
dropa = Dropout(0.1)(poola)

convb1 = Conv2D(128, (5, 5), padding='same', activation='relu')(dropa)
convb2 = Conv2D(128, (5, 5), padding='same', activation='relu')(convb1)
poolb = MaxPool2D(pool_size=(3, 3))(convb2)
dropb = Dropout(0.1)(poolb)

flat = Flatten()(dropb)
dropc = Dropout(0.5)(flat)
out = Dense(len(listOfLabels), activation='softmax')(dropc)
print(out.shape)
model = Model(inputs=inp, outputs=out)
lrSet = SGD(lr=0.01, clipvalue=0.5)
model.compile(loss='categorical_crossentropy', optimizer=lrSet, metrics=['accuracy'])
model.fit(trainingInput, trainingOutput, batch_size=32, epochs=50, verbose=1, validation_split=0.1)
print(model.predict(testingInput))

Answer 1

Is there anything in particular that I am doing wrong?

不一定是“错误”，但我可以建议的一些建议是:

1. 重新调整数据非常重要，以防万一您不这样做。与其处理 [0,255] 范围内的值，不如将所有值除以 255 并处理 [0,1] 范围内的数据。这有助于模型的权重更快地收敛，因为每次梯度更新都比未缩放的版本更重要。

2. 我认为您的退学可能会影响您的表现。更重要的是，在将数据传递到输出时，您正在使用 CNN 和强 (0.5) Dropout。引用this很好的答案:

   In the original paper that proposed dropout layers, by Hinton (2012), dropout (with p=0.5) was used on each of the fully connected (dense) layers before the output; it was not used on the convolutional layers. This became the most commonly used configuration.

   More recent research has shown some value in applying dropout also to convolutional layers, although at much lower levels: p=0.1 or 0.2.

   所以也许减少你的辍学或者稍微尝试一下会产生更好的结果。请注意，您正在对数据进行连续丢失，在我看来，这似乎不太有帮助，而且还可能导致问题，因此请考虑重新设计该部分:

   dropb = Dropout(0.1)(poolb) #drop
   flat = Flatten()(dropb) #flatten 
   dropc = Dropout(0.5)(flat) #then drop again?
   

3. 您的学习率可能高于通常使用的学习率。尽管这是 SGD 的默认学习率，但如果学习值较高，您可能会“仓促”训练，而无法找到可以产生更好性能的更好最小值。考虑使用较低的学习率(0.001或更低，根据需要调整时期)，或者在 SGD 上添加权重衰减实例。这将防止您的模型陷入局部最小值而导致次优结果。