python - 进度条中显示的keras准确度是如何计算的？它是根据哪些输入计算的？如何复制它？

Question

我试图了解纪元结束时 keras 进度条中显示的准确度“acc”是多少:

13/13 [==============================] - 0s 76us/step - loss: 0.7100 - acc: 0.4615

在一个时期结束时，它应该是所有训练样本的模型预测的准确性。然而，当在相同的训练样本上评估模型时，实际精度可能会有很大差异。

下面是 MLP for binary classification from keras webpage 的改编示例。一个简单的顺序神经网络正在对随机生成的数字进行二元分类。批量大小与训练示例的数量 (13) 相同，因此每个 epoch 只包含一个步骤。由于损失设置为binary_crossentropy，因此使用metrics.py中定义的binary_accuracy来计算精度。 。 MyEval 类定义回调，该回调在每个纪元结束时调用。它使用两种方法计算训练数据的准确性:a) 模型评估和 b) 模型预测来获得预测，然后与 keras binary_accuracy 函数中使用的代码几乎相同。这两个精度是一致的，但是大多数时候和进度条上的不一样。为什么它们不同？是否可以计算出与进度条中相同的精度？还是我的假设有误？

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras import callbacks

np.random.seed(1) # fix random seed for reproducibility 
# Generate dummy data
x_train = np.random.random((13, 20))
y_train = np.random.randint(2, size=(13, 1))

model = Sequential()
model.add(Dense(64, input_dim=20, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

class MyEval(callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        my_accuracy_1 = self.model.evaluate(x_train, y_train, verbose=0)[1]
        y_pred = self.model.predict(x_train)
        my_accuracy_2 = np.mean(np.equal(y_train, np.round(y_pred)))
        print("my accuracy 1: {}".format(my_accuracy_1))
        print("my accuracy 2: {}".format(my_accuracy_2))

my_eval = MyEval()

model.fit(x_train, y_train,
          epochs=5,
          batch_size=13,
          callbacks=[my_eval],
          shuffle=False)

上述代码的输出:

13/13 [==============================] - 0s 25ms/step - loss: 0.7303 - acc: 0.5385
my accuracy 1: 0.5384615659713745
my accuracy 2: 0.5384615384615384
Epoch 2/5
13/13 [==============================] - 0s 95us/step - loss: 0.7412 - acc: 0.4615
my accuracy 1: 0.9230769276618958
my accuracy 2: 0.9230769230769231
Epoch 3/5
13/13 [==============================] - 0s 77us/step - loss: 0.7324 - acc: 0.3846
my accuracy 1: 0.9230769276618958
my accuracy 2: 0.9230769230769231
Epoch 4/5
13/13 [==============================] - 0s 72us/step - loss: 0.6543 - acc: 0.5385
my accuracy 1: 0.9230769276618958
my accuracy 2: 0.9230769230769231
Epoch 5/5
13/13 [==============================] - 0s 76us/step - loss: 0.6459 - acc: 0.6923
my accuracy 1: 0.8461538553237915
my accuracy 2: 0.8461538461538461

使用:Python 3.5.2，tensorflow-gpu==1.14.0 Keras==2.2.4 numpy==1.15.2

Answer 1

我认为这与Dropout的使用有关。 Dropout 仅在训练期间启用，但在评估或预测期间不启用。因此，训练和评估/预测期间的准确性存在差异。

此外，栏中显示的训练准确度显示了训练时期的平均准确度，即每个批处理后计算的批处理准确度的平均值。请记住，模型参数在每个批处理后都会进行调整，因此最后栏中显示的准确度与纪元完成后验证的准确度并不完全匹配(因为训练准确度是使用每个批处理的不同模型参数计算的)批处理，验证准确度是使用所有批处理的相同参数计算的)。

这是您的示例，具有更多数据(因此超过一个时期)，并且没有丢失:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras import callbacks

np.random.seed(1) # fix random seed for reproducibility 
# Generate dummy data
x_train = np.random.random((200, 20))
y_train = np.random.randint(2, size=(200, 1))

model = Sequential()
model.add(Dense(64, input_dim=20, activation='relu'))
# model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
# model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

class MyEval(callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        my_accuracy_1 = self.model.evaluate(x_train, y_train, verbose=0)[1]
        y_pred = self.model.predict(x_train)
        my_accuracy_2 = np.mean(np.equal(y_train, np.round(y_pred)))
        print("my accuracy 1 after epoch {}: {}".format(epoch + 1,my_accuracy_1))
        print("my accuracy 2 after epoch {}: {}".format(epoch + 1,my_accuracy_2))


my_eval = MyEval()

model.fit(x_train, y_train,
          epochs=5,
          batch_size=13,
          callbacks=[my_eval],
          shuffle=False)

输出内容为:

Train on 200 samples
Epoch 1/5
my accuracy 1 after epoch 1: 0.5450000166893005
my accuracy 2 after epoch 1: 0.545
200/200 [==============================] - 0s 2ms/sample - loss: 0.6978 - accuracy: 0.5350
Epoch 2/5
my accuracy 1 after epoch 2: 0.5600000023841858
my accuracy 2 after epoch 2: 0.56
200/200 [==============================] - 0s 383us/sample - loss: 0.6892 - accuracy: 0.5550
Epoch 3/5
my accuracy 1 after epoch 3: 0.5799999833106995
my accuracy 2 after epoch 3: 0.58
200/200 [==============================] - 0s 496us/sample - loss: 0.6844 - accuracy: 0.5800
Epoch 4/5
my accuracy 1 after epoch 4: 0.6000000238418579
my accuracy 2 after epoch 4: 0.6
200/200 [==============================] - 0s 364us/sample - loss: 0.6801 - accuracy: 0.6150
Epoch 5/5
my accuracy 1 after epoch 5: 0.6050000190734863
my accuracy 2 after epoch 5: 0.605
200/200 [==============================] - 0s 393us/sample - loss: 0.6756 - accuracy: 0.6200

该纪元之后的验证准确度与现在该纪元结束时的平均训练准确度非常相似。