python - 如何计算神经网络模型中的精度、召回率和 F1 分数？

Question

我使用 Keras 来预测输出是 1 还是 0。数据如下所示:

    funded_amnt  emp_length  avg_cur_bal  num_actv_rev_tl    loan_status
    10000       5.60088      19266                 2                  1
    13750       5.60088      2802                  6                  0
    26100       10.0000      19241                17                  1

目标是loan_status，功能是其余的。在开始构建神经网络模型之前，我已经对数据进行了标准化。

这是我的训练和测试数据的形状:

    print(X_train.shape,Y_train.shape) 
    # Output: (693, 4) (693,)

    print(X_test.shape,Y_test.shape) 
    # Output: (149, 4) (149,)

我构建神经网络的过程是:

     # define the keras model
     model = Sequential()
     model.add(Dense(4, input_dim=4,activation='relu'))
     model.add(Dense(4 ,activation='relu'))
     model.add(Dense(1,activation='sigmoid'))

     # compile the keras model
     model.compile(loss='binary_crossentropy',optimizer='adam', metrics=['accuracy'])

     # fit the keras model on the dataset
     hist = model.fit(X_train, Y_train, validation_data=(X_test, Y_test) ,epochs=10, batch_size=2)

运行hist后的输出:

Epoch 1/20
693/693 [==============================] - 1s 2ms/step - loss: 0.5974 - acc: 0.7605 - val_loss: 0.5499 - val_acc: 0.7785
Epoch 2/20
693/693 [==============================] - 0s 659us/step - loss: 0.5369 - acc: 0.7778 - val_loss: 0.5380 - val_acc: 0.7785
Epoch 3/20
693/693 [==============================] - 0s 700us/step - loss: 0.5330 - acc: 0.7778 - val_loss: 0.5369 - val_acc: 0.7785
Epoch 4/20
693/693 [==============================] - 0s 670us/step - loss: 0.5316 - acc: 0.7778 - val_loss: 0.5355 - val_acc: 0.7785
Epoch 5/20
693/693 [==============================] - 0s 720us/step - loss: 0.5307 - acc: 0.7778 - val_loss: 0.5345 - val_acc: 0.7785
Epoch 6/20
693/693 [==============================] - 0s 668us/step - loss: 0.5300 - acc: 0.7778 - val_loss: 0.5339 - val_acc: 0.7785
Epoch 7/20

现在，我想计算精度、召回率和F1分数，而不仅仅是准确率。我尝试过关注 this 。但我不断收到以下错误:

ValueError: Classification metrics can't handle a mix of binary and continuous targets

还有别的办法吗？

Answer 1

您可以尝试以下操作吗:

import numpy as np
from keras.callbacks import Callback
from sklearn.metrics import confusion_matrix, f1_score, precision_score, recall_score


class Metrics(Callback):

    def on_train_begin(self, logs={}):
        self.val_f1s = []
        self.val_recalls = []
        self.val_precisions = []

    def on_epoch_end(self, epoch, logs={}):
        val_predict = (np.asarray(self.model.predict(
            self.model.validation_data[0]))).round()
        val_targ = self.model.validation_data[1]
        _val_f1 = f1_score(val_targ, val_predict)
        _val_recall = recall_score(val_targ, val_predict)
        _val_precision = precision_score(val_targ, val_predict)
        self.val_f1s.append(_val_f1)
        self.val_recalls.append(_val_recall)
        self.val_precisions.append(_val_precision)
        print(f" — val_f1: {_val_f1} — val_precision: {_val_precision} — val_recall _val_recall")
        return


metrics = Metrics()

在您的代码中，您需要按如下方式使用它:

hist = model.fit(X_train, Y_train,
                 validation_data=(X_test, Y_test), epochs=10,
                 batch_size=2, callbacks=[metrics])