python-3.x - 如何将特异性定义为模型评估的可调用评分器

Question

我正在使用此代码来比较多个模型的性能:

from sklearn import model_selection

X = input data
Y = binary labels

models = []
models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('CART', DecisionTreeClassifier()))

results = []
names = []
scoring = 'accuracy'

for name, model in models:
    kfold = model_selection.KFold(n_splits=10, random_state=7)
    cv_results = model_selection.cross_val_score(model, X, Y, cv=kfold,scoring=scoring)
    results.append(cv_results)
    names.append(name)
    msg = "%s: %.2f (%.2f)" % (name, cv_results.mean(), cv_results.std())
    print(msg)

我可以使用“准确性”和“记忆”作为评分，这些将提供准确性和敏感性。我怎样才能创建一个给我“特异性”的记分员

特异性= TN/(TN+FP)

其中 TN 和 FP 是混淆矩阵中的真负值和假正值

我试过这个

def tp(y_true, y_pred): 
error= confusion_matrix(y_true, y_pred)[0,0]/(confusion_matrix(y_true,y_pred)[0,0] + confusion_matrix(y_true, y_pred)[0,1])
return error

my_scorer = make_scorer(tp, greater_is_better=True)

进而

cv_results = model_selection.cross_val_score(model, X,Y,cv=kfold,scoring=my_scorer)

但它不适用于 n_split >=10
我在计算 my_scorer 时收到此错误

索引错误:索引 1 超出轴 1 的范围，大小为 1

Answer 1

如果您更改 recall_score二元分类器的参数 pos_label=0你得到特异性(默认是敏感性， pos_label=1 )

scoring = {
    'accuracy': make_scorer(accuracy_score),
    'sensitivity': make_scorer(recall_score),
    'specificity': make_scorer(recall_score,pos_label=0)
}