python - Scikit-learn 具有不同评分者的学习曲线，并排除一组 cv 产生相同的值

Question

我想使用不同的方法在经过训练的 SVM 分类器上绘制学习曲线分数，并使用 Leave One Group Out 作为交叉验证的方法。我我以为我已经弄清楚了，但是两个不同的得分手 - 'f1_micro' 和'accuracy' - 将产生相同的值。我很困惑，这是应该的吗是这样吗？

这是我的代码(不幸的是，我无法共享数据，因为它未开放):

from sklearn import svm
SVC_classifier_LOWO_VC0 = svm.SVC(cache_size=800, class_weight=None,
  coef0=0.0, decision_function_shape=None, degree=3, gamma=0.01,  
  kernel='rbf', max_iter=-1, probability=False, random_state=1, 
  shrinking=True, tol=0.001, verbose=False)
training_data = pd.read_csv('training_data.csv')
X = training_data.drop(['Groups', 'Targets'], axis=1).values
scaler = preprocessing.StandardScaler().fit(X)
X = scaler.transform(X)
y = training_data['Targets'].values
groups = training_data["Groups"].values
Fscorer = make_scorer(f1_score, average = 'micro')
logo = LeaveOneGroupOut()
parm_range0 = np.logspace(-2, 6, 9)
train_scores0, test_scores0 = validation_curve(SVC_classifier_LOWO_VC0, X,
  y, "C", parm_range0, cv =logo.split(X, y, groups=groups), scoring = Fscorer)

现在，来自:

train_scores_mean0 = np.mean(train_scores0, axis=1)
train_scores_std0 = np.std(train_scores0, axis=1)
test_scores_mean0 = np.mean(test_scores0, axis=1)
test_scores_std0 = np.std(test_scores0, axis=1)
print test_scores_mean0
print np.amax(test_scores_mean0)
print  np.logspace(-2, 6, 9)[test_scores_mean0.argmax(axis=0)]

我得到:

[ 0.20257407 0.35551122 0.40791047 0.49887676 0.5021742
0.50030438 0.49426622 0.48066419 0.4868987 ]

0.502174200206

100.0

如果我创建一个新的分类器，但具有相同的参数，然后运行一切都和以前一样，除了评分，例如:

parm_range1 = np.logspace(-2, 6, 9)
train_scores1, test_scores1 = validation_curve(SVC_classifier_LOWO_VC1, X,
y, "C", parm_range1, cv =logo.split(X, y, groups=groups), scoring =
'accuracy')
train_scores_mean1 = np.mean(train_scores1, axis=1)
train_scores_std1= np.std(train_scores1, axis=1)
test_scores_mean1 = np.mean(test_scores1, axis=1)
test_scores_std1 = np.std(test_scores1, axis=1)
print test_scores_mean1
print np.amax(test_scores_mean1)
print  np.logspace(-2, 6, 9)[test_scores_mean1.argmax(axis=0)]

我得到了完全相同的答案:

[ 0.20257407 0.35551122 0.40791047 0.49887676 0.5021742
0.50030438 0.49426622 0.48066419 0.4868987 ]

0.502174200206

100.0

这怎么可能，我做错了什么，或者错过了什么吗？

谢谢

Answer 1

F1 = precision 当且仅当 TP = TN，即真阳性数等于真阴性数，如果您的类别是完美平衡。所以要么就是这样，要么你的代码中有错误。您在哪里初始化记分器，如下所示:scorer = make_scorer(accuracy_score,average = 'micro')？