python - 如何进行 K 折交叉验证以拆分训练集和测试集？

Question

我有一套文件和一套标签。现在，我正在使用 train_test_split 以 90:10 的比例拆分我的数据集。但是，我希望使用 Kfold 交叉验证。

train=[]

with open("/Users/rte/Documents/Documents.txt") as f:
    for line in f:
        train.append(line.strip().split())

labels=[]
with open("/Users/rte/Documents/Labels.txt") as t:
    for line in t:
        labels.append(line.strip().split())

X_train, X_test, Y_train, Y_test= train_test_split(train, labels, test_size=0.1, random_state=42)

当我尝试 scikit 文档中提供的方法学习时:我收到一条错误消息:

kf=KFold(len(train), n_folds=3)

for train_index, test_index in kf:
     X_train, X_test = train[train_index],train[test_index]
     y_train, y_test = labels[train_index],labels[test_index]

错误

   X_train, X_test = train[train_index],train[test_index]
TypeError: only integer arrays with one element can be converted to an index

如何对我的文档和标签执行 10 折交叉验证？

Answer 1

有两种方法可以解决这个错误:

第一种方式:

将您的数据转换为 numpy 数组:

import numpy as np
[...]
train = np.array(train)
labels = np.array(labels)

然后它应该可以与您当前的代码一起使用。

第二种方式:

使用列表理解将训练和标签列表与 train_index 和 test_index 列表建立索引

for train_index, test_index in kf:
    X_train, X_test = [train[i] for i in train_index],[train[j] for j in test_index]
    y_train, y_test = [labels[i] for i in train_index],[labels[j] for j in test_index]

(对于此解决方案，另请参阅相关问题 index list with another list)