python - OneHotEncoder 在调用 SimpleImputer 后引发 NaN 问题

Question

我无法理解流水线在 Sklearn 中的工作方式。以下是使用 titanic 数据集的示例。

data = pd.read_csv('datasets/train.csv')

cat_attribs = ["Embarked", "Cabin", "Ticket", "Name"]

num_pipeline = Pipeline([
        ('imputer', SimpleImputer(strategy="median")),
    ])


str_pipeline = Pipeline([
        ('imputer', SimpleImputer(strategy="most_frequent")),
    ])


full_pipeline = ColumnTransformer([
        ("num", num_pipeline, ["Pclass", "Age", "SibSp", "Parch", "Fare"]),
        ("str", str_pipeline, ["Cabin", "Sex"]),
        ("cat", OneHotEncoder(), ["Cabin"]),
    ])

full_pipeline.fit_transform(data)

我希望这会填充所有缺失的 NaN 值(包括数字和字符串)属性，然后最终将 Cabin 属性转换为数字。

相反，代码以以下错误结束:

ValueError: Input contains NaN. If I remove the line calling the OneHotEncoder and printing the transformed array, there is no NaN value.

所以我想知道。在这种情况下，我应该如何调用 OneHotEncoder。

Answer 1

我建议将 OneHotEncoder 应用于所有分类变量。因此，将其作为一个单独的管道。

由于它是数字列的单步过程，您可以直接使用 ColumnTransformer。

试试这个!

from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer, make_column_transformer
from sklearn.preprocessing import OneHotEncoder
from sklearn.pipeline import Pipeline, make_pipeline

cat_preprocess = make_pipeline(SimpleImputer(strategy="most_frequent"), OneHotEncoder())

ct = make_column_transformer([
        ("num", SimpleImputer(strategy="median"), ["Pclass", "Age", "SibSp", "Parch", "Fare"]),
        ("str", cat_preprocess, ["Cabin", "Sex"]),
    ])

pipeline = Pipeline([('preprocess', ct)])