python - sklearn.preprocessing.OneHotEncoder : using drop and handle_unknown ='ignore'

Question

我有一些 pandas.Series – s ，下面 - 我想单热编码。我通过研究发现'b'级别对于我的预测建模任务并不重要。我可以像这样从我的分析中排除它:

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

s = pd.Series(['a', 'b', 'c']).values.reshape(-1, 1)

enc = OneHotEncoder(drop=['b'], sparse=False, handle_unknown='error')
enc.fit_transform(s)
# array([[1., 0.],
#        [0., 0.],
#        [0., 1.]])
enc.get_feature_names()
# array(['x0_a', 'x0_c'], dtype=object)

但是当我去改造一个新系列时，一个包含 'b'和一个新的水平， 'd' ，我收到一个错误:

new_s = pd.Series(['a', 'b', 'c', 'd']).values.reshape(-1, 1)
enc.transform(new_s)

Traceback (most recent call last): File "", line 1, in File "/Users/user/Documents/assets/envs/data-science/venv/lib/python3.7/site-packages/sklearn/preprocessing/_encoders.py", line 390, in transform X_int, X_mask = self._transform(X, handle_unknown=self.handle_unknown) File "/Users/user/Documents/assets/envs/data-science/venv/lib/python3.7/site-packages/sklearn/preprocessing/_encoders.py", line 124, in _transform raise ValueError(msg) ValueError: Found unknown categories ['d'] in column 0 during transform

这是可以预料的，因为我设置了 handle_unknown='error'以上。但是，我想完全忽略除 ['a', 'c'] 之外的所有类。在拟合和随后的转换步骤中。我试过这个:

enc = OneHotEncoder(drop=['b'], sparse=False, handle_unknown='ignore')
enc.fit_transform(s)
enc.transform(new_s)

Traceback (most recent call last): File "", line 1, in File "/Users/user/Documents/assets/envs/data-science/venv/lib/python3.7/site-packages/sklearn/preprocessing/_encoders.py", line 371, in fit_transform self._validate_keywords() File "/Users/user/Documents/assets/envs/data-science/venv/lib/python3.7/site-packages/sklearn/preprocessing/_encoders.py", line 289, in _validate_keywords "handle_unknown must be 'error' when the drop parameter is " ValueError: handle_unknown must be 'error' when the drop parameter is specified, as both would create categories that are all zero.

scikit-learn 似乎不支持这种模式。有谁知道一个 scikit-learn-compatible 模式来完成这个任务？

Answer 1

它看起来像 sklearn.preprocessing.LabelBinarizer 可以适用于这个用例，因为它没有任何参数指定是出错还是忽略新类:

>>> import pandas as pd
>>> from sklearn.preprocessing import LabelBinarizer
>>> s = pd.Series(['a', 'b', 'c']).values.reshape(-1, 1)
>>> enc = LabelBinarizer()
>>> enc.fit_transform(s)
array([[1, 0, 0],
       [0, 1, 0],
       [0, 0, 1]])
>>> enc.classes_
array(['a', 'b', 'c'], dtype='<U1')
>>> new_s = pd.Series(['a', 'b', 'c', 'd']).values.reshape(-1, 1)
>>> enc.transform(new_s)
array([[1, 0, 0],
       [0, 1, 0],
       [0, 0, 1],
       [0, 0, 0]])