python - sklearn 的 MLP predict_proba 函数在内部是如何工作的？

Question

我正在尝试了解如何 sklearn's MLP Classifier检索其 predict_proba 函数的结果。

该网站仅列出:

Probability estimates

还有很多其他的，例如 logistic regression ，有更详细的答案:概率估计。

返回的所有类别的估计值按类别标签排序。

For a multi_class problem, if multi_class is set to be “multinomial” the softmax function is used to find the predicted probability of each class. Else use a one-vs-rest approach, i.e calculate the probability of each class assuming it to be positive using the logistic function. and normalize these values across all the classes.

其他模型类型也有更多细节。以 support vector machine classifier 为例

还有this very nice Stack Overflow post这对其进行了深入解释。

Compute probabilities of possible outcomes for samples in X.

The model need to have probability information computed at training time: fit with attribute probability set to True.

其他例子

Random Forest :

Predict class probabilities for X.

The predicted class probabilities of an input sample are computed as the mean predicted class probabilities of the trees in the forest. The class probability of a single tree is the fraction of samples of the same class in a leaf.

Gaussian Process Classifier:

我希望了解与上述帖子相同的内容，但对于 MLPClassifier。 MLPClassifier 在内部是如何工作的？

Answer 1

在 source code 中寻找，我发现:

def _initialize(self, y, layer_units):

    # set all attributes, allocate weights etc for first call
    # Initialize parameters
    self.n_iter_ = 0
    self.t_ = 0
    self.n_outputs_ = y.shape[1]

    # Compute the number of layers
    self.n_layers_ = len(layer_units)

    # Output for regression
    if not is_classifier(self):
        self.out_activation_ = 'identity'
    # Output for multi class
    elif self._label_binarizer.y_type_ == 'multiclass':
        self.out_activation_ = 'softmax'
    # Output for binary class and multi-label
    else:
        self.out_activation_ = 'logistic'

似乎 MLP 分类器使用 logistic 函数进行二元分类，使用 softmax 函数进行多标签分类以构建输出层。这表明网络的输出是一个概率向量，网络基于该向量推导出预测。

如果我查看 predict_proba 方法:

def predict_proba(self, X):
    """Probability estimates.
    Parameters
    ----------
    X : {array-like, sparse matrix} of shape (n_samples, n_features)
        The input data.
    Returns
    -------
    y_prob : ndarray of shape (n_samples, n_classes)
        The predicted probability of the sample for each class in the
        model, where classes are ordered as they are in `self.classes_`.
    """
    check_is_fitted(self)
    y_pred = self._predict(X)

    if self.n_outputs_ == 1:
        y_pred = y_pred.ravel()

    if y_pred.ndim == 1:
        return np.vstack([1 - y_pred, y_pred]).T
    else:
        return y_pred

这证实了 softmax 或 logistic 作为输出层激活函数的作用，以获得概率向量。

希望对您有所帮助。