python - Xgboost:bst.best_score、bst.best_iteration 和 bst.best_ntree_limit 有什么区别？

Question

当我使用 xgboost 为 2-cates classification problem 训练我的数据时，我想使用提前停止来获得最佳模型，但我对在我的预测中使用哪一个感到困惑，因为提前停止将返回 3 个不同的选择。
例如，我应该使用

preds = model.predict(xgtest, ntree_limit=bst.best_iteration)

或者我应该使用

preds = model.predict(xgtest, ntree_limit=bst.best_ntree_limit)

还是两者都对，它们应该适用于不同的情况？如果是这样，我如何判断使用哪一个？

这是xgboost文档的原始引用，但它没有给出原因，我也没有找到这些参数之间的比较:

Early Stopping

If you have a validation set, you can use early stopping to find the optimal number of boosting rounds. Early stopping requires at least one set in evals. If there's more than one, it will use the last.

train(..., evals=evals, early_stopping_rounds=10)

The model will train until the validation score stops improving. Validation error needs to decrease at least every early_stopping_rounds to continue training.

If early stopping occurs, the model will have three additional fields: bst.best_score, bst.best_iteration and bst.best_ntree_limit. Note that train() will return a model from the last iteration, not the best one. Pr ediction

A model that has been trained or loaded can perform predictions on data sets.

# 7 entities, each contains 10 features 
data = np.random.rand(7, 10) 
dtest = xgb.DMatrix(data) 
ypred = bst.predict(dtest)

If early stopping is enabled during training, you can get predictions from the best iteration with bst.best_ntree_limit:

ypred = bst.predict(dtest,ntree_limit=bst.best_ntree_limit)

提前致谢。

Answer 1

在我看来，这两个参数指的是相同的想法，或者至少有相同的目标。但我宁愿使用:

preds = model.predict(xgtest, ntree_limit=bst.best_iteration)

从源码我们可以看到 here那个 best_ntree_limit将被放弃以支持 best_iteration .

def _get_booster_layer_trees(model: "Booster") -> Tuple[int, int]:
    """Get number of trees added to booster per-iteration.  This function will be removed
    once `best_ntree_limit` is dropped in favor of `best_iteration`.  Returns
    `num_parallel_tree` and `num_groups`.
    """

此外， best_ntree_limit已从 EarlyStopping 中删除文档页面。
所以我认为这个属性只存在于向后兼容的原因。因此，根据此代码片段和文档，我们可以假设 best_ntree_limit正在或将被弃用。