python - XGBoost 对列表与数组的预测略有不同，哪个是正确的？

Question

我注意到我正在传递测试特征值的双括号列表

print(test_feats)
>> [[23.0, 3.0, 35.0, 0.28, -3.0, 18.0, 0.0, 0.0, 0.0, 3.33, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 39.0, 36.0, 113.0, 76.0, 0.0, 0.0, 1.0, 0.34, -999.0, -999.0, -999.0, -999.0, -999.0, -999.0, -999.0, -999.0, 0.0, 25.0, 48.0, 48.0, 0.0, 29.0, 52.0, 53.0, 99.0, 368.0, 676.0, 691.0, 4.0, 9.0, 12.0, 13.0]]

我注意到，当我将其传递给 XBGBoost 进行预测时，当我将其转换为数组时，它会返回不同的结果

array_test_feats = np.array(test_feats)
print(regr.predict_proba(test_feats)[:,1][0])
print(regr.predict_proba(aray_test_feats)[:,1][0])
>> 0.46929297
>> 0.5161868

一些基本检查表明值是相同的

print(sum(test_feats[0]) == array_test_feats.sum())
print(test_feats == array_test_feats)) 
>> True
>> array([[ True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True]])

我猜数组是正确的选择，但我真的不知道如何判断。这些预测足够接近，很容易就会被忽略，所以我真的很想了解为什么会发生这种情况。

Answer 1

您刚刚遇到了此处描述的问题:https://github.com/dmlc/xgboost/pull/3970

The documentation does not include lists as an allowed type for the data inputted into DMatrix. Despite this, a list can be passed in without an error. This change would prevent a list form being passed in directly.

I experienced an issue where passing in a list vs a np.array resulted in different predictions (sometimes over 10% relative difference) for the same data. Though these differences were infrequent (~1.5% of cases tested), in certain applications this could cause serious issues.

本质上，在幕后发生的事情是，XGBoost 不正式支持直接传递 Python 列表，但无论如何它都可以工作，因为它命中 a fall through case在XGBoost的数据转换中。

这会导致 XGBoost 使用 XGDMatrixCreateFromCSREx 函数而不是 XGDMatrixCreateFromMat 来创建数据的底层矩阵。然后有一个difference in behavior sprase 与密集表示中缺失的元素之间:

"Sparse" elements are treated as "missing" by the tree booster and as zeros by the linear booster.