python-3.x - h2o 与 scikit 学习混淆矩阵

Question

有人能将 sklearn 混淆矩阵与 h2o 相匹配吗？

他们从不匹配....

用 Keras 做类似的事情会产生完美的匹配。

但在 h2o 中它们总是关闭的。各种方法都试过了...

借用了一些代码: Any difference between H2O and Scikit-Learn metrics scoring?

# In[30]:
import pandas as pd
import h2o
from h2o.estimators.gbm import H2OGradientBoostingEstimator
h2o.init()

# Import a sample binary outcome train/test set into H2O
train = h2o.import_file("https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv")
test = h2o.import_file("https://s3.amazonaws.com/erin-data/higgs/higgs_test_5k.csv")

# Identify predictors and response
x = train.columns
y = "response"
x.remove(y)

# For binary classification, response should be a factor
train[y] = train[y].asfactor()
test[y] = test[y].asfactor()

# Train and cross-validate a GBM
model = H2OGradientBoostingEstimator(distribution="bernoulli", seed=1)
model.train(x=x, y=y, training_frame=train)

# In[31]:
# Test AUC
model.model_performance(test).auc()
# 0.7817203808052897

# In[32]:

# Generate predictions on a test set
pred = model.predict(test)

# In[33]:

from sklearn.metrics import roc_auc_score, confusion_matrix

pred_df = pred.as_data_frame()
y_true = test[y].as_data_frame()

roc_auc_score(y_true, pred_df['p1'].tolist())
#pred_df.head()

# In[36]:

y_true = test[y].as_data_frame().values
cm = pd.DataFrame(confusion_matrix(y_true, pred_df['predict'].values))

# In[37]:

print(cm)
    0     1
0  1354   961
1   540  2145

# In[38]:
model.model_performance(test).confusion_matrix()

Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.353664307031828: 

    0         1     Error   Rate
0   964.0   1351.0  0.5836  (1351.0/2315.0)
1   274.0   2411.0  0.102   (274.0/2685.0)
Total   1238.0  3762.0  0.325   (1625.0/5000.0)

# In[39]:
h2o.cluster().shutdown()

Answer 1

这就成功了，感谢 Vivek 的预感。仍然不是完全匹配，但非常接近。

perf = model.model_performance(train)
threshold = perf.find_threshold_by_max_metric('f1')
model.model_performance(test).confusion_matrix(thresholds=threshold)