r - 在插入符号 : Error for class probabilities 中使用 eml

Question

我正在尝试使用方法 "nnet" 将标准神经网络方法与极限学习机分类器(基于 ROC 度量)进行比较。和 "elm"在 R 包中 caret .对于 nnet，一切正常，但使用 method = "elm"我收到以下错误:

Error in evalSummaryFunction(y, wts = weights, ctrl = trControl, lev = classLevels,  : 
  train()'s use of ROC codes requires class probabilities. See the classProbs option of trainControl()
In addition: Warning messages:
1: In train.default(x, y, weights = w, ...) :
  At least one of the class levels are not valid R variables names; This may cause errors if class probabilities are generated because the variables names will be converted to: X1, X2
2: In train.default(x, y, weights = w, ...) :
  Class probabilities were requested for a model that does not implement them

我也遇到了第一个错误 method = "nnet" ，但在这里我可以通过将分数设为因子变量来解决问题。因此，这不可能是这里的问题。

我对 R 比较陌生，也许错误很简单，但现在我被卡住了......由于 elmNN 似乎是相对较新的实现，我也无法在网上找到任何关于如何在 caret 中使用 elm 的信息。 .

gc <- read.table("germanCreditNum.txt")
colnames(gc)[25]<-"score"

gc_inTrain <- createDataPartition(y = gc$score,
    ## the outcome data are needed
    p = .8,
    ## The percentage of data in the
    ## training set
    list = FALSE)

str(gc_inTrain)
gc_training <- gc[ gc_inTrain,]
gc_testing <- gc[-gc_inTrain,]
nrow(gc_training) ## No of rows 
nrow(gc_testing)

gc_training$score <- as.factor(gc_training$score)

gc_ctrl <- trainControl(method = "boot",
    repeats = 1,
    classProbs = TRUE,
    summaryFunction = twoClassSummary)

neuralnetFit <- train(score ~ .,
    data = gc_training,
    method = "nnet",
    trControl = gc_ctrl,
    metric = "ROC",
    preProc = c("center", "scale"))

neuralnetFit
plot(neuralnetFit)
nnClasses <- predict(neuralnetFit, newdata = gc_testing)
str(nnClasses)

## start with ELM for German Credit

gc_ctrl2 <- trainControl(classProbs = TRUE, summaryFunction = twoClassSummary)
elmFit <- train(score ~ ., 
    data = gc_training, 
    method = "elm", 
    trControl = gc_ctrl2, 
    metric = "ROC", 
    preProc = c("center", "scale"))

elmFit
plot(elmFit)

elmClasses <- predict(elmFit, newdata = gc_testing)
str(elmClasses)
elmProbs <- predict(elmFit, newdata = gc_testing, type = "prob")
head(elmProbs) 

Answer 1

我不记得为什么我没有包含 ELM 的概率模型(我可能有一个很好的理由)。您可以使用 custom method获取 softmax 值:

library(caret)

set.seed(1)
dat <- twoClassSim(100)

elm_fun <- getModelInfo("elm")[[1]]
elm_fun$prob <- function (modelFit, newdata, submodels = NULL)  {
  out <- exp(predict(modelFit, newdata))
  t(apply(out, 1, function(x) x/sum(x)))
}
mod <- train(Class ~ ., data = dat, 
             method = elm_fun,
             metric = "ROC",
             trControl = trainControl(classProbs = TRUE,
                                      summaryFunction = twoClassSummary))

最大限度