java - 无法将预测映射到 JavaRDD

Question

我正在尝试将预测映射到 LinearRegression 模型，以便将它们传递到 BinaryClassificationMetrics boject:

// Make predictions on test documents. cvModel uses the best model found (lrModel).
DataFrame predictions = cvModel.transform(testingFrame);
JavaRDD<Tuple2<Object, Object>> scoreAndLabels = predictions.map(
        new Function<Row, Tuple2<Object, Object>>() {
            @Override
            public Tuple2<Object, Object> call(Row r) {
                Double score = r.getDouble(1);
                return new Tuple2<Object, Object>(score, r.getDouble(0));
            }
        }
);
BinaryClassificationMetrics metrics
        = new BinaryClassificationMetrics(JavaRDD.toRDD(scoreAndLabels));

但是，当我调用 predictions.map(...) 时，出现以下编译错误:

method map in class DataFrame cannot be applied to given types;
  required: Function1<Row,R>,ClassTag<R>
  found: <anonymous Function<Row,Tuple2<Object,Object>>>
  reason: cannot infer type-variable(s) R
    (actual and formal argument lists differ in length)
  where R is a type-variable:
    R extends Object declared in method <R>map(Function1<Row,R>,ClassTag<R>)

关于如何映射预测 DataFrame 的数据有什么建议吗？

Answer 1

想通了!我必须将 DataFrame 转换为 JavaRDD，从那里开始就很简单了:

DataFrame predictions = cvModel.transform(testingFrame);
JavaRDD<Tuple2<Object, Object>> scoreAndLabels = predictions.toJavaRDD().map(
        new Function<Row, Tuple2<Object, Object>>() {
            @Override
            public Tuple2<Object, Object> call(Row r) {
                Double score = r.getDouble(4);
                Double label = r.getDouble(1);
                return new Tuple2<Object, Object>(score, label);
            }
        });

BinaryClassificationMetrics metrics
        = new BinaryClassificationMetrics(JavaRDD.toRDD(scoreAndLabels));