java - 使用 Apache 数学进行带权重的线性回归

Question

我已经使用 Apache math 一段时间了，通过 OLSMultipleLinearRegression 进行多元线性回归。现在我需要扩展我的解决方案以包含每个数据点的权重因子。

我正在尝试复制 MATLAB 函数 fitlm。

我有一个 MATLAB 调用，例如:

table_data = table(points_scored, height, weight, age);
model = fitlm( table_data, 'points_scored ~ -1, height, weight, age', 'Weights', data_weights)

从“模型”中，我得到高度、体重、年龄的回归系数。

在 Java 中，我现在的代码(大致)是:

double[][] variables = double[grades.length][3];
// Fill in variables for height, weight, age, 
...

OLSMultipleLinearRegression regression = new OLSMultipleLinearRegression();
regression.setNoIntercept(true);
regression.newSampleData(points_scored, variables);

似乎没有办法向 OLSMultipleLinearRegression 添加权重。似乎确实有一种方法可以向 LeastSquaresBuilder 添加权重。但是我很难弄清楚如何使用它。我认为最大的问题是创建预期的雅可比矩阵。

这是我尝试过的大部分内容:

double[] points_scored = //fill in points scored
double[] height = //fill in 
double[] weight = //fill in
double[] age = // fill in

MultivariateJacobianFunction distToResidual= coeffs -> {
  RealVector value = new ArrayRealVector(points_scored.length);
  RealMatrix jacobian = new Array2DRowRealMatrix(points_scored.length, 3);

  for (int i = 0; i < measures.length; ++i) {
    double residual = points_scored[i];
    residual -= coeffs.getEntry(0) * height[i];  
    residual -= coeffs.getEntry(1) * weight[i];  
    residual -= coeffs.getEntry(2) * age[i];  
    value.setEntry(i, residual);
    //No idea how to set up the jacobian here
   }

   return new Pair<RealVector, RealMatrix>(value, jacobian);
};

double[] prescribedDistancesToLine = new double[measures.length];
Arrays.fill(prescribedDistancesToLine, 0);
double[] starts = new double[] {1, 1, 1};

LeastSquaresProblem problem = new LeastSquaresBuilder().
            start(starts).
            model(distToResidual).
            target(prescribedDistancesToLine).
            lazyEvaluation(false).
            maxEvaluations(1000).
            maxIterations(1000).
            build();
 LeastSquaresOptimizer.Optimum optimum = new LevenbergMarquardtOptimizer().optimize(problem);

由于我不知道如何生成雅可比值，所以我一直在黑暗中摸索，得到的系数与 MATLAB 答案相差甚远。一旦我让这部分工作，我知道添加权重应该是 LeastSquaresBuilder 中非常直接的额外行。

感谢您提前提供的任何帮助!

Answer 1

您可以使用类 GLSMultipleLinearRegression来自 Apache 数学。例如，让我们找到三个平面数据点的线性回归(0, 0), (1, 2), (2, 0)，权重为 1, 2, 1:

import org.apache.commons.math3.stat.regression.GLSMultipleLinearRegression;

public class Main {
    public static void main(String[] args) {
        GLSMultipleLinearRegression regr = new GLSMultipleLinearRegression();
        regr.setNoIntercept(false);
        double[] y = new double[]{0.0, 2.0, 0.0};
        double[][] x = new double[3][];
        x[0] = new double[]{0.0};
        x[1] = new double[]{1.0};
        x[2] = new double[]{2.0};
        double[][] omega = new double[3][];
        omega[0] = new double[]{1.0, 0.0, 0.0};
        omega[1] = new double[]{0.0, 0.5, 0.0};
        omega[2] = new double[]{0.0, 0.0, 1.0};
        regr.newSampleData(y, x, omega);
        double[] params = regr.estimateRegressionParameters();
        System.out.println("Slope: " + params[1] + ", intercept: " + params[0]);
    }
}

请注意，omega 矩阵是对角矩阵，其对角元素是权重的倒数。

查看 documentation对于多变量情况。