python - 使用 Levenberg-Marquardt 的非线性最小二乘 : Reproducing Matlabs lsqnonlin with Scipy. optimize.least_squares

Question

我正在尝试最小化一个函数，该函数采用长度为 N 的 1darray 并通过 Levenberg-Marquardt (:= LM) 返回标量。

它可以在 Matlab 中运行:

beta_initial = [-0.7823, -0.1441, -0.7669]; 

% substitution for my long, convoluted function
% but it also works with the proper function
F = @(beta) sum(exp(beta))+3; 

options = optimset('Algorithm','Levenberg-Marquardt');

beta_arma = lsqnonlin(F,beta_initial,[],[],options) % -21.7814  -15.9156  -21.5420

F(beta_arma) % 3

当我在 Python 中尝试时，出现值错误:

ValueError: Method 'lm' doesn't work when the number of residuals is less than the number of variables.

import numpy as np
from scipy.optimize import least_squares as lsq

# substitution for my long, convoluted function
F = lambda beta: np.sum(np.exp(beta))+3 

beta_initial = [-0.7823, -0.1441, -0.7669]

beta_arma = lsq(F, beta_initial,method='lm')['x']

我理解错误 scipy 的方式需要这样

out = F(in)，这样 len(out) >= len(in) ，但 matlab 没有

我查看了文档，scipy和 matlab 。

来自 scipy 文档:

Method ‘lm’ (Levenberg-Marquardt) calls a wrapper over least-squares algorithms implemented in MINPACK (lmder, lmdif). It runs the Levenberg-Marquardt algorithm formulated as a trust-region type algorithm. The implementation is based on paper [JJMore], it is very robust and efficient with a lot of smart tricks. It should be your first choice for unconstrained problems. Note that it doesn’t support bounds. Also it doesn’t work when m < n.

看起来没有 LM 实现可以在 m>=n 时工作

我的问题是:

如何使用像 Python 中的 Matlab 一样的 LM 实现非线性最小二乘法？

Answer 1

我通过将函数分成两个找到了解决方法:

• 第一个函数接受一个数组并返回一个数组
• 第二个函数从第一个函数获取处理后的数组并返回标量输出

然后我让优化器在第一个函数上运行。

在上面的最小示例的上下文中:

import numpy as np
from scipy.optimize import least_squares as lsq

F1 = lambda beta: np.exp(beta)
F2 = lambda processed_beta: np.sum(np.exp(processed_beta))+3


beta_initial = [-0.7823, -0.1441, -0.7669]

# parameters that minimze F1
beta_arma = lsq(F1, beta_initial,method='lm')['x'] 

F2(beta_arma) # 3.0