python - 为什么 scipy.optimize.curve_fit 会重复评估初始猜测(并且可能是成本高昂的)？

Question

使用curve_fit时，模型函数会在不更改参数的情况下重复进行无用的(并且可能成本高昂)评估。为什么会出现这种情况？

考虑以下示例来识别二次函数的参数:

from scipy.optimize import curve_fit

i=0

def f(x, a, b):
    global i; i += 1
    print('run: {:2}, p: {:<11.10}, {:<11.10}'.format(i, a, b))
    return(x**a+b)

popt, pcov = curve_fit(f, [0, 1, 2, 3], [1, 2, 5, 10], p0 = [1.5, 0.5])
print('\npopt:', popt)

输出:

run:  1, p: 1.5        , 0.5        
run:  2, p: 1.5        , 0.5        
run:  3, p: 1.5        , 0.5        
run:  4, p: 1.500000022, 0.5        
run:  5, p: 1.5        , 0.5000000075
run:  6, p: 2.166073038, 0.9668143807
run:  7, p: 2.166073071, 0.9668143807
run:  8, p: 2.166073038, 0.9668143951
run:  9, p: 2.014374939, 0.9956632744
run: 10, p: 2.014374969, 0.9956632744
run: 11, p: 2.014374939, 0.9956632892
run: 12, p: 2.000113621, 0.9999686478
run: 13, p: 2.000113651, 0.9999686478
run: 14, p: 2.000113621, 0.9999686627
run: 15, p: 2.000000007, 0.999999998
run: 16, p: 2.000000037, 0.999999998
run: 17, p: 2.000000007, 1.000000013
run: 18, p: 2.0        , 1.0        

popt: [2. 1.]

第一次评估计算初始猜测的值，但随后在第二次和第三次运行中再次执行此操作。仅在第四次和第五次评估中，通过计算导数开始优化。如果功能评估成本高昂并且可以接受一些更大范围的容差，则这些冗余评估可能会占用相当长的时间。

Answer 1

使用 full_output(请参阅 leastsq 文档):

In [125]: curve_fit(f, [0, 1, 2, 3], [1, 2, 5, 10], p0 = [1.5, 0.5],full_output=True)            
run:  1, p: 1.5        , 0.5        
run:  2, p: 1.5        , 0.5        
run:  3, p: 1.5        , 0.5        
run:  4, p: 1.500000022, 0.5        
run:  5, p: 1.5        , 0.5000000075
run:  6, p: 2.166073038, 0.9668143807
run:  7, p: 2.166073071, 0.9668143807
run:  8, p: 2.166073038, 0.9668143951
run:  9, p: 2.014374939, 0.9956632744
run: 10, p: 2.014374969, 0.9956632744
run: 11, p: 2.014374939, 0.9956632892
run: 12, p: 2.000113621, 0.9999686478
run: 13, p: 2.000113651, 0.9999686478
run: 14, p: 2.000113621, 0.9999686627
run: 15, p: 2.000000007, 0.999999998
run: 16, p: 2.000000037, 0.999999998
run: 17, p: 2.000000007, 1.000000013
run: 18, p: 2.0        , 1.0        
Out[125]: 
(array([2., 1.]), array([[ 0., -0.],
        [-0.,  0.]]), {'fvec': array([0., 0., 0., 0.]),
  'nfev': 16,
  'fjac': array([[-10.2688908 ,   0.        ,   0.26999885,   0.96286064],
         [ -1.2328595 ,  -1.5748198 ,   0.2521752 ,  -0.73019944]]),
  'ipvt': array([1, 2], dtype=int32),
  'qtf': array([-7.08708997e-08,  3.07498129e-09])}, 'The relative error between two consecutive iterates is at most 0.000000', 2)

返回的信息是:

infodict : dict
a dictionary of optional outputs with the keys:

``nfev``
    The number of function calls
``fvec``
    The function evaluated at the output
``fjac``
    A permutation of the R matrix of a QR
    factorization of the final approximate
    Jacobian matrix, stored column wise.
    Together with ipvt, the covariance of the
    estimate can be approximated.
``ipvt``
    An integer array of length N which defines
    a permutation matrix, p, such that
    fjac*p = q*r, where r is upper triangular
    with diagonal elements of nonincreasing
    magnitude. Column j of p is column ipvt(j)
    of the identity matrix.
``qtf``
    The vector (transpose(q) * fvec).

所以它声称要评估 16 次，而不是你的 18 次。因此一次或多次初始评估可能来自 curve_fit 参数检查(或类似的东西)。

使用其他方法:

In [134]: i=0                                                                                    
In [135]: curve_fit(f, [0, 1, 2, 3], [1, 2, 5, 10], p0 = [1.5, 0.5],method='trf')                
run:  1, p: 1.5        , 0.5        
run:  2, p: 1.500000022, 0.5        
run:  3, p: 1.5        , 0.5000000149
run:  4, p: 2.166073038, 0.9668143807
run:  5, p: 2.166073071, 0.9668143807
run:  6, p: 2.166073038, 0.9668143956
run:  7, p: 2.01437494 , 0.9956632758
run:  8, p: 2.01437497 , 0.9956632758
run:  9, p: 2.01437494 , 0.9956632907
run: 10, p: 2.000113621, 0.9999686478
run: 11, p: 2.000113651, 0.9999686478
run: 12, p: 2.000113621, 0.9999686627
run: 13, p: 2.000000007, 0.999999998
run: 14, p: 2.000000037, 0.999999998
run: 15, p: 2.000000007, 1.000000013
run: 16, p: 2.0        , 1.0        
run: 17, p: 2.00000003 , 1.0        
run: 18, p: 2.0        , 1.000000015
Out[135]: 
(array([2., 1.]), array([[ 3.77052335e-34, -1.19338002e-33],
        [-1.19338002e-33,  9.94005329e-33]]))

和

In [136]: i=0                                                                                    
In [137]: curve_fit(f, [0, 1, 2, 3], [1, 2, 5, 10], p0 = [1.5, 0.5],method='dogbox')             
run:  1, p: 1.5        , 0.5        
run:  2, p: 1.500000022, 0.5        
run:  3, p: 1.5        , 0.5000000149
run:  4, p: 2.166073038, 0.9668143807
run:  5, p: 2.166073071, 0.9668143807
run:  6, p: 2.166073038, 0.9668143956
run:  7, p: 2.01437494 , 0.9956632758
run:  8, p: 2.01437497 , 0.9956632758
run:  9, p: 2.01437494 , 0.9956632907
run: 10, p: 2.000113621, 0.9999686478
run: 11, p: 2.000113651, 0.9999686478
run: 12, p: 2.000113621, 0.9999686627
run: 13, p: 2.000000007, 0.999999998
run: 14, p: 2.000000037, 0.999999998
run: 15, p: 2.000000007, 1.000000013
run: 16, p: 2.0        , 1.0        
run: 17, p: 2.00000003 , 1.0        
run: 18, p: 2.0        , 1.000000015
Out[137]: 
(array([2., 1.]), array([[ 3.77052335e-34, -1.19338002e-33],
        [-1.19338002e-33,  9.94005329e-33]]))