python - 如何为 SciPy fmin_l_bfgs_b 创建伪函数和函数素数？

Question

我想使用 scipy.optimize.fmin_l_bfgs_b 找到成本函数的最小值。

为此，我想首先创建一个one_batch的实例(one_batch的代码在下面给出)来指定训练示例的批处理和那些不包含在损失函数中但计算损失所必需的参数。

因为模块 loss_calc 旨在同时返回损失和损失素数，所以我面临着为 scipy 分离损失函数和损失函数素数的问题。优化.fmin_l_bfgs_b。

从one_batch的代码可以看出，给定一批训练样例，[loss, dloss/dParameters]会为每个样例并行计算的例子。我不想对 get_loss 和 get_loss_prime 进行两次完全相同的计算。

那么如何设计get_loss和get_loss_prime方法，只需要并行计算一次呢？

这里是one_batch的代码

from calculator import loss_calc

class one_batch:

    def __init__(self, 
                 auxiliary_model_parameters, 
                 batch_example):

        # auxiliary_model_parameters are parameters need to specify 
        # the loss calculator but are not included in the loss function.

        self.auxiliary_model_parameters = auxiliary_model_parameters 
        self.batch_example = batch_example

    def parallel(self, func, args):
        pool = multiprocessing.Pool(multiprocessing.cpu_count())
        result = pool.map(func, args)
        return result 

    def one_example(self, example):
        temp_instance = loss_calc(self.auxiliary_model_parameters, 
                                  self.model_vector)
        loss, dloss = temp_instance(example).calculate()
        return [loss, dloss]

    def main(self, model_vector):
        self.model_vector = model_vector

        # model_vector and auxiliary_model_parameters are necessary 
        # for creating an instance of loss function calculator 

        result_list = parallel(self.one_example, 
                               self.batch_examples)

        # result_list is a list of sublists, each sublist is 
        # [loss, dloss/dParameter] for each training example 

   def get_loss(self):
       ?

   def get_loss_prime(self):
       ?

Answer 1

您可以使用直接返回两个函数值的目标函数作为 fmin_l_bfgs_b 的输入:

from scipy.optimize import fmin_l_bfgs_b
import numpy as np

def obj_fun(x):
    fx = 2*x**2 + 2*x + 1
    grad = np.array([4*x + 2])
    return fx, grad

fmin_l_bfgs_b(obj_fun, x0=[12])

(array([-0.5]), array([0.5]), {'grad': array([[-3.55271368e-15]]),
'task': b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL',
'funcalls': 4, 'nit': 2, 'warnflag': 0})