python - 在评估等同于 2 个函数调用的结果时，寻找一种方法(最好是在 Python 中)获取第二个参数值

Question

我有两个函数，g1(x,y) 和 g2(x,y) 返回一个 float

例如，g1(1,2) --> 返回 0.345
g2(1,2) --> 返回 0.453

现在，为了绘制决策边界，我想满足:
g2(x,y) == g1(x,y),
或者重新排列为:
g1(x,y) - g2(x,y) == 0

如果我生成一系列 x 值 1,2,3,4,5，我如何找到产生 g1( x,y) - g2(x,y) == 0?

我真的不知道该怎么做，如果有任何想法，我将不胜感激。你认为 scipy.optimize.minimize 是个好方法吗？如果是这样，我究竟该怎么做(我尝试过语法但失败了)。

感谢您的帮助!

编辑:

你问的是 g1() 和 g2() 的方程式，它们在这里:)

$\Rightarrow g_1(\pmb{x}) =\pmb{x}^{\,t} -\frac{1}{2}\Sigma_1^{-1}\pmb{x} +\bigg (\Sigma_1^{-1}\pmb{\mu}{\,1}\bigg)^t +\bigg( -\frac{1}{2}\pmb{\mu} {\,1}^{\,t}\Sigma_{1}^{-1}\pmb{\mu}_{\,1} -\frac{1}{2} ln(|\Sigma_1|)\比格)\\quad g_2(\pmb{x}) =\pmb{x}^{\,t} -\frac{1}{2}\Sigma_2^{-1}\pmb{x} +\bigg(\Sigma_2^ {-1}\pmb{\mu}{\,2}\bigg)^t +\bigg( -\frac{1}{2}\pmb{\mu}{\,2 }^{\,t}\Sigma_{2}^{-1}\pmb{\mu}_{\,2} -\frac{1}{2} ln(|\Sigma_2|)\bigg) $

(嗯，不知何故 Latex 无法正常工作，我将其作为图片上传):

这就是我实现它们的方式:

def discriminant_function(x_vec, cov_mat, mu_vec):
    """
    Calculates the value of the discriminant function for a dx1 dimensional
    sample given covariance matrix and mean vector.

    Keyword arguments:
        x_vec: A dx1 dimensional numpy array representing the sample.
        cov_mat: numpy array of the covariance matrix.
        mu_vec: dx1 dimensional numpy array of the sample mean.

    Returns a float value as result of the discriminant function.

    """
    W_i = (-1/2) * np.linalg.inv(cov_mat)
    assert(W_i.shape[0] > 1 and W_i.shape[1] > 1), 'W_i must be a matrix'

    w_i = np.linalg.inv(cov_mat).dot(mu_vec)
    assert(w_i.shape[0] > 1 and w_i.shape[1] == 1), 'w_i must be a column vector'

    omega_i_p1 = (((-1/2) * (mu_vec).T).dot(np.linalg.inv(cov_mat))).dot(mu_vec)
    omega_i_p2 = (-1/2) * np.log(np.linalg.det(cov_mat))
    omega_i = omega_i_p1 - omega_i_p2
    assert(omega_i.shape == (1, 1)), 'omega_i must be a scalar'

    g = ((x_vec.T).dot(W_i)).dot(x_vec) + (w_i.T).dot(x_vec) + omega_i
    return float(g)

为了对我写的数据进行分类:

导入操作符

def classify_data(x_vec, g, mu_vecs, cov_mats):
    """
    Classifies an input sample into 1 out of x classes determined by
    maximizing the discriminant function g_i().

    Keyword arguments:
        x_vec: A dx1 dimensional numpy array representing the sample.
        g: The discriminant function.
        mu_vecs: A list of mean vectors as input for g.
        cov_mats: A list of covariance matrices as input for g.

    Returns a tuple (g_i()_value, class label).

    """
    assert(len(mu_vecs) == len(cov_mats)), 'Number of mu_vecs and cov_mats must be equal.'

    g_vals = []
    for m,c in zip(mu_vecs, cov_mats): 
        g_vals.append(g(x_vec, mu_vec=m, cov_mat=c))

    max_index, max_value = max(enumerate(g_vals), key=operator.itemgetter(1))
    return (max_value, max_index + 1)

到目前为止，该代码可用于分类，例如，

import prettytable

classification_dict, error = empirical_error(all_samples, [1,2], classify_data, [discriminant_function,\
        [mu_est_1, mu_est_2],
        [cov_est_1, cov_est_2]])

labels_predicted = ['w{} (predicted)'.format(i) for i in [1,2]]
labels_predicted.insert(0,'training dataset')

train_conf_mat = prettytable.PrettyTable(labels_predicted)
for i in [1,2]:
    a, b = [classification_dict[i][j] for j in [1,2]]
    # workaround to unpack (since Python does not support just '*a')
    train_conf_mat.add_row(['w{} (actual)'.format(i), a, b])
print(train_conf_mat)
print('Empirical Error: {:.2f} ({:.2f}%)'.format(error, error * 100))


+------------------+----------------+----------------+
| training dataset | w1 (predicted) | w2 (predicted) |
+------------------+----------------+----------------+
|   w1 (actual)    |       49       |       1        |
|   w2 (actual)    |       1        |       49       |
+------------------+----------------+----------------+
Empirical Error: 0.02 (2.00%)

对于像这样的简单数据集:

编辑:

对于协方差相等的简单情况(线性决策边界)，我能够使用 fsolve 函数:

from scipy.optimize import fsolve
x = list(np.arange(-2, 6, 0.1))
y = [fsolve(lambda y: discr_func(i, y, cov_mat=cov_est_1, mu_vec=mu_est_1) - \
                 discr_func(i, y, cov_mat=cov_est_2, mu_vec=mu_est_2), 0) for i in x]

但是，它不适用于二次解，我明白了

/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/scipy/optimize/minpack.py:236: RuntimeWarning: The iteration is not making good progress, as measured by the 
  improvement from the last five Jacobian evaluations.
  warnings.warn(msg, RuntimeWarning)

有任何提示或替代方案吗？

编辑 2:

我能够通过 scipy.optimize.bisect(模拟到 fsolve)来解决它。结果看起来是“正确的”——我求解了一个更简单的方程，其中决策边界是一个线性函数 (x2 = 3-x1)，当我对它使用 bisect 时，它计算例如，x1 = 3 和 x2 = 3 的精确结果。

无论如何，这是二次函数的结果(我在这里通过最大似然估计来估计参数)和协方差相等的线性情况!非常感谢您的宝贵时间和帮助!

为了

from matplotlib import pyplot as plt
import numpy as np
import scipy.optimize

x = np.arange(-6,6,0.1)
true_y = [true_dec_bound(x1) for x1 in x]



for i in [50,1000,10000]:

    # compute boundary for MLE estimate
    y_est = []
    for j in x:
        y_est.append(scipy.optimize.bisect(lambda y: discr_func(j, y, cov_mat=cov1_ests[i], mu_vec=mu1_ests[i]) - \
                      discr_func(j, y, cov_mat=cov2_ests[i], mu_vec=mu2_ests[i]), -10, 10))
    y_est = [float(i) for i in y_est]

    # plot data
    f, ax = plt.subplots(figsize=(7, 7))
    plt.ylabel('$x_2$', size=20)
    plt.xlabel('$x_1$', size=20)
    ax.scatter(samples_c1[i][:,0], samples_c1[i][:,1], \
           marker='o', color='green', s=40, alpha=0.5, label='$\omega_1$')
    ax.scatter(samples_c2[i][:,0], samples_c2[i][:,1], \
           marker='^', color='red', s=40, alpha=0.5, label='$\omega_2$')
    plt.title('%s bivariate random training samples per class' %i)
    plt.legend()

    # plot boundaries
    plt.plot(x_true50, y_true50, 'b--', lw=3, label='true param. boundary')
    plt.plot(x_est50, y_est50, 'k--', lw=3, label='MLE boundary')

    plt.legend(loc='lower left')
    plt.show()

Answer 1

现在只是想发布我的初步解决方案。但这可能不是最佳的......

def discr_func(x, y, cov_mat, mu_vec):
    """
    Calculates the value of the discriminant function for a dx1 dimensional
    sample given covariance matrix and mean vector.

    Keyword arguments:
        x_vec: A dx1 dimensional numpy array representing the sample.
        cov_mat: numpy array of the covariance matrix.
        mu_vec: dx1 dimensional numpy array of the sample mean.

    Returns a float value as result of the discriminant function.

    """
    x_vec = np.array([[x],[y]])

    W_i = (-1/2) * np.linalg.inv(cov_mat)
    assert(W_i.shape[0] > 1 and W_i.shape[1] > 1), 'W_i must be a matrix'

    w_i = np.linalg.inv(cov_mat).dot(mu_vec)
    assert(w_i.shape[0] > 1 and w_i.shape[1] == 1), 'w_i must be a column vector'

    omega_i_p1 = (((-1/2) * (mu_vec).T).dot(np.linalg.inv(cov_mat))).dot(mu_vec)
    omega_i_p2 = (-1/2) * np.log(np.linalg.det(cov_mat))
    omega_i = omega_i_p1 - omega_i_p2
    assert(omega_i.shape == (1, 1)), 'omega_i must be a scalar'

    g = ((x_vec.T).dot(W_i)).dot(x_vec) + (w_i.T).dot(x_vec) + omega_i
    return float(g)

#g1 = discr_func(x, y, cov_mat=cov_mat1, mu_vec=mu_vec_1)
#g2 = discr_func(x, y, cov_mat=cov_mat2, mu_vec=mu_vec_2)

x_est50 = list(np.arange(-6, 6, 0.1))
y_est50 = []
for i in x_est50:
    y_est50.append(scipy.optimize.bisect(lambda y: discr_func(i, y, cov_mat=cov_est_1, mu_vec=mu_est_1) - \
                      discr_func(i, y, cov_mat=cov_est_2, mu_vec=mu_est_2), -10,10))
y_est50 = [float(i) for i in y_est50]

结果如下:(蓝色为二次情况，红色为线性情况(等方差)