python - 如何向量化逻辑回归？

Question

我正在尝试使用 python 为 coursera ML 类实现正则化逻辑回归，但在对其进行矢量化时遇到了很多麻烦。使用this存储库:

我尝试了很多不同的方法，但从未获得正确的梯度或成本，这是我当前的实现:

h = utils.sigmoid( np.dot(X, theta) )
J = (-1/m) * ( y.T.dot( np.log(h) ) + (1 - y.T).dot( np.log( 1 - h ) ) ) + ( lambda_/(2*m) ) * np.sum( np.square(theta[1:]) )
grad = ((1/m) * (h - y).T.dot( X )).T + grad_theta_reg

结果如下:

Cost         : 0.693147

预期

cost: 2.534819

渐变:

[-0.100000, -0.030000, -0.080000, -0.130000]

预期梯度:

[0.146561, -0.548558, 0.724722, 1.398003]

如果有人知道发生了什么，我们将不胜感激。

Answer 1

下面是逻辑回归矢量化版本的工作片段。您可以在这里查看更多 https://github.com/hzitoun/coursera_machine_learning_matlab_python

主要

theta_t = np.array([[-2], [-1], [1], [2]])

data =  np.arange(1, 16).reshape(3, 5).T

X_t = np.c_[np.ones((5,1)), data/10]

y_t =  (np.array([[1], [0], [1], [0], [1]]) >= 0.5) * 1

lambda_t = 3

J, grad = lrCostFunction(theta_t, X_t, y_t, lambda_t), lrGradient(theta_t, X_t, y_t, lambda_t, flattenResult=False)

print('\nCost: f\n', J)
print('Expected cost: 2.534819\n')
print('Gradients:\n')
print(' f \n', grad)
print('Expected gradients:\n')
print(' 0.146561\n -0.548558\n 0.724722\n 1.398003\n')

lrCostFunction

from sigmoid import sigmoid
import numpy as np

def lrCostFunction(theta, X, y, reg_lambda):

     """LRCOSTFUNCTION Compute cost and gradient for logistic regression with 
       regularization
       J = LRCOSTFUNCTION(theta, X, y, lambda) computes the cost of using
       theta as the parameter for regularized logistic regression and the
       gradient of the cost w.r.t. to the parameters. 
     """

     m, n = X.shape #number of training examples
     theta = theta.reshape((n,1))

     prediction = sigmoid(X.dot(theta))

     cost_y_1 = (1 - y) * np.log(1 - prediction)
     cost_y_0 = -1 * y * np.log(prediction)

     J = (1.0/m) * np.sum(cost_y_0 - cost_y_1) + (reg_lambda/(2.0 * m)) * np.sum(np.power(theta[1:], 2))

return J

lr渐变

from sigmoid import sigmoid
import numpy as np

def lrGradient(theta, X,y, reg_lambda, flattenResult=True):
     m,n = X.shape     
     theta = theta.reshape((n,1))
     prediction = sigmoid(np.dot(X, theta))
     errors = np.subtract(prediction, y)
     grad = (1.0/m) * np.dot(X.T, errors)

     grad_with_regul = grad[1:] + (reg_lambda/m) * theta[1:]
     firstRow = grad[0, :].reshape((1,1))
     grad = np.r_[firstRow, grad_with_regul]

     if  flattenResult:    
         return grad.flatten()

return grad

希望有帮助!