gpt4 book ai didi

python - 使用 Climin 在 Theano 中实现逻辑回归

转载 作者:太空宇宙 更新时间:2023-11-04 03:14:31 24 4
gpt4 key购买 nike

我想使用 Theano 在 MNIST 上通过小批量实现逻辑回归。我想尝试不同的优化器,所以我决定使用一个名为 climin 的库. Climin 提供了几个函数,它们接收模型的参数、损失和/或损失的梯度,以及数据流作为参数。由于需要编译 theano 函数,我设计了以下模型。

# loads the MNIST dataset
train_set_x, train_set_y = load_data(dataset)[0]

print('... building the model')

W = theano.shared(
value = np.zeros((28 * 28, 10), dtype = theano.config.floatX),
name = 'W',
borrow = True
) # weights dimension 28 * 28, 10

b = theano.shared(
value = np.zeros((10,), dtype = theano.config.floatX),
name = 'b',
borrow = True
) # biases

x = T.matrix('x') # data, represented as rasterized images dimension 28 * 28
y = T.ivector('y') # labes, represented as 1D vector of [int] labels dimension 10

p_y_given_x = T.nnet.softmax(T.dot(x, W) + b)
y_pred = T.argmax(p_y_given_x, axis=1)

NLL = -T.sum(T.log(p_y_given_x)[T.arange(y.shape[0]), y])

# negative_log_likelihood = -T.mean(T.log(y_pred)[T.arange(y.shape[0]), y])

loss = theano.function(inputs = [ x, y ], outputs = NLL )

g_W = theano.function(inputs = [ x, y ], outputs = T.grad(cost=loss, wrt=W))
g_b = theano.function(inputs = [ x, y ], outputs = T.grad(cost=loss, wrt=b))

def d_loss_wrt_pars(parameters, inputs, targets):
# climin passes the parametes
# as 1-D concatinated array to
# this function so I need to
# unpack the parameter array
W = parameters[:28 * 28 * 10].reshape((28 * 28, 10))
b = parameters[28 * 28 * 10:].reshape((1, 10))

return np.concatinate([g_W(x, y).flatten(), g_b(x, y)])

wrt = np.empty(7850) # allocate space for the parameters (MNIST dimensionality 28 * 28 * 10)
cli.initialize.randomize_normal(wrt, 0, 1) # initialize the parameters with random numbers

if batch_size is None:
args = itertools.repeat(([train_set_x, train_set_y], {}))
batches_per_pass = 1
else:
args = cli.util.iter_minibatches([train_set_x, train_set_y], batch_size, [0, 0])
args = ((i, {}) for i in args)
batches_per_pass = train_set_x.shape[0] / batch_size

if optimizer == 'gd':
opt = cli.GradientDescent(wrt, d_loss_wrt_pars, step_rate=0.1, momentum=.95, args=args)
else:
print('unknown optimizer')
return 1

print('... training the model')

for info in opt:
if info['n_iter'] >= n_epochs and (not done_looping):
break

不幸的是,这只会产生:

Traceback (most recent call last):
File "logreg/logistic_sgd.py", line 160, in <module> sys.exit(sgd_optimization_mnist())
File "logreg/logistic_sgd.py", line 69, in sgd_optimization_mnist
g_W = theano.function(inputs = [ x, y ], outputs = T.grad(cost=loss, wrt=W))
File "/Users/romancpodolski/anaconda/lib/python2.7/site-packages/theano/gradient.py", line 430, in grad
if cost is not None and isinstance(cost.type, NullType):

AttributeError: 'Function' 对象没有属性 'type'

任何人都知道如何使这项工作和结合这两个库?

最佳答案

您的代码将编译后的函数(而不是 Theano 表达式)传递给 T.grad。将 loss 替换为 NLL 就可以了

g_W = theano.function(inputs = [ x, y ], outputs = T.grad(cost=NLL, wrt=W)) 
g_b = theano.function(inputs = [ x, y ], outputs = T.grad(cost=NLL, wrt=b))

此外,您可能想尝试使用支持 Theano 的库(例如 Lasagne )进行优化。

关于python - 使用 Climin 在 Theano 中实现逻辑回归,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/36742032/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com