python - 从theano中给定的pmf中选择一个数字

Question

假设我有一个数组p = [ 0.27, 0.23, 0.1, 0.15, 0.2 ,0.05]。令p 为随机变量X 的概率质量函数。现在，我正在编写一个 theano 代码，其中每次迭代都会生成一个 p ，并且我还有 n 个权重矩阵。 (此处[n = 6]。)

现在，在每次迭代中，我想选择这些权重矩阵之一以进行进一步传播。有人可以帮忙了解如何编写这段代码吗？我不确定我是否可以编写启用反向传播所需的确切代码(即正确校正梯度)

请注意，所有 W_i 以及输入 p 都是模型参数。

Edit

 W1,W2,W3,W4,W5,W6,x,eps = T.dmatrices("W1","W2","W3","W4","W5","W6","x","eps")

    b1,b2,b3,b4,b5,b6,pi = T.dcols("b1","b2","b3","b4","b5","b6","pi")

   h_encoder = T.tanh(T.dot(W1,x) + b1)

    rng = T.shared_randomstreams.RandomStreams(seed=124)

    i = rng.choice(size=(1,), a=self.num_model, p=T.nnet.softmax(pi))

    mu_encoder = T.dot(W2[i[0]*self.dimZ:(1+i[0])*self.dimZ].nonzero(),h_encoder) + b2[i[0]*self.dimZ:(1+i[0])*self.dimZ].nonzero()

    log_sigma_encoder = (0.5*(T.dot(W3[i[0]*self.dimZ:(1+i[0])*self.dimZ].nonzero(),h_encoder)))+ b3[i[0]*self.dimZ:(1+i[0])*self.dimZ].nonzero()

    z = mu_encoder + T.exp(log_sigma_encoder)*eps`

我的 grad 变量是 gradvariables = [W1,W2,W3,W4,W5,b1,b2,b3,b4,b5,pi] 忽略其他变量，因为它们是在其他地方定义的。现在，我收到以下错误

Traceback (most recent call last): File "trainmnist_mixture.py", line 55, in encoder.createGradientFunctions()

File "/home/amartya/Variational-Autoencoder/Theano/VariationalAutoencoder_mixture.py", line 118, in createGradientFunctions derivatives = T.grad(logp,gradvariables)

File "/usr/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/gradient.py", line 543, in grad grad_dict, wrt, cost_name)

File "/usr/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/gradient.py", line 1273, in _populate_grad_dict rval = [access_grad_cache(elem) for elem in wrt]

File "/usr/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/gradient.py", line 1233, in access_grad_cache term = access_term_cache(node)[idx]

File "/usr/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/gradient.py", line 944, in access_term_cache output_grads = [access_grad_cache(var) for var in node.outputs]

File "/usr/lib/python2.7/site-packages/Theano-0.6.0-py2.7.egg/theano/gradient.py", line 1243, in access_grad_cache term.type.why_null)

theano.gradient.NullTypeGradError: tensor.grad encountered a NaN. This variable is Null because the grad method for input 0 (Subtensor{int64:int64:}.0) of the Nonzero op is mathematically undefined

Answer 1

您可以使用 RandomStreams 实例的 choice 方法。有关 Theano 中随机数的更多信息，请参阅文档 here和 here .

这是一个例子:

import numpy
import theano
import theano.tensor as tt
import theano.tensor.shared_randomstreams

n = 6
alpha = [1] * n
seed = 1
w = theano.shared(numpy.random.randn(n, 2, 2).astype(theano.config.floatX))
p = theano.shared(numpy.random.dirichlet(alpha).astype(theano.config.floatX))
rng = tt.shared_randomstreams.RandomStreams(seed=seed)
i = rng.choice(size=(1,), a=n, p=p)
f = theano.function([], [p, i, w[i]])
print f()
print f()
print f()
print f()