Keras weight scaling-计算图

Question

模型代码:

vfnet = Sequential()
for (i, layeroutsize) in enumerate(hid_sizes):
    inshp = dict(input_shape=(ob_space.shape[0]+1,)) if i==0 else {} # add one extra feature for timestep
    vfnet.add(Dense(layeroutsize, activation=cfg["activation"], init="glorot_uniform",**inshp))
vfnet.add(Dense(1))

for layer in vfnet.layers:
    layer.W.set_value(layer.W.get_value(borrow=True)*wt_scale)

问题:

凯拉斯版本:1.0.1

• 第 7-8 行是在开始时只发生一次还是每次都发生？我的意思是它被添加到计算图中，是按比例缩放的权重每次 wt_scale？

打印图表的代码:

import theano
for i in vfnet.layers:
    theano.printing.debugprint(i.input)
    theano.printing.debugprint(i.output)
ipdb.set_trace()

图表(令人困惑!!):

Using Theano backend.
[2017-07-22 17:07:27,139] Making new env: CartPole-v0
dense_input_2 [id A]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |dense_input_2 [id D]
   | |dense_4_W [id E]
   |InplaceDimShuffle{x,0} [id F] ''
     |dense_4_b [id G]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |dense_input_2 [id D]
   | |dense_4_W [id E]
   |InplaceDimShuffle{x,0} [id F] ''
     |dense_4_b [id G]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |Elemwise{tanh,no_inplace} [id D] ''
   | | |Elemwise{add,no_inplace} [id E] ''
   | |   |dot [id F] ''
   | |   | |dense_input_2 [id G]
   | |   | |dense_4_W [id H]
   | |   |InplaceDimShuffle{x,0} [id I] ''
   | |     |dense_4_b [id J]
   | |dense_5_W [id K]
   |InplaceDimShuffle{x,0} [id L] ''
     |dense_5_b [id M]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |Elemwise{tanh,no_inplace} [id D] ''
   | | |Elemwise{add,no_inplace} [id E] ''
   | |   |dot [id F] ''
   | |   | |dense_input_2 [id G]
   | |   | |dense_4_W [id H]
   | |   |InplaceDimShuffle{x,0} [id I] ''
   | |     |dense_4_b [id J]
   | |dense_5_W [id K]
   |InplaceDimShuffle{x,0} [id L] ''
     |dense_5_b [id M]
Elemwise{add,no_inplace} [id A] ''
 |dot [id B] ''
 | |Elemwise{tanh,no_inplace} [id C] ''
 | | |Elemwise{add,no_inplace} [id D] ''
 | |   |dot [id E] ''
 | |   | |Elemwise{tanh,no_inplace} [id F] ''
 | |   | | |Elemwise{add,no_inplace} [id G] ''
 | |   | |   |dot [id H] ''
 | |   | |   | |dense_input_2 [id I]
 | |   | |   | |dense_4_W [id J]
 | |   | |   |InplaceDimShuffle{x,0} [id K] ''
 | |   | |     |dense_4_b [id L]
 | |   | |dense_5_W [id M]
 | |   |InplaceDimShuffle{x,0} [id N] ''
 | |     |dense_5_b [id O]
 | |dense_6_W [id P]
 |InplaceDimShuffle{x,0} [id Q] ''
   |dense_6_b [id R]

可视化:

from keras.utils.visualize_util import plot  
plot(vfnet, to_file='model.png') 

Answer 1

Do lines 7-8 just happen once in the start or every time? I mean is it added to the computational graph, are the weights scaled by wt_scale every time?

它们绝对只会发生一次。您访问权重一次并将它们乘以 wt_scale。之后所有训练将照常进行。