python - GraphKeys.TRAINABLE_VARIABLES 与 tf.trainable_variables()

Question

是GraphKeys.TRAINABLE_VARIABLES与 tf.trainable_variables() 相同？

是GraphKeys.TRAINABLE_VARIABLES实际上tf.GraphKeys.TRAINABLE_VARIABLES ？

看起来网络成功训练了:

optimizer = tf.train.AdamOptimizer(config.LEARNING_RATE)
with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
    self.train_op = optimizer.minimize(self.loss, var_list=tf.trainable_variables())

但不与

optimizer = tf.train.AdamOptimizer(config.LEARNING_RATE)
with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
    self.train_op = optimizer.minimize(self.loss)

根据documentation :

var_list: Optional list or tuple of Variable objects to update to minimize loss. Defaults to the list of variables collected in the graph under the key GraphKeys.TRAINABLE_VARIABLES.

正如我在批量规范化示例代码 var_list 中看到的那样被省略:

  x_norm = tf.layers.batch_normalization(x, training=training)

  # ...

  update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
  with tf.control_dependencies(update_ops):
    train_op = optimizer.minimize(loss)

Answer 1

如果没有通过var_list到minimize()函数将通过以下方式检索变量(取自 compute_gradients() source code ):

if var_list is None:
      var_list = (
          variables.trainable_variables() +
          ops.get_collection(ops.GraphKeys.TRAINABLE_RESOURCE_VARIABLES))

如果您还没有定义任何ResourceVariable不知何故不在 tf.trainable_variables() 中的实例结果应该是一样的。我的猜测是问题出在其他地方。

您可以尝试在调用 minimize() 之前执行一些测试确保您没有ResourceVariable不在 tf.trainable_variables() 中的 s :

import tensorflow as tf

with tf.Graph().as_default():
    x = tf.placeholder(tf.float32, shape=[None, 2])
    with tf.name_scope('network'):
        logits = tf.layers.dense(x, units=2)

    var_list = (tf.trainable_variables()
                + tf.get_collection(tf.GraphKeys.TRAINABLE_RESOURCE_VARIABLES))
    assert set(var_list) == set(tf.trainable_variables())