python - tensorflow 中不同范围的子网权重共享

Question

使用 tensorflow，我试图在不同变量范围内共享来自相同网络的相同权重以节省内存。但是，似乎没有简单的方法可以做到这一点。我准备了一个小代码示例，以在较小的规模上说明我想对较大的子网执行的操作:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    with tf.variable_scope("super_scope_one"):
        scope1 = tf.variable_scope("sub_scope_one")
        with scope1:
            number_one = tf.get_variable("number_one", shape=[1],
                                         initializer=tf.ones_initializer)
    with tf.variable_scope("super_scope_two"):
        with tf.variable_scope("sub_scope_one", reuse=True) as scope2:
            # Here is the problem.
            # scope1.reuse_variables() # this crashes too if reuse=None.
            number_one = tf.get_variable("number_one", shape=[1])
        with tf.variable_scope("sub_scope_two"):
            number_two = tf.get_variable("number_two", shape=[1],
                                         initializer=tf.ones_initializer)
        number_three = number_one + number_two

    init_op = tf.global_variables_initializer()

with tf.Session(graph=graph):
    init_op.run()
    print(number_three.eval())

有没有办法在不删除的情况下共享两个子作用域中的变量上面的范围？如果不是，是否有充分的理由说明这将是一个坏主意？

Answer 1

您可以在 "super_scope_one" 中简单地定义一次 number_one 并在 "super_scope_two" 中使用它。

不同作用域的两个变量可以一起使用。见下文:

import tensorflow as tf

graph = tf.Graph()
with graph.as_default():
    with tf.variable_scope("super_scope_one"):
        scope1 = tf.variable_scope("sub_scope_one")
        with scope1:
            number_one = tf.get_variable("number_one", shape=[1],
                                         initializer=tf.ones_initializer)
    with tf.variable_scope("super_scope_two"):
        with tf.variable_scope("sub_scope_two"):
            number_two = tf.get_variable("number_two", shape=[1],
                                         initializer=tf.ones_initializer)
        number_three = number_one + number_two

    init_op = tf.global_variables_initializer()

    with tf.Session(graph=graph):
        init_op.run()
        print(number_three.eval())

返回 [2]