gpt4 book ai didi

python - TensorFlow 2.0 : how to group graph using tf. 喀拉斯? tf.name_scope/tf.variable_scope 不再使用了吗?

转载 作者:太空狗 更新时间:2023-10-29 20:29:44 31 4
gpt4 key购买 nike

回到 TensorFlow < 2.0 中,我们过去常常定义层,尤其是更复杂的设置,例如 inception 模块,通过使用 tf.name_scopetf 将它们分组.variable_scope.

利用这些运算符,我们能够方便地构造计算图,从而使 TensorBoard 的图 View 更容易解释。

只是结构化组的一个例子: enter image description here

这对于调试复杂的架构非常方便。

不幸的是,tf.keras 似乎忽略了 tf.name_scope 并且 tf.variable_scope 在 TensorFlow >= 2.0 中消失了。因此,像这样的解决方案......

with tf.variable_scope("foo"):
with tf.variable_scope("bar"):
v = tf.get_variable("v", [1])
assert v.name == "foo/bar/v:0"

...不再可用。有没有替代品?

我们如何在 TensorFlow >= 2.0 中对层和整个模型进行分组?如果我们不对层进行分组,tf.keras 只是将所有内容依次放置在图形 View 中,这会给复杂模型造成很大的困惑。

tf.variable_scope 有替代品吗?到目前为止我找不到任何方法,但大量使用了 TensorFlow < 2.0 中的方法。


编辑:我现在已经为 TensorFlow 2.0 实现了一个示例。这是一个使用 tf.keras 实现的简单 GAN:

# Generator
G_inputs = tk.Input(shape=(100,), name=f"G_inputs")

x = tk.layers.Dense(7 * 7 * 16)(G_inputs)
x = tf.nn.leaky_relu(x)
x = tk.layers.Flatten()(x)
x = tk.layers.Reshape((7, 7, 16))(x)

x = tk.layers.Conv2DTranspose(32, (3, 3), padding="same")(x)
x = tk.layers.BatchNormalization()(x)
x = tf.nn.leaky_relu(x)
x = tf.image.resize(x, (14, 14))

x = tk.layers.Conv2DTranspose(32, (3, 3), padding="same")(x)
x = tk.layers.BatchNormalization()(x)
x = tf.nn.leaky_relu(x)
x = tf.image.resize(x, (28, 28))

x = tk.layers.Conv2DTranspose(32, (3, 3), padding="same")(x)
x = tk.layers.BatchNormalization()(x)
x = tf.nn.leaky_relu(x)

x = tk.layers.Conv2DTranspose(1, (3, 3), padding="same")(x)
x = tf.nn.sigmoid(x)

G_model = tk.Model(inputs=G_inputs,
outputs=x,
name="G")
G_model.summary()

# Discriminator
D_inputs = tk.Input(shape=(28, 28, 1), name=f"D_inputs")

x = tk.layers.Conv2D(32, (3, 3), padding="same")(D_inputs)
x = tf.nn.leaky_relu(x)
x = tk.layers.MaxPooling2D((2, 2))(x)
x = tk.layers.Conv2D(32, (3, 3), padding="same")(x)
x = tf.nn.leaky_relu(x)
x = tk.layers.MaxPooling2D((2, 2))(x)
x = tk.layers.Conv2D(64, (3, 3), padding="same")(x)
x = tf.nn.leaky_relu(x)

x = tk.layers.Flatten()(x)

x = tk.layers.Dense(128)(x)
x = tf.nn.sigmoid(x)
x = tk.layers.Dense(64)(x)
x = tf.nn.sigmoid(x)
x = tk.layers.Dense(1)(x)
x = tf.nn.sigmoid(x)

D_model = tk.Model(inputs=D_inputs,
outputs=x,
name="D")

D_model.compile(optimizer=tk.optimizers.Adam(learning_rate=1e-5, beta_1=0.5, name="Adam_D"),
loss="binary_crossentropy")
D_model.summary()

GAN = tk.Sequential()
GAN.add(G_model)
GAN.add(D_model)
GAN.compile(optimizer=tk.optimizers.Adam(learning_rate=1e-5, beta_1=0.5, name="Adam_GAN"),
loss="binary_crossentropy")

tb = tk.callbacks.TensorBoard(log_dir="./tb_tf2.0", write_graph=True)

# dummy data
noise = np.random.rand(100, 100).astype(np.float32)
target = np.ones(shape=(100, 1), dtype=np.float32)

GAN.fit(x=noise,
y=target,
callbacks=[tb])

这些模型的 TensorBoard 中的图表 看起来像 this .这些层完全是一团糟,模型“G”和“D”(右侧)覆盖了一些困惑。 “GAN”完全不见了。无法正常打开训练操作“Adam”:从左到右绘制的图层太多,到处都是箭头。很难以这种方式检查 GAN 的正确性。


尽管同一 GAN 的 TensorFlow 1.X 实现包含大量“样板代码”...

# Generator
Z = tf.placeholder(tf.float32, shape=[None, 100], name="Z")


def model_G(inputs, reuse=False):
with tf.variable_scope("G", reuse=reuse):
x = tf.layers.dense(inputs, 7 * 7 * 16)
x = tf.nn.leaky_relu(x)
x = tf.reshape(x, (-1, 7, 7, 16))

x = tf.layers.conv2d_transpose(x, 32, (3, 3), padding="same")
x = tf.layers.batch_normalization(x)
x = tf.nn.leaky_relu(x)
x = tf.image.resize_images(x, (14, 14))

x = tf.layers.conv2d_transpose(x, 32, (3, 3), padding="same")
x = tf.layers.batch_normalization(x)
x = tf.nn.leaky_relu(x)
x = tf.image.resize_images(x, (28, 28))

x = tf.layers.conv2d_transpose(x, 32, (3, 3), padding="same")
x = tf.layers.batch_normalization(x)
x = tf.nn.leaky_relu(x)

x = tf.layers.conv2d_transpose(x, 1, (3, 3), padding="same")
G_logits = x
G_out = tf.nn.sigmoid(x)

return G_logits, G_out


# Discriminator
D_in = tf.placeholder(tf.float32, shape=[None, 28, 28, 1], name="D_in")


def model_D(inputs, reuse=False):
with tf.variable_scope("D", reuse=reuse):
with tf.variable_scope("conv"):
x = tf.layers.conv2d(inputs, 32, (3, 3), padding="same")
x = tf.nn.leaky_relu(x)
x = tf.layers.max_pooling2d(x, (2, 2), (2, 2))
x = tf.layers.conv2d(x, 32, (3, 3), padding="same")
x = tf.nn.leaky_relu(x)
x = tf.layers.max_pooling2d(x, (2, 2), (2, 2))
x = tf.layers.conv2d(x, 64, (3, 3), padding="same")
x = tf.nn.leaky_relu(x)

with tf.variable_scope("dense"):
x = tf.reshape(x, (-1, 7 * 7 * 64))

x = tf.layers.dense(x, 128)
x = tf.nn.sigmoid(x)
x = tf.layers.dense(x, 64)
x = tf.nn.sigmoid(x)
x = tf.layers.dense(x, 1)
D_logits = x
D_out = tf.nn.sigmoid(x)

return D_logits, D_out

# models
G_logits, G_out = model_G(Z)
D_logits, D_out = model_D(D_in)
GAN_logits, GAN_out = model_D(G_out, reuse=True)

# losses
target = tf.placeholder(tf.float32, shape=[None, 1], name="target")
d_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=D_logits, labels=target))
gan_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=GAN_logits, labels=target))

# train ops
train_d = tf.train.AdamOptimizer(learning_rate=1e-5, name="AdamD") \
.minimize(d_loss, var_list=tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope="D"))
train_gan = tf.train.AdamOptimizer(learning_rate=1e-5, name="AdamGAN") \
.minimize(gan_loss, var_list=tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope="G"))

# dummy data
dat_noise = np.random.rand(100, 100).astype(np.float32)
dat_target = np.ones(shape=(100, 1), dtype=np.float32)

sess = tf.Session()
tf_init = tf.global_variables_initializer()
sess.run(tf_init)

# merged = tf.summary.merge_all()
writer = tf.summary.FileWriter("./tb_tf1.0", sess.graph)

ret = sess.run([gan_loss, train_gan], feed_dict={Z: dat_noise, target: dat_target})

...结果TensorBoard graph看起来相当干净。请注意右上角的“AdamD”和“AdamGAN”范围是多么干净。您可以直接检查您的优化器是否附加到正确的范围/梯度。

最佳答案

根据社区 RFC Variables in TensorFlow 2.0 :

  • to control variable naming users can use tf.name_scope + tf.Variable

确实,tf.name_scope在 TensorFlow 2.0 中仍然存在,所以你可以这样做:

with tf.name_scope("foo"):
with tf.name_scope("bar"):
v = tf.Variable([0], dtype=tf.float32, name="v")
assert v.name == "foo/bar/v:0"

此外,如上一点所述:

  • the tf 1.0 version of variable_scope and get_variable will be left in tf.compat.v1

所以你可以回到tf.compat.v1.variable_scopetf.compat.v1.get_variable如果你真的需要。

变量范围和 tf.get_variable 可能很方便,但充满了小陷阱和极端情况,特别是因为它们的行为相似但不完全像名称范围,它实际上是一种并行机制.我认为只有名称范围会更加一致和直接。

关于python - TensorFlow 2.0 : how to group graph using tf. 喀拉斯? tf.name_scope/tf.variable_scope 不再使用了吗?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/55318952/

31 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com