machine-learning - 使用 tf.layers 指定的神经网络插入不需要的 reshape 操作

Question

这个用 python 定义的简单神经网络:

input_layer = tf.placeholder(tf.float32, shape=[1, 1, 8000, 1], name='input_layer')

# Convolutional Layer
conv = tf.layers.conv2d(
        inputs=input_layer,
        filters=32,
        kernel_size=[1,11],
        padding="same",
        strides=1,
        activation=tf.nn.relu)

# Output layer
logits = tf.layers.dense(inputs=conv, units=5, name='logit')

结果如下图拓扑:

0 input_layer Placeholder
1 conv2d/kernel/Initializer/random_uniform/shape Const
2 conv2d/kernel/Initializer/random_uniform/min Const
3 conv2d/kernel/Initializer/random_uniform/max Const
4 conv2d/kernel/Initializer/random_uniform/RandomUniform RandomUniform
└─── Input0 ─ conv2d/kernel/Initializer/random_uniform/shape
5 conv2d/kernel/Initializer/random_uniform/sub Sub
└─── Input0 ─ conv2d/kernel/Initializer/random_uniform/max
└─── Input1 ─ conv2d/kernel/Initializer/random_uniform/min
6 conv2d/kernel/Initializer/random_uniform/mul Mul
└─── Input0 ─ conv2d/kernel/Initializer/random_uniform/RandomUniform
└─── Input1 ─ conv2d/kernel/Initializer/random_uniform/sub
7 conv2d/kernel/Initializer/random_uniform Add
└─── Input0 ─ conv2d/kernel/Initializer/random_uniform/mul
└─── Input1 ─ conv2d/kernel/Initializer/random_uniform/min
8 conv2d/kernel VariableV2
9 conv2d/kernel/Assign Assign
└─── Input0 ─ conv2d/kernel
└─── Input1 ─ conv2d/kernel/Initializer/random_uniform
10 conv2d/kernel/read Identity
└─── Input0 ─ conv2d/kernel
11 conv2d/bias/Initializer/zeros Const
12 conv2d/bias VariableV2
13 conv2d/bias/Assign Assign
└─── Input0 ─ conv2d/bias
└─── Input1 ─ conv2d/bias/Initializer/zeros
14 conv2d/bias/read Identity
└─── Input0 ─ conv2d/bias
15 conv2d/convolution/Shape Const
16 conv2d/convolution/dilation_rate Const
17 conv2d/convolution Conv2D
└─── Input0 ─ input_layer
└─── Input1 ─ conv2d/kernel/read
18 conv2d/BiasAdd BiasAdd
└─── Input0 ─ conv2d/convolution
└─── Input1 ─ conv2d/bias/read
19 conv2d/Relu Relu
└─── Input0 ─ conv2d/BiasAdd
20 logit/kernel/Initializer/random_uniform/shape Const
21 logit/kernel/Initializer/random_uniform/min Const
22 logit/kernel/Initializer/random_uniform/max Const
23 logit/kernel/Initializer/random_uniform/RandomUniform RandomUniform
└─── Input0 ─ logit/kernel/Initializer/random_uniform/shape
24 logit/kernel/Initializer/random_uniform/sub Sub
└─── Input0 ─ logit/kernel/Initializer/random_uniform/max
└─── Input1 ─ logit/kernel/Initializer/random_uniform/min
25 logit/kernel/Initializer/random_uniform/mul Mul
└─── Input0 ─ logit/kernel/Initializer/random_uniform/RandomUniform
└─── Input1 ─ logit/kernel/Initializer/random_uniform/sub
26 logit/kernel/Initializer/random_uniform Add
└─── Input0 ─ logit/kernel/Initializer/random_uniform/mul
└─── Input1 ─ logit/kernel/Initializer/random_uniform/min
27 logit/kernel VariableV2
28 logit/kernel/Assign Assign
└─── Input0 ─ logit/kernel
└─── Input1 ─ logit/kernel/Initializer/random_uniform
29 logit/kernel/read Identity
└─── Input0 ─ logit/kernel
30 logit/bias/Initializer/zeros Const
31 logit/bias VariableV2
32 logit/bias/Assign Assign
└─── Input0 ─ logit/bias
└─── Input1 ─ logit/bias/Initializer/zeros
33 logit/bias/read Identity
└─── Input0 ─ logit/bias
34 logit/Tensordot/transpose/perm Const
35 logit/Tensordot/transpose Transpose
└─── Input0 ─ conv2d/Relu
└─── Input1 ─ logit/Tensordot/transpose/perm
36 logit/Tensordot/Reshape/shape Const
37 logit/Tensordot/Reshape Reshape
└─── Input0 ─ logit/Tensordot/transpose
└─── Input1 ─ logit/Tensordot/Reshape/shape
38 logit/Tensordot/transpose_1/perm Const
39 logit/Tensordot/transpose_1 Transpose
└─── Input0 ─ logit/kernel/read
└─── Input1 ─ logit/Tensordot/transpose_1/perm
40 logit/Tensordot/Reshape_1/shape Const
41 logit/Tensordot/Reshape_1 Reshape
└─── Input0 ─ logit/Tensordot/transpose_1
└─── Input1 ─ logit/Tensordot/Reshape_1/shape
42 logit/Tensordot/MatMul MatMul
└─── Input0 ─ logit/Tensordot/Reshape
└─── Input1 ─ logit/Tensordot/Reshape_1
43 logit/Tensordot/shape Const
44 logit/Tensordot Reshape
└─── Input0 ─ logit/Tensordot/MatMul
└─── Input1 ─ logit/Tensordot/shape
45 logit/BiasAdd BiasAdd
└─── Input0 ─ logit/Tensordot
└─── Input1 ─ logit/bias/read

节点 36 和 44 之间的 reshape 操作的目的是什么？我正在使用 Snapdragon 神经处理引擎(SNPE)，它不允许 reshape 操作。有没有办法在没有 reshape 操作的情况下表达这个模型？

Answer 1

所有这些reshape操作都是由tf.tensordot添加的，由 tf.layers.dense 使用对于高维输入(在您的情况下为 4D)。来自其documentation :

Note: if the inputs tensor has a rank greater than 2, then it is flattened prior to the initial matrix multiply by kernel.

The link到源代码。

如果您的环境不需要 reshape ，请尝试手动定义权重和偏差，并通过 tf.matmul 应用 dot 产品。