python - 当我增加神经元或层数时，我的 tensorflow 模型无法训练

Question

我引用tensorflow教程[1]做了一个使用tensorflow识别手写体的卷积神经网络模型。该模型使用卷积filter1:[5,5,1,16],filter2:[5,5,16, 32]，完全组合层[7*7*32,1024]和[1024,10]，然后使用softmax将其转换为概率。我运行这个模型并失败了，因为“损失”并没有减少，并且所有输出都是 [0,0,1,0,0,0,0,0,0,0,0]。

然后，我减少了过滤器和神经元的数量，结果成功了，准确率约为 97%。

为什么我用相同数量的滤波器和神经元制作模型却无法训练成功？

这是我失败的模型。(我使用“mnist.csv”)

x = tf.placeholder(tf.float32,[None,28*28])
t = tf.placeholder(tf.float32,[None,10])
def weight(shape):
   init = tf.truncated_normal(shape, stddev=0.1)
   return tf.Variable(init)
def bias(shape):
   init = tf.constant(0.1, shape=shape)
   return tf.Variable(init)

def conv2d(x,W):
   return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding="SAME")
def max_pool_22(x):
   return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding="SAME")

W_conv1 = weight([5,5,1,16])
b_conv1 = bias([16])

x_image = tf.reshape(x,[-1,28,28,1])


h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)

h_pool1 = max_pool_22(h_conv1)
print(h_pool1.shape)

W_conv2 = weight([5,5,16,64])
b_conv2 = bias([64])

h_conv2 = tf.nn.relu(conv2d(h_pool1,W_conv2) + b_conv2)
h_pool2 = max_pool_22(h_conv2)
W_fc1 = weight([7*7*64,1024])
b_fc1 = bias([1024])

h_pool2_flat = tf.reshape(h_pool2,[-1,7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1)

W_fc2 = weight([1024,10])
b_fc2 = bias([10])

prediction = tf.nn.softmax(tf.matmul(h_fc1,W_fc2) + b_fc2) 
cross_entropy=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=t,logits=prediction))
train_step = tf.train.AdamOptimizer().minimize(cross_entropy)

correct_prediction =tf.equal(tf.argmax(prediction,1),tf.argmax(t,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))

sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
for epoch in range(20):
   avg_loss = 0.
   avg_accuracy = 0.
   for i in range(1000):
       ind = np.random.choice(len(x_train),50)
       x_train_batch = x_train[ind]
       t_train_batch = t_train[ind]
       _, loss, a = sess.run([train_step,cross_entropy, accuracy],feed_dict={x:x_train_batch,t:t_train_batch})
       avg_loss += loss/1000
       avg_accuracy += a/1000
   if epoch % 1 == 0:
      print("Step:{0} Loss:{1} TrainAccuracy:{2}".format(epoch,avg_loss,avg_accuracy))

print("test_accuracy:{0}".format(accuracy.eval(feed_dict={x:x_test,t:t_test})))

[1]:https://www.tensorflow.org/get_started/mnist/pros 在此处输入代码

Answer 1

您正在对 softmax 的输出调用 softmax_cross_entropy_with_logits。这会应用 softmax 两次，从而导致错误的结果。在应用 softmax 之前，应在最后一层的线性输出上调用 softmax_cross_entropy_with_logits:

y = tf.matmul(h_fc1,W_fc2) + b_fc2
cross_entropy=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=t, logits=y))

prediction_probabilities = tf.nn.softmax(y)
prediction_class = tf.argmax(y, 1)

仅当您需要每个类的概率时，才需要上面的 prediction_probabilities 张量。否则，您可以直接在 y 上调用 argmax 来获取预测类别。