machine-learning - 神经网络盲猜

Question

我正在尝试训练简单的神经网络包括:

1. 卷积层过滤器 (5x5) x 8，步长 2。
2. 最大池化 25x25(图像的细节量有点少)
3. 将输出扁平化为 (2x2x8) 向量
4. 采用逻辑回归的分类器

网络总共有 < 1000 个权重。

文件:nn.py

#!/bin/python 
import tensorflow as tf
import create_batch

# Prepare data
batch = create_batch.batch

x = tf.reshape(batch[0], [-1,100,100,3])
y_ = batch[1]


# CONVOLUTION NETWORK

# For initialization  
def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.3)
  return tf.Variable(initial)

def bias_variable(shape):
  initial = tf.constant(0.2, shape=shape)
  return tf.Variable(initial)

# Convolution with stride 1
def conv2d(x, W):
  return tf.nn.conv2d(x, W, strides=[1, 2, 2, 1], padding='SAME')

def max_pool_25x25(x):
  return tf.nn.max_pool(x, ksize=[1, 25, 25, 1],
                    strides=[1, 25, 25, 1], padding='SAME')

# First layer
W_conv1 = weight_variable([5, 5, 3, 8])
b_conv1 = bias_variable([8])

x_image = tf.reshape(x, [-1,100,100,3])

# First conv1
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_25x25(h_conv1)


# Dense connection layer
# make data flat
W_fc1 = weight_variable([2 * 2 * 8, 2])
b_fc1 = bias_variable([2])

h_pool1_flat = tf.reshape(h_pool1, [-1, 2*2*8])
y_conv = tf.nn.softmax(tf.matmul(h_pool1_flat, W_fc1) + b_fc1)

#Learning
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv),    reduction_indices=[1]))
train_step =    tf.train.GradientDescentOptimizer(0.001).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

# Session
sess = tf.Session()
sess.run(tf.initialize_all_variables())

# Start input enqueue threads.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)

for i in range(200):
  if i%10 == 0:
    train_accuracy = accuracy.eval(session=sess)
    print("step %d, training accuracy %g"%(i, train_accuracy))

  train_step.run(session=sess)

文件:create_batch.py​​

#!/bin/python
import tensorflow as tf

PATH1 = "../dane/trening/NK/"
PATH2 = "../dane/trening/K/"


def create_labeled_image_list():

    filenames = [(PATH1 + "nk_%d.png" % i) for i in range(300)]
    labels = [[1,0] for i in range(300)]

    filenames += [(PATH2 + "kulki_%d.png" % i) for i in range(300)]
    labels += [[0,1] for i in range(300)]

    return filenames, labels

def read_images_from_disk(input_queue):
    label = input_queue[1]
    file_contents = tf.read_file(input_queue[0])
    example = tf.image.decode_png(file_contents, channels=3)
    example.set_shape([100, 100, 3])
    example = tf.to_float(example)
    print ("READ, label:")
    print(label)
    return example, label

# Start
image_list, label_list = create_labeled_image_list()

# Create appropriate tensors for naming
images = tf.convert_to_tensor(image_list, dtype=tf.string)
labels = tf.convert_to_tensor(label_list, dtype=tf.float32)

input_queue = tf.train.slice_input_producer([images, labels],
                                        shuffle=True)

image, label = read_images_from_disk(input_queue)
batch = tf.train.batch([image, label], batch_size=600)

我正在输入 100x100 的图像，我有两个类，每个类 300 张图像。基本上，第 0 步随机初始化的网络比经过训练的网络具有更好的准确性。网络在达到 0.5 精度后停止学习(基本上是抛硬币)。图像包含蓝色的东西(第 1 类)或草(第 2 类)。

我正在一次使用整个图像集(600张图像)训练网络，损失函数是交叉熵。

我做错了什么？

Answer 1

好的，我已经找到了两个错误的修复方法，现在网络正在学习。

1. 图像是 RGBA，尽管我在 tf 中将它们声明为 RGB
2. 我没有将图像标准化为 [-1,1] float32。

在 tensorflow 中，应该这样做:

# i use "im" for image
tf.image.convert_image_dtype(im, dtype=float32)
im = tf.sub(im, -0.5)
im = tf.mul(im, 2.0)

致所有机器学习新手 - 谨慎准备数据!

谢谢。