python - 如何设计一个神经网络从数组中预测数组

Question

我正在尝试设计一个神经网络，以从包含高斯噪声的数据集数组中预测平滑底层函数的数组。我创建了一个由 10000 个数组组合而成的训练和数据集。现在我试图预测实际函数的数组值，但它似乎失败了，而且准确性也不好。有人可以指导我如何进一步改进我的模型以获得更好的准确性并能够预测好的数据。我使用的代码如下:
用于生成测试和训练数据:

noisy_data = []
pure_data =[]
time = np.arange(1,100)
for i in tqdm(range(10000)):
    array = []
    noise = np.random.normal(0,1/10,99)
    for j in range(1,100):
        array.append( np.log(j))
    array = np.array(array)
    pure_data.append(array)
    noisy_data.append(array+noise)
    

pure_data=np.array(pure_data)
noisy_data=np.array(noisy_data)
    
print(noisy_data.shape)
print(pure_data.shape)

training_size=6000


x_train = noisy_data[:training_size]
y_train = pure_data[:training_size]
x_test = noisy_data[training_size:]
y_test = pure_data[training_size:]
print(x_train.shape)

我的型号:

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape=(99,)))
model.add(tf.keras.layers.Dense(768, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(768, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(99, activation=tf.nn.softmax))

model.compile(optimizer = 'adam',
         loss = 'categorical_crossentropy',
         metrics = ['accuracy'])

model.fit(x_train, y_train, epochs = 20)

精度差的结果:

Epoch 1/20
125/125 [==============================] - 2s 16ms/step - loss: 947533.1875 - accuracy: 0.0000e+00
Epoch 2/20
125/125 [==============================] - 2s 15ms/step - loss: 9756863.0000 - accuracy: 0.0000e+00
Epoch 3/20
125/125 [==============================] - 2s 16ms/step - loss: 30837548.0000 - accuracy: 0.0000e+00
Epoch 4/20
125/125 [==============================] - 2s 15ms/step - loss: 63707028.0000 - accuracy: 0.0000e+00
Epoch 5/20
125/125 [==============================] - 2s 16ms/step - loss: 107545128.0000 - accuracy: 0.0000e+00
Epoch 6/20
125/125 [==============================] - 1s 12ms/step - loss: 161612192.0000 - accuracy: 0.0000e+00
Epoch 7/20
125/125 [==============================] - 1s 12ms/step - loss: 225245360.0000 - accuracy: 0.0000e+00
Epoch 8/20
125/125 [==============================] - 1s 12ms/step - loss: 297850816.0000 - accuracy: 0.0000e+00
Epoch 9/20
125/125 [==============================] - 1s 12ms/step - loss: 378894176.0000 - accuracy: 0.0000e+00
Epoch 10/20
125/125 [==============================] - 1s 12ms/step - loss: 467893216.0000 - accuracy: 0.0000e+00
Epoch 11/20
125/125 [==============================] - 2s 17ms/step - loss: 564412672.0000 - accuracy: 0.0000e+00
Epoch 12/20
125/125 [==============================] - 2s 15ms/step - loss: 668056384.0000 - accuracy: 0.0000e+00
Epoch 13/20
125/125 [==============================] - 2s 13ms/step - loss: 778468480.0000 - accuracy: 0.0000e+00
Epoch 14/20
125/125 [==============================] - 2s 18ms/step - loss: 895323840.0000 - accuracy: 0.0000e+00
Epoch 15/20
125/125 [==============================] - 2s 13ms/step - loss: 1018332672.0000 - accuracy: 0.0000e+00
Epoch 16/20
125/125 [==============================] - 1s 11ms/step - loss: 1147227136.0000 - accuracy: 0.0000e+00
Epoch 17/20
125/125 [==============================] - 2s 12ms/step - loss: 1281768448.0000 - accuracy: 0.0000e+00
Epoch 18/20
125/125 [==============================] - 2s 14ms/step - loss: 1421732608.0000 - accuracy: 0.0000e+00
Epoch 19/20
125/125 [==============================] - 1s 11ms/step - loss: 1566927744.0000 - accuracy: 0.0000e+00
Epoch 20/20
125/125 [==============================] - 1s 10ms/step - loss: 1717172480.0000 - accuracy: 0.0000e+00

和我使用的预测代码:

model.predict([noisy_data[0]])

这会抛出错误:

WARNING:tensorflow:Model was constructed with shape (None, 99) for input Tensor("flatten_5_input:0", shape=(None, 99), dtype=float32), but it was called on an input with incompatible shape (None, 1).


ValueError: Input 0 of layer dense_15 is incompatible with the layer: expected axis -1 of input shape to have value 99 but received input with shape [None, 1]

Answer 1

您正在尝试构建的内容称为 De-noising autoencoder .这里的目标是能够通过人为地在数据集中引入噪声来重建无噪声样本，并将其提供给 encoder ，然后尝试使用 decoder 在没有噪音的情况下重新生成它.

这可以通过任何形式的数据完成，包括图像和文本。
我建议阅读更多关于这方面的内容。有各种概念可以确保模型的正确训练，包括理解中间瓶颈的要求以确保正确的压缩和信息丢失，否则模型只会学习乘以 1 并返回输出。
这是一段示例代码。您可以阅读有关此类架构的更多信息 here ，由 Keras 的作者本人编写。

from tensorflow.keras import layers, Model, utils, optimizers

#Encoder
enc = layers.Input((99,))
x = layers.Dense(128, activation='relu')(enc)
x = layers.Dense(56, activation='relu')(x)
x = layers.Dense(8, activation='relu')(x) #Compression happens here

#Decoder
x = layers.Dense(8, activation='relu')(x)
x = layers.Dense(56, activation='relu')(x)
x = layers.Dense(28, activation='relu')(x)
dec = layers.Dense(99)(x)

model = Model(enc, dec)

opt = optimizers.Adam(learning_rate=0.01)

model.compile(optimizer = opt, loss = 'MSE')

model.fit(x_train, y_train, epochs = 20)

请注意，自动编码器假设输入数据具有一些底层结构，因此可以是 compressed进入一个低维空间，解码器可以使用它来重新生成数据。使用随机生成的序列作为数据可能不会显示出任何好的结果，因为它的压缩不会在没有大量信息丢失的情况下工作，而信息本身没有结构。
正如大多数其他答案所暗示的那样，您没有正确使用激活。由于目标是重新生成具有连续值的 99 维向量，因此不使用 sigmoid 是有意义的，而是使用 tanh照原样 compresses (-1,1)或没有最后一层激活，而不是 gates (0-1)值(value)。

这是一个带有 conv1d 的降噪自编码器和 deconv1d层。这里的问题是输入太简单了。看看您是否可以为输入数据生成更复杂的参数函数。

from tensorflow.keras import layers, Model, utils, optimizers

#Encoder with conv1d
inp = layers.Input((99,))
x = layers.Reshape((99,1))(inp)
x = layers.Conv1D(5, 10)(x)
x = layers.MaxPool1D(10)(x)
x = layers.Flatten()(x)
x = layers.Dense(4, activation='relu')(x) #<- Bottleneck!

#Decoder with Deconv1d
x = layers.Reshape((-1,1))(x)
x = layers.Conv1DTranspose(5, 10)(x)
x = layers.Conv1DTranspose(2, 10)(x)
x = layers.Flatten()(x)
out = layers.Dense(99)(x)

model = Model(inp, out)

opt = optimizers.Adam(learning_rate=0.001)
model.compile(optimizer = opt, loss = 'MSE')
model.fit(x_train, y_train, epochs = 10, validation_data=(x_test, y_test))

Epoch 1/10
188/188 [==============================] - 1s 7ms/step - loss: 2.1205 - val_loss: 0.0031
Epoch 2/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0032 - val_loss: 0.0032
Epoch 3/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0032 - val_loss: 0.0030
Epoch 4/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0031 - val_loss: 0.0029
Epoch 5/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0030 - val_loss: 0.0030
Epoch 6/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0029 - val_loss: 0.0027
Epoch 7/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0028 - val_loss: 0.0029
Epoch 8/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0028 - val_loss: 0.0025
Epoch 9/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0028 - val_loss: 0.0025
Epoch 10/10
188/188 [==============================] - 1s 5ms/step - loss: 0.0026 - val_loss: 0.0024

utils.plot_model(model, show_layer_names=False, show_shapes=True)