python - 具有三个特征的 LSTM 训练，但由于输入形状而无法预测

Question

训练具有三个特征的 LSTM 模型，但在预测时，我收到一个 matmul 错误，看起来预测调用的输入只期望一个特征。

matmul:In[0]:[2,3]，In[1]:[1,101]

这看起来像是三个输入，但该层只需要 1 个输入，如果该层需要 3 个输入(如训练中那样)，那么第二个矩阵将为 [3,101]，并且 matmul 将起作用。

带数据的简化代码和下面的错误。

import tensorflow as tf
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
print(tf.__version__)

lag = 1
epochs = 2
learning_rate = 0.001
dropout = 0.2
feature_count = 3
data_raw = np.array([[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1],[1,1,1]])
dataset = pd.DataFrame(data_raw)
times = pd.date_range('1990-01-01', periods=dataset.shape[0], freq='M')
dataset = dataset.set_index(times)

def MakeLayerLSTMInput(units, input_shape, return_sequences=False):
      return tf.keras.layers.RNN(tf.keras.layers.LSTMCell(units=units, dropout=dropout, input_shape=input_shape), 
      return_sequences=return_sequences)
def MakeLayerLSTM(units, return_sequences=False):
      return tf.keras.layers.RNN(tf.keras.layers.LSTMCell(units=units, dropout=dropout), 
      return_sequences=return_sequences)
def MakeLayerDense(units):
    return tf.keras.layers.Dense(units=units)
def MakeModelStart():
    return tf.keras.models.Sequential()
def MakeOptimizer():
    return tf.train.AdamOptimizer(learning_rate=learning_rate)
def MakeModel():
  model_inputs = (1, feature_count)
  print("model_inputs "+str(model_inputs))
  regressor = MakeModelStart()
  regressor.add(MakeLayerLSTMInput(101, model_inputs))
  regressor.add(MakeLayerDense(1))
  regressor.compile(optimizer = MakeOptimizer(), loss = 'mean_squared_error')
  return regressor

train_dataset = dataset.take(range(10))
test_dataset = dataset.take(range(10,12))
normalize = MinMaxScaler(feature_range = (0, 1))
train_dataset = normalize.fit_transform(train_dataset)
print('train shape: '+str(train_dataset.shape))
x_train = []
y_train = []
for i in range(lag, train_dataset.shape[0]):
  x_train.append(train_dataset[i-lag:i, 0:feature_count])
  y_train.append(train_dataset[i, 0])
x_train, y_train = np.array(x_train), np.array(y_train)
x_train = np.reshape(x_train, (train_dataset.shape[0]-lag, feature_count, 1))
print('normalized x train shape: '+str(x_train.shape))
print('normalized y train shape: '+str(y_train.shape))
regressor = MakeModel()
regressor.fit(x_train, y_train, epochs = epochs, batch_size = x_train.shape[0], verbose=1)

p_count=2
print("last years shape "+str(dataset.shape))
inputs = dataset[len(dataset) - len(test_dataset) - p_count :].values
print("inputs shape "+str(inputs.shape))
inputs = normalize.transform(inputs)
X_test = []
for i in range(p_count, len(inputs)):
    X_test.append(inputs[i-p_count:i, 0:feature_count])
X_test = np.array(X_test)
print("X test shape "+str(X_test.shape))
X_test = np.reshape(X_test, (len(inputs) - p_count, X_test.shape[1], feature_count))
print("X test re-shape "+str(X_test.shape))
predicted = regressor.predict(X_test)
predicted = normalize.inverse_transform(predicted)
print("mse: "+str(mean_squared_error(test_dataset.values, predicted)))

输出:

(p36) D:\src\ml\MLPhase2>python ea-simple1.py
1.13.1
d:\apps\Miniconda3\envs\p36\lib\site-packages\sklearn\preprocessing\data.py:323: DataConversionWarning: Data with input dtype int32 were all converted to float64 by MinMaxScaler.
  return self.partial_fit(X, y)
train shape: (10, 3)
normalized x train shape: (9, 3, 1)
normalized y train shape: (9,)
model_inputs (1, 3)
WARNING:tensorflow:From d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\keras\backend.py:4010: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\keras\utils\losses_utils.py:170: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/2
2019-04-24 13:19:13.920954: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2019-04-24 13:19:14.192634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635
pciBusID: 0000:42:00.0
totalMemory: 11.00GiB freeMemory: 9.03GiB
2019-04-24 13:19:14.204693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2019-04-24 13:19:14.858476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-04-24 13:19:14.863341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0
2019-04-24 13:19:14.866858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N
2019-04-24 13:19:14.872024: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8698 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:42:00.0, compute capability: 7.5)
2019-04-24 13:19:15.863877: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library cublas64_100.dll locally
9/9 [==============================] - 2s 261ms/sample - loss: 0.0000e+00
Epoch 2/2
9/9 [==============================] - 0s 1ms/sample - loss: 0.0000e+00
last years shape (12, 3)
inputs shape (4, 3)
X test shape (2, 2, 3)
X test re-shape (2, 2, 3)
Traceback (most recent call last):
  File "ea-simple1.py", line 69, in <module>
    predicted = regressor.predict(X_test)
  File "d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1113, in predict
    self, x, batch_size=batch_size, verbose=verbose, steps=steps)
  File "d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 329, in model_iteration
    batch_outs = f(ins_batch)
  File "d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\keras\backend.py", line 3076, in __call__
    run_metadata=self.run_metadata)
  File "d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\client\session.py", line 1439, in __call__
    run_metadata_ptr)
  File "d:\apps\Miniconda3\envs\p36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix size-incompatible: In[0]: [2,3], In[1]: [1,101]
         [[{{node while/MatMul_2}}]]
         [[{{node BiasAdd_4}}]]

Answer 1

问题在于 x_train.reshape 与 x_test.reshape 分别作为拟合和预测的输入不匹配。我有(-1,3,1)用于训练输入，(-1,1,3)用于预测输入。这只是我的错误。一点点橡皮鸭会有很大帮助。