python - 层序的输入 0 与预期的 ndim=3 层不兼容，发现 ndim=2。收到完整形状 : [None, 1]

Question

我正在使用 keras 进行文本分类。预处理和矢量化后，我的训练和验证数据详细信息如下所示:

print(X_train.shape, ',', X_train.ndim, ',', type(X_train))
print(y_train.shape, ',', y_train.ndim, ',', type(y_train))
print(X_valid.shape, ',', X_valid.ndim, ',', type(X_valid))
print(y_valid.shape, ',', y_valid.ndim, ',', type(y_valid))
print(data_dim)

输出是:

(14904,) , 1 , <class 'numpy.ndarray'>
(14904,) , 1 , <class 'numpy.ndarray'>
(3725,) , 1 , <class 'numpy.ndarray'>
(3725,) , 1 , <class 'numpy.ndarray'>
15435

那么模型定义是:

model = Sequential()
model.add(LSTM(100, input_shape=(data_dim,1 ), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(200))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics = ['accuracy'])
model.summary()

模型总结:

模型拟合:

model.fit(X_train,y_train, validation_data = (X_valid, y_valid),
          batch_size=batch_size, epochs=epochs)

为什么会出现这个错误？

----> 1 model.fit(X_train,y_train, validation_data = (X_valid, y_valid),
      2           batch_size=batch_size, epochs=epochs)
...
...

    ValueError: Input 0 of layer sequential is incompatible with the layer:
              expected ndim=3, found ndim=2. Full shape received: [None, 1]

Answer 1

我终于在this kaggle notebook的帮助下克服了这个问题.
我将数据维度更改为:

print(X_train.shape)
print(y_train.shape)
print(X_valid.shape)
print(y_valid.shape)
print(X_test.shape)
print(y_test.shape)
print(data_dim)
########################## output ###########################
(14904, 15435)
(14904,)
(3725, 15435)
(3725,)
(5686, 15435)
(5686,)
15435

然后将数据 reshape 为:

X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_valid = np.reshape(X_valid, (X_valid.shape[0], 1, X_valid.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))
########################## output ###########################
(14904, 1, 15435)
(3725, 1, 15435)
(5686, 1, 15435)

终于改了 LSTM input_shape到:

model.add(LSTM(units=50, input_shape=(1, data_dim), return_sequences=True))

现在，模型摘要是:

现在没有问题 model.fit执行得很好。