python - convLSTM 中用于视频分类的 channel 数

Question

我创建了一个 convLSTM 来对灰度视频进行分类，这意味着它们只有一个 channel 。即使我将 1 定义为 channel 数，我也会收到错误消息:

ValueError: Error when checking input: expected conv_lst_m2d_1_inputto have 5 dimensions, but got array with shape (128, 176, 256, 256)

128是训练数据集的大小，176*256是每帧的分辨率，256是每段视频的帧数。

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.20, shuffle=True, random_state=0)
 
model = Sequential()
model.add(ConvLSTM2D(filters = 64, kernel_size = (3, 3), return_sequences = False, data_format = "channels_last", input_shape = (seq_len, img_height, img_width, 1)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(256, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(6, activation = "softmax"))
 
model.summary()
 
opt = keras.optimizers.SGD(lr=0.001)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=["accuracy"])
 
earlystop = EarlyStopping(patience=7)
callbacks = [earlystop]

history = model.fit(x = X_train, y = y_train, epochs=40, batch_size = 8 , shuffle=True, validation_split=0.2, callbacks=callbacks)

Answer 1

您只需要扩展数据的最后一个维度

batch_dim, seq_len, img_height, img_width = 3, 17, 25, 25
X = np.random.uniform(0,1, (batch_dim, seq_len, img_height, img_width))
y = np.random.randint(0,6, batch_dim)
print(X.shape)

# expand input dimension
X = X[...,np.newaxis]
print(X.shape)

model = Sequential()
model.add(ConvLSTM2D(filters = 64, kernel_size = (3, 3), return_sequences = False, 
                     data_format = "channels_last", 
                     input_shape = (seq_len, img_height, img_width, 1)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(256, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(6, activation = "softmax"))
model.summary()

model.predict(X).shape