machine-learning - 使用 keras 示例 pretrained_word_embeddings 时出错

Question

我正在尝试修改可用的 keras 示例 pretrained_word_embeddings here我遇到了以下问题:如果我将 MAX_SEQUENCE_LENGTH varibae 减少到 95 值，我会收到以下错误:

Traceback (most recent call last): File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 670, in _call_cpp_shape_fn_impl status) File "C:\Program Files\Anaconda3\lib\contextlib.py", line 66, in exit next(self.gen) File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status pywrap_tensorflow.TF_GetCode(status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Negative dimension size caused by subtracting 5 from 2 for 'Conv2D_2' (op: 'Conv2D') with input shapes: [?,2,1,128], [5,1,128,128].

当我需要处理像推文这样的小消息时，我需要更改它。我使用 Tensorflow 后端。

请帮我澄清一下1)MAX_SEQUENCE_LENGTH有什么问题？2) 为什么我在模型中使用的是 Conv2D_2 而不是 Conv1D？

Answer 1

让我们回顾一下网络定义并分析 MAX_SEQUENCE_LENGTH=95 时层输出的形状:

sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
# Output shape: (95, EMBEDDING_DIM)
x = Conv1D(128, 5, activation='relu')(embedded_sequences)
# Output shape: (91, 128) (because of valid border mode) 
x = MaxPooling1D(5)(x)
# Output shape: (18, 128)
x = Conv1D(128, 5, activation='relu')(x)
# Output shape: (14, 128)
x = MaxPooling1D(5)(x)
# Output shape: (2, 128)
x = Conv1D(128, 5, activation='relu')(x)
# Output shape: (2 - 4??, 128) - PROBLEM!!
x = MaxPooling1D(35)(x) # In the easiest way - change 35 to 2.
x = Flatten()(x)
x = Dense(128, activation='relu')(x)
preds = Dense(100, activation='softmax')(x)

正如您所看到的，问题在于最后一个 Conv1D 层，其中没有足够的维度来应用具有 valid 边界模式的卷积。有很多方法可以解决这个问题。最简单的方法是裁剪最后一个 Conv-MaxPool duo 并将网络定义更改为:

sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
# Output shape: (95, EMBEDDING_DIM)
x = Conv1D(128, 5, activation='relu')(embedded_sequences)
# Output shape: (91, 128) (because of valid border mode) 
x = MaxPooling1D(5)(x)
# Output shape: (18, 128)
x = Conv1D(128, 5, activation='relu')(x)
# Output shape: (14, 128)
x = MaxPooling1D(5)(x)
# Output shape: (2, 128)
x = Flatten()(x) # Here - everything is ok.
x = Dense(128, activation='relu')(x)
preds = Dense(100, activation='softmax')(x)

当然 - 还有更多方法可以做到这一点(例如调整池大小等)。

使用 Conv2D 的原因在于，在 TensorFlow 后端的情况下，Conv1D 是通过二维卷积实现的，其中一个尺寸被压缩为 1。