python - Tensorflow InvalidArgumentError : 2 root error(s) found. 索引 [28,0] = 11292 不在 [0, 11272)

Question

我已经使用 Keras Sequential API 创建了一个模型，并使用了 Glove pretraining embeddings

def create_model(
        input_length=20,
        output_length=20):

    encoder_input = tf.keras.Input(shape=(input_length,))
    decoder_input = tf.keras.Input(shape=(output_length,))

    encoder = tf.keras.layers.Embedding(original_embedding_matrix.shape[0], original_embedding_dim, weights=[original_embedding_matrix], mask_zero=True)(encoder_input)
    encoder, h_encoder, u_encoder = tf.keras.layers.LSTM(64, return_state=True)(encoder)

    decoder = tf.keras.layers.Embedding(clone_embedding_matrix.shape[0], clone_embedding_dim, weights=[clone_embedding_matrix], mask_zero=True)(decoder_input)
    decoder = tf.keras.layers.LSTM(64, return_sequences=True)(decoder, initial_state=[h_encoder, u_encoder])
    decoder = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(clone_vocab_size+1))(decoder)

    model = tf.keras.Model(inputs=[encoder_input, decoder_input], outputs=[decoder])
    model.compile(optimizer='adam', loss=tf.keras.losses.MeanSquaredError(), metrics=['accuracy'])

    return model

model = create_model()

这是我的编码器/解码器形状:

training_encoder_input.shape --> (2500, 20) 
training_decoder_input.shape --> (2500, 20) 
training_decoder_output.shape ---> (2500, 20, 11272) 
clone_vocab_size ---> 11271

model.summary() 的输出:

Model: "functional_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 20)]         0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            [(None, 20)]         0                                            
__________________________________________________________________________________________________
embedding (Embedding)           (None, 20, 50)       564800      input_1[0][0]                    
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 20, 50)       563600      input_2[0][0]                    
__________________________________________________________________________________________________
lstm (LSTM)                     [(None, 64), (None,  29440       embedding[0][0]                  
__________________________________________________________________________________________________
lstm_1 (LSTM)                   (None, 20, 64)       29440       embedding_1[0][0]                
                                                                 lstm[0][1]                       
                                                                 lstm[0][2]                       
__________________________________________________________________________________________________
time_distributed (TimeDistribut (None, 20, 11272)    732680      lstm_1[0][0]                     
==================================================================================================
Total params: 1,919,960
Trainable params: 1,919,960
Non-trainable params: 0
__________________________________________________________________________________________________

但是当我尝试训练模型时:

model.fit(x=[training_encoder_input, training_decoder_input],
          y=training_decoder_output,
          verbose=2,
          batch_size=128,
          epochs=10)

我收到此错误:

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  indices[28,0] = 11292 is not in [0, 11272)
     [[node functional_1/embedding_1/embedding_lookup (defined at <ipython-input-11-967d0351a90e>:31) ]]
  (1) Invalid argument:  indices[28,0] = 11292 is not in [0, 11272)
     [[node functional_1/embedding_1/embedding_lookup (defined at <ipython-input-11-967d0351a90e>:31) ]]
     [[broadcast_weights_1/assert_broadcastable/AssertGuard/else/_13/broadcast_weights_1/assert_broadcastable/AssertGuard/Assert/data_7/_78]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_13975]

Errors may have originated from an input operation.
Input Source operations connected to node functional_1/embedding_1/embedding_lookup:
 functional_1/embedding_1/embedding_lookup/8859 (defined at /usr/lib/python3.6/contextlib.py:81)

Input Source operations connected to node functional_1/embedding_1/embedding_lookup:
 functional_1/embedding_1/embedding_lookup/8859 (defined at /usr/lib/python3.6/contextlib.py:81)

Function call stack:
train_function -> train_function

已经有人问了 this question但是没有一个响应对我有用，可能错误在损失函数内或在嵌入层的词汇表内，但我无法弄清楚到底是什么问题。

Answer 1

解决方案其实很简单，在错误中:

(0) Invalid argument:  indices[28,0] = 11292 is not in [0, 11272)

11292是一个输入元素(映射到我的 Tokenizer 字典中的一个词)

11272是我词汇的长度

为什么我有一个带数字的词 11292如果我的标记器的长度只是 11272 ?

我有两个标记器，一个用于输入，另一个用于输出，因此解决方案是采用较小的 ans 的长度在模型中使用它。

您还可以限制在 Tensorflow 中的分词器中使用的单词数:

tokenizer = Tokenizer(num_words=20000)

它将取 20000 个重复最多的单词。