python-3.x - 连接模型时，Keras 出现无梯度错误

Question

我正在尝试使用带有分层 RNN 模型的 Keras 实现视觉讲故事模型，基本上是神经图像字幕风格，但在解码器 RNN 之上使用双向 RNN 的一系列照片。

我分别实现并测试了这个模型的三个部分，CNN、BRNN 和解码器 RNN，但是在尝试连接它们时出现了这个错误:

ValueError: An operation has None for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.

我的代码如下:

#vgg16 model with the fc2 layer as output
cnn_base_model = self.cnn_model.base_model
brnn_model = self.brnn_model.model
rnn_model = self.rnn_model.model

cnn_part = TimeDistributed(cnn_base_model)

img_input = Input((self.story_length,) + self.cnn_model.input_shape, name='brnn_img_input')

extracted_feature = cnn_part(img_input)

#[None, 5, 512], a 512 length vector for each picture in the story
brnn_feature = brnn_model(extracted_feature)

#[None, 5, 25], input groundtruth word indices fed as input when training
decoder_input = Input((self.story_length, self.max_length), name='brnn_decoder_input')

decoder_outputs = []

for i in range(self.story_length):
  #separate timesteps for decoding
  decoder_input_i = Lambda(lambda x: x[:, i, :])(decoder_input)
  brnn_feature_i = Lambda(lambda x: x[:, i, :])(brnn_feature)

  #the problem persists when using Dense instead of the Lambda layers above
  #decoder_input_i = Dense(25)(Reshape((125,))(decoder_input))
  #brnn_feature_i = Dense(512)(Reshape((5 * 512,))(brnn_feature))

  decoder_output_i = rnn_model([decoder_input_i, brnn_feature_i])
  decoder_outputs.append(decoder_output_i)

decoder_output = Concatenate(axis=-2, name='brnn_decoder_output')(decoder_outputs)

self.model = Model([img_input, decoder_input], decoder_output)

以及 BRNN 的代码:

image_feature = Input(shape=(self.story_length, self.img_feature_dim,))
image_emb = TimeDistributed(Dense(self.lstm_size))(image_feature)

brnn = Bidirectional(LSTM(self.lstm_size, return_sequences=True), merge_mode='concat')(image_emb)
brnn_emb = TimeDistributed(Dense(self.lstm_size))(brnn)

self.model = Model(inputs=image_feature, outputs=brnn_emb)

和RNN:

#[None, 512], the vector to be decoded
initial_input = Input(shape=(self.input_dim,), name='rnn_initial_input')

#[None, 25], the groundtruth word indices fed as input when training 
decoder_inputs = Input(shape=(None,), name='rnn_decoder_inputs')

decoder_input_masking = Masking(mask_value=0.0)(decoder_inputs)
decoder_input_embeddings = Embedding(self.vocabulary_size, self.emb_size,
  embeddings_regularizer=l2(regularizer))(decoder_input_masking)
decoder_input_dropout = Dropout(.5)(decoder_input_embeddings)

initial_emb = Dense(self.emb_size,
  kernel_regularizer=l2(regularizer))(initial_input)

initial_reshape = Reshape((1, self.emb_size))(initial_emb)
initial_masking = Masking(mask_value=0.0)(initial_reshape)
initial_dropout = Dropout(.5)(initial_masking)

decoder_lstm = LSTM(self.hidden_dim, return_sequences=True, return_state=True,
  recurrent_regularizer=l2(regularizer),
  kernel_regularizer=l2(regularizer),
  bias_regularizer=l2(regularizer))

_, initial_hidden_h, initial_hidden_c = decoder_lstm(initial_dropout)

decoder_outputs, decoder_state_h, decoder_state_c = decoder_lstm(decoder_input_dropout,
  initial_state=[initial_hidden_h, initial_hidden_c])

decoder_output_dense_layer = TimeDistributed(Dense(self.vocabulary_size, activation='softmax',
  kernel_regularizer=l2(regularizer)))

decoder_output_dense = decoder_output_dense_layer(decoder_outputs)

self.model = Model([decoder_inputs, initial_input], decoder_output_dense)

我正在使用 adam作为优化器和 sparse_categorical_crossentropy作为损失。

起初我以为问题出在 Lambda用于分割时间步长的层，但当我用 Dense 替换它们时问题仍然存在层(这是保证

Answer 1

我有一个类似的错误，结果我想在 中构建层(在我的自定义层或模型中)初始化 () 像这样:

self.lstm_custom_1 = keras.layers.LSTM(128,batch_input_shape=batch_input_shape, return_sequences=False,stateful=True)
self.lstm_custom_1.build(batch_input_shape)

self.dense_custom_1 = keras.layers.Dense(32,  activation = 'relu')
self.dense_custom_1.build(input_shape=(batch_size, 128))```