tensorflow - 如何在每个纪元后重置 tensorflow 中 GRU 的状态

Question

我正在使用 tensorflow GRU 单元来实现 RNN。我将上述内容用于最长 5 分钟的视频。因此，由于下一个状态会自动输入到 GRU 中，因此如何在每个时期之后手动重置 RNN 的状态。换句话说，我希望训练开始时的初始状态始终为 0。这是我的代码片段:

with tf.variable_scope('GRU'):
    latent_var = tf.reshape(latent_var, shape=[batch_size, time_steps, latent_dim])

    cell = tf.nn.rnn_cell.GRUCell(cell_size)   
    H, C = tf.nn.dynamic_rnn(cell, latent_var, dtype=tf.float32)  
    H = tf.reshape(H, [batch_size, cell_size]) 
....

非常感谢任何帮助!

Answer 1

使用tf.nn.dynamic_rnn的initial_state参数:

initial_state: (optional) An initial state for the RNN. If cell.state_size is an integer, this must be a Tensor of appropriate type and shape [batch_size, cell.state_size]. If cell.state_size is a tuple, this should be a tuple of tensors having shapes [batch_size, s] for s in cell.state_size.

文档中的改编示例:

# create a GRUCell
cell = tf.nn.rnn_cell.GRUCell(cell_size)

# 'outputs' is a tensor of shape [batch_size, max_time, cell_state_size]

# defining initial state
initial_state = cell.zero_state(batch_size, dtype=tf.float32)

# 'state' is a tensor of shape [batch_size, cell_state_size]
outputs, state = tf.nn.dynamic_rnn(cell, input_data,
                                   initial_state=initial_state,
                                   dtype=tf.float32)

<小时/>

另请注意，尽管 initial_state 不是占位符，您也可以向其提供值。因此，如果希望保留一个纪元内的状态，但在纪元开始时从零开始，您可以这样做:

# Compute the zero state array of the right shape once
zero_state = sess.run(initial_state)

# Start with a zero vector and update it 
cur_state = zero_state
for batch in get_batches():
  cur_state, _ = sess.run([state, ...], feed_dict={initial_state=cur_state, ...})