python - 从 python 列表创建动态形状的张量以提供 tensorflow RNN

Question

我正在创建一个端到端语音识别架构，其中我的数据是一个分段频谱图列表。我的数据具有 (batch_size, timesteps, 8, 65, 1) 形状，其中 batch_size 是固定的，但 timesteps 是变化的。我不知道如何将这些数据放入具有适当形状的张量中以提供我的模型。这是一段显示我的问题的代码:

import numpy as np
import tensorflow as tf
import tensorflow.keras as keras
from tensorflow.keras.layers import Conv2D, MaxPool2D, Dense, Dropout, Flatten, TimeDistributed
from tensorflow.keras.layers import SimpleRNN, LSTM
from tensorflow.keras import Input, layers
from tensorflow.keras import backend as K

segment_width = 8
segment_height = 65
segment_channels = 1

batch_size = 4

segment_lengths = [28, 33, 67, 43]
label_lengths = [16, 18, 42, 32]

TARGET_LABELS = np.arange(35)

# Generating data
X = [np.random.uniform(0,1, size=(segment_lengths[k], segment_width, segment_height, segment_channels))
     for k in range(batch_size)]

y = [np.random.choice(TARGET_LABELS, size=label_lengths[k]) for k in range(batch_size)]

# Model definition
input_segments_data = tf.keras.Input(name='input_segments_data', shape=(None, segment_width, segment_height, segment_channels),
                               dtype='float32')
input_segment_lengths = tf.keras.Input(name='input_segment_lengths', shape=[1], dtype='int64')
input_label_lengths = tf.keras.Input(name='input_label_lengths', shape=[1], dtype='int64')
# More complex architecture comes here
outputs = Flatten()(input_segments_data)

model = tf.keras.Model(inputs=[input_segments_data, input_segment_lengths, input_label_lengths], outputs = outputs)

def dummy_loss(y_true, y_pred):
  return y_pred

model.compile(optimizer="Adam", loss=dummy_loss)
model.summary()

输出:

Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_segments_data (InputLayer [(None, None, 8, 65, 0                                            
__________________________________________________________________________________________________
input_segment_lengths (InputLay [(None, 1)]          0                                            
__________________________________________________________________________________________________
input_label_lengths (InputLayer [(None, 1)]          0                                            
__________________________________________________________________________________________________
flatten (Flatten)               (None, None)         0           input_segments_data[0][0]        
==================================================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
__________________________________________________________________________________________________

现在，当我尝试根据我的随机数据进行预测时:

model.predict([X, segment_lengths, segment_lengths])

我收到这个错误:

ValueError: Error when checking input: expected input_segments_data to have 5 dimensions, but got array with shape (4, 1)

如何将 X(这是一个数组列表)转换为形状为 (None, None, 8, 65, 1) 的张量并将其提供给我的模型？我不想使用零填充!

Answer 1

Keras 模型以 numpy 数组(张量)作为输入。您不能拥有具有可变时间步长的张量。相反，您可以做的是将所有数据填充成相同的形状，例如使用pad_sequence然后，您可以添加 Masking layer到您的模型以忽略填充值。