tensorflow - ValueError : Input 0 of layer "model" is incompatible with the layer: expected shape=(None, 50), found shape=(None, 1, 512)

Question

学习使用bert-base-cased和分类模型......模型的代码如下:

def mao_func(input_ids, masks, labels):
return {'input_ids':input_ids, 'attention_mask':masks}, labels

dataset = dataset.map(mao_func)

BATCH_SIZE = 32
dataset = dataset.shuffle(100000).batch(BATCH_SIZE)

split = .8
ds_len = len(list(dataset))

train = dataset.take(round(ds_len * split))
val = dataset.skip(round(ds_len * split))

from transformers import TFAutoModel
bert = TFAutoModel.from_pretrained('bert-base-cased')

模型:“tf_bert_model”

---

图层(类型)输出形状参数#

bert (TFBertMainLayer) 多个 108310272

============================================= ==================总参数:108,310,272可训练参数:108,310,272不可训练的参数:0

然后是神经网络构建:

input_ids = tf.keras.layers.Input(shape=(50,), name='input_ids', dtype='int32')
mask = tf.keras.layers.Input(shape=(50,), name='attention_mask', dtype='int32')

embeddings = bert(input_ids, attention_mask=mask)[0]

X = tf.keras.layers.GlobalMaxPool1D()(embeddings)
X = tf.keras.layers.BatchNormalization()(X)
X = tf.keras.layers.Dense(128, activation='relu')(X)
X = tf.keras.layers.Dropout(0.1)(X)
X = tf.keras.layers.Dense(32, activation='relu')(X)
y = tf.keras.layers.Dense(3, activation='softmax',name='outputs')(X)

model = tf.keras.Model(inputs=[input_ids, mask], outputs=y)

model.layers[2].trainable = False

model.summary 是:

 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_ids (InputLayer)         [(None, 50)]         0           []                               
                                                                                                  
 attention_mask (InputLayer)    [(None, 50)]         0           []                               
                                                                                                  
 tf_bert_model (TFBertModel)    TFBaseModelOutputWi  108310272   ['input_ids[0][0]',              
                                thPoolingAndCrossAt               'attention_mask[0][0]']         
                                tentions(last_hidde                                               
                                n_state=(None, 50,                                                
                                768),                                                             
                                 pooler_output=(Non                                               
                                e, 768),                                                          
                                 past_key_values=No                                               
                                ne, hidden_states=N                                               
                                one, attentions=Non                                               
                                e, cross_attentions                                               
                                =None)                                                            
                                                                                                  
 global_max_pooling1d (GlobalMa  (None, 768)         0           ['tf_bert_model[0][0]']          
 xPooling1D)                                                                                      
                                                                                                  
 batch_normalization (BatchNorm  (None, 768)         3072        ['global_max_pooling1d[0][0]']   
 alization)                                                                                       
                                                                                                  
 dense (Dense)                  (None, 128)          98432       ['batch_normalization[0][0]']    
                                                                                                  
 dropout_37 (Dropout)           (None, 128)          0           ['dense[0][0]']                  
                                                                                                  
 dense_1 (Dense)                (None, 32)           4128        ['dropout_37[0][0]']             
                                                                                                  
 outputs (Dense)                (None, 3)            99          ['dense_1[0][0]']                
                                                                                                  
==================================================================================================
Total params: 108,416,003
Trainable params: 104,195
Non-trainable params: 108,311,808
__________________________________________________________________________________________________

最后是模型拟合

optimizer = tf.keras.optimizers.Adam(0.01)
loss = tf.keras.losses.CategoricalCrossentropy()
acc = tf.keras.metrics.CategoricalAccuracy('accuracy')

model.compile(optimizer,loss=loss, metrics=[acc])

history = model.fit(
    train,
    validation_data = val,
     epochs=140
)

第 7 行执行错误 -> model.fit(...):

ValueError: Input 0 of layer "model" is incompatible with the layer: expected shape=(None, 50), found shape=(None, 1, 512)

谁能帮我解决我做错了什么以及为什么...谢谢:)

更新:这是代码为 https://github.com/CharlieArreola/OnlinePosts 的 git

Answer 1

看来，您的训练数据形状与输入层的预期输入形状不匹配。您可以使用 train.shape()

检查火车数据的形状

您输入层 Input_ids = tf.keras.layers.Input(shape=(50,), name='input_ids', dtype='int32') 期望训练数据为 50 列，但如果我们查看您的错误，您很可能有 512。因此，要解决此问题，您只需更改输入形状即可。

Input_ids = tf.keras.layers.Input(shape=(512,), name='input_ids', dtype='int32')

如果您在数据集中拆分 x 和 y，您可以通过以下方式使其更加灵活:

Input_ids = tf.keras.layers.Input(shape=(train_x.shape[0],), name='input_ids', dtype='int32')

另外不要忘记，您必须对所有输入层进行此更改!