python - Keras:加载模型的训练速度慢了 6 倍

Question

TensorFlow 2.0(GPU 版)、CUDA 10.0、NVIDIA GeForce RTX 2060、Windows 10 (1903)

我在做文本分类。我使用字符嵌入和 LSTM 将每个单词编码为一个向量。这是我的模型:

import tensorflow as tf
from tensorflow.keras.models import Sequential, Model, load_model, model_from_json
from tensorflow.keras.layers import Input, Embedding, LSTM, TimeDistributed, Multiply, Masking, Concatenate, Dense
from tensorflow.keras.utils import to_categorical

import numpy as np

import re

.

target_word_length = 15
target_sentence_length = 100
mask_embedding = 27

def NetEncoder(string):

    d = dict(zip('abcdefghijklmnopqrstuvwxyz', range(1,27)))

    text_enc = []

    for w in re.findall('[a-z]+', string.lower()):
        word_enc = []
        for c in w:
            word_enc.append(d[c])
        if len(word_enc) < target_word_length:
            for i in range(target_word_length - len(word_enc)):
                word_enc.append(0)
        text_enc.append(word_enc)
        
    const = []
    for i in range(target_word_length):
        const.append(0)
    const[0] = mask_embedding

    sentence_length = len(text_enc)
    if sentence_length < target_sentence_length:
        for i in range(target_sentence_length - sentence_length):
            text_enc.append(const)
        mask = []
        for i in range(target_sentence_length):
            if i < sentence_length:
                mask.append([1])
            else:
                mask.append([0])
            
    return [np.array(text_enc, dtype=np.float32), np.array(mask, dtype=np.float32)]

.

emb = Sequential([
    Embedding(28, 3, mask_zero=True, input_shape=(target_word_length,)),
    LSTM(16, return_sequences=False)
])

input_1 = Input(shape=(target_sentence_length, target_word_length))
mask_1 = Input(shape=(target_sentence_length, 1))
net_1 = TimeDistributed(emb)(input_1)
net_1 = Multiply()([net_1, mask_1])
net_1 = Masking(mask_value=np.full(16, 0.))(net_1)
net_1 = LSTM(64, return_sequences=False)(net_1)

input_2 = Input(shape=(target_sentence_length, target_word_length))
mask_2 = Input(shape=(target_sentence_length, 1))
net_2 = TimeDistributed(emb)(input_2)
net_2 = Multiply()([net_2, mask_2])
net_2 = Masking(mask_value=np.full(16, 0.))(net_2)
net_2 = LSTM(64, return_sequences=False)(net_2)

cat = Concatenate()([net_1, net_2])
clf = Dense(32, activation='relu')(cat)
clf = Dense(2, activation='softmax')(clf)

model = Model(inputs=[input_1, mask_1, input_2, mask_2], outputs=clf)

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

[input_1, mask_1] = NetEncoder('Hello Max')
[input_2, mask_2] = NetEncoder('Hello Max')
Y = to_categorical([0], num_classes=2)

model.fit([[input_1], [mask_1], [input_2], [mask_2]], Y, epochs=10)

训练时间:每个样本 23 毫秒

Train on 1 samplesEpoch 1/104/1 [==============================================] - 16s 4s/sample - loss: 0.6840 - accuracy: 0.7500Epoch 2/104/1 [==============================================] - 0s 23ms/sample - loss: 0.6618 - accuracy: 1.0000Epoch 3/104/1 [==============================================] - 0s 23ms/sample - loss: 0.6335 - accuracy: 1.0000

Prediction time: 63 ms per sample

%%timemodel.predict([[input_1], [mask_1], [input_2], [mask_2]])

Wall time: 63 ms

array([[1.0000000e+00, 2.6085422e-08]], dtype=float32)

Save and load model (#1)

model.save('model.h5', save_format='tf', include_optimizer=False)

model_1 = load_model('model.h5')

预测时间更短:37 与 63 毫秒。但输出略有不同。

%%timemodel_1.predict([[input_1], [mask_1], [input_2], [mask_2]])

Wall time: 37.4 ms

array([[1.000000e+00, 2.608547e-08]], dtype=float32)

model_1.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model_1.fit([[input_1], [mask_1], [input_2], [mask_2]], Y, epochs=10)

训练速度慢 6 倍:每个样本 139 毫秒对 23 毫秒!

Train on 1 samplesEpoch 1/104/1 [==============================================] - 4s 1s/sample - loss: 0.0000e+00 - accuracy: 1.0000Epoch 2/104/1 [==============================================] - 1s 133ms/sample - loss: 0.0000e+00 - accuracy: 1.0000Epoch 3/104/1 [==============================================] - 1s 139ms/sample - loss: 0.0000e+00 - accuracy: 1.0000

Save and load model (#2)

model.save_weights('model_V2.h5')
model_json = model.to_json()
with open('model_V2.json', "w") as json_file:
    json_file.write(model_json)
json_file.close()

json_file = open("model_V2.json", 'r')
loaded_model_json = json_file.read()
json_file.close()

model_2 = model_from_json(loaded_model_json)
#model_2.load_weights("model_V2.h5")

model_2.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model_2.fit([[input_1], [mask_1], [input_2], [mask_2]], Y, epochs=10)

训练时间:每个样本 149 与 23 毫秒!

Train on 1 samplesEpoch 1/104/1 [==============================================] - 4s 1s/sample - loss: 0.6805 - accuracy: 0.7500Epoch 2/104/1 [==============================================] - 1s 145ms/sample - loss: 0.6526 - accuracy: 1.0000Epoch 3/104/1 [==============================================] - 1s 149ms/sample - loss: 0.6186 - accuracy: 1.0000

问题是什么？

Answer 1

使用 mask 时会发生这种情况。我不知道原因，但是，当我在嵌入层中设置 mask_zero = True 时，我的训练也很慢。