keras - 在 Keras 中循环使用 model.fit 是否合乎逻辑？

Question

为了不耗尽内存，在 Keras 中执行以下操作是否合乎逻辑？

for path in ['xaa', 'xab', 'xac', 'xad']:
    x_train, y_train = prepare_data(path)
    model.fit(x_train, y_train, batch_size=50, epochs=20, shuffle=True)

model.save('model')

Answer 1

是的，但更喜欢 model.train_on_batch如果每次迭代生成单个批次。这消除了 fit 带来的一些开销。 .

您也可以尝试创建一个生成器并使用 model.fit_generator() :

def dataGenerator(pathes, batch_size):

    while True: #generators for keras must be infinite
        for path in pathes:
            x_train, y_train = prepare_data(path)

            totalSamps = x_train.shape[0]
            batches = totalSamps // batch_size

            if totalSamps % batch_size > 0:
                batches+=1

            for batch in range(batches):
                section = slice(batch*batch_size,(batch+1)*batch_size)
                yield (x_train[section], y_train[section])

创建和使用:

gen = dataGenerator(['xaa', 'xab', 'xac', 'xad'], 50)
model.fit_generator(gen,
                    steps_per_epoch = expectedTotalNumberOfYieldsForOneEpoch
                    epochs = epochs)