python - FailedPreconditionError(请参阅上面的回溯): GetNext() failed because the iterator has not been initialized

Question

我在进行预测时为输入数据构建了一个数据集管道。但是，当我尝试代码时，发生了错误

FailedPreconditionError (see above for traceback): GetNext() failed because the iterator has not been initialized. Ensure that you have run the initializer operation for this iterator before getting the next element. [[Node: IteratorGetNext_259 = IteratorGetNextoutput_shapes=[[?,227,227,6]], output_types=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

遍历数据集的迭代器定义如下:

for k in range(num_init_ops):
with tf.device('/cpu:0'):


    pre_data.append(PreDataGenerator(pre_file,
                                mode='predicting',
                                batch_size=batch_size,
                                num_classes=num_classes,
                                shuffle=False,
                                iterator_size=iterator_size,
                                kth_init_op=k))

    # create an reinitializable iterator given the dataset structure
    iterator = Iterator.from_structure(pre_data[k].data.output_types,
                                       pre_data[k].data.output_shapes)
    next_batch = iterator.get_next()

# Ops for initializing the two different iterators
predicting_init_op.append(iterator.make_initializer(pre_data[k].data))

编写for循环是因为我希望创建多个数据集初始化操作来将数据拆分到不同的迭代器中，以防止累积内存调用导致OOM错误(我想知道这样做是否可行) .

我确定迭代器已初始化(调试时它输出正确的结构)。这是我的 Tensorflow session 代码:

with tf.Session(config=config) as sess:
sess.run(tf.global_variables_initializer())
# sess.run(tf.local_variables_initializer())
saver.restore(sess, './checkpoints_grade1/model_epoch46.ckpt')  # todo:

print("{} Start predicting...".format(datetime.now()))

for j in range(num_init_ops+1):#todo:

    print('{} Initializing {} iterator'.format(datetime.now(),j))

    # Initialize iterator with the predicting dataset
    sess.run(predicting_init_op[j])

    for i in range(iterator_size):

        # get next batch of data
        img_batch = sess.run(next_batch)#todo:?

        # And run the predicting op
        img_batch = tf.reshape(img_batch, (1, 227, 227, 6))
        pred = sess.run(softmax, feed_dict={x: sess.run(img_batch)})
        predicted_label = pred.argmax(axis=1)
        predictions.append(predicted_label[0])
        output_file.write(str(i) + ' , ' + str(predicted_label[0]) + '\n')

Answer 1

您需要初始化迭代器:

sess.run(iterator.initializer)

并进行这样的训练:

next_batch = iterator.get_next()
sess.run(iterator.initializer)

for epoch in range(n_epochs):
    while True:
        try:
            batch = sess.run(next_batch)
            # feed data, train
            # ...
        except tf.errors.OutOfRangeError:
            sess.run(iterator.initializer)
            break

或者，在定义 tf.data.Dataset.from_tensor_slices 实例时，您可以指定要训练的轮数:

data = tf.data.Dataset.from_tensor_slices({
            'x':train_data,
            'y':train_labels
            }).repeat(n_epochs).batch(batch_size)
iterator = data.make_initializable_iterator()

有了这个，您不需要 for epoch in range(n_epochs) 循环:

next_batch = iterator.get_next()
sess.run(iterator.initializer)

while True:
    try:
        batch = sess.run(next_batch)
        # feed data, train
        # ...
    except tf.errors.OutOfRangeError:
        break