python - TensorFlow 数据集 API from_generator 序列结束错误

Question

这是重复我在尝试使用生成器动态生成/提供训练数据时遇到的问题的玩具代码。

def makeRand():
    yield np.random.rand(1)

dataset = tf.data.Dataset.from_generator(makeRand, (tf.float32))

iterator = tf.contrib.data.Iterator.from_structure(tf.float32, tf.TensorShape([]))

next_x = iterator.get_next()

init_op = iterator.make_initializer(dataset)

with tf.Session() as sess:
    sess.run(init_op)
    a = sess.run(next_x)
    print(a)
    a = sess.run(next_x)
    print(a)

跟踪看起来像:

Traceback (most recent call last):
  File “test_iterator_gen.py", line 31, in <module>
    a = sess.run(next_x)
 tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
     [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[]], output_types=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](Iterator)]]

Caused by op 'IteratorGetNext', defined at:
  File "test_iterator_gen.py", line 23, in <module>
    next_x = iterator.get_next()
OutOfRangeError (see above for traceback): End of sequence
     [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[]], output_types=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](Iterator)]]

Answer 1

这是由生成器的错误实例化引起的。

该错误是由于 makeRand() 耗尽了要产生的元素而引起的。通过将其更改为以下内容可以解决此问题:

def makeRand():
   while True:
      yield np.random.rand(1)