python - Tensorflow Dataset.from_generator 在tensorflow 2.0中是否已弃用？它抛出 tf.py_func 弃用错误

Question

当我从生成器创建 tf 数据集并尝试运行 tf2.0 代码时，它会向我发出贬值消息警告。

代码:

import tensorflow as tf

from tensorflow.keras.layers import Dense, Flatten, Conv2D
from tensorflow.keras import Model


def my_function():
    import numpy as np
    for i in range(1000):
        yield np.random.random(size=(28, 28, 1)), [1.0]


train_ds = tf.data.Dataset.from_generator(my_function, output_types=(tf.float32, tf.float32)).batch(32)


class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = Conv2D(32, 3, activation='relu')
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10, activation='softmax')

    def call(self, x):
        x = self.conv1(x)
        x = self.flatten(x)
        x = self.d1(x)
        return self.d2(x)

    # def __call__(self, *args, **kwargs):
    #     return super().__call(*args,**kwargs)


model = MyModel()

loss_object = tf.keras.losses.SparseCategoricalCrossentropy()

optimizer = tf.keras.optimizers.Adam()

train_loss = tf.keras.metrics.Mean(name='train_loss')
train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')


@tf.function
def train_step(images, labels):
    with tf.GradientTape() as tape:
        predictions = model(images)
        loss = loss_object(labels, predictions)
    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))

    train_loss(loss)
    train_accuracy(labels, predictions)


EPOCHS = 5

for epoch in range(EPOCHS):
    for images, labels in train_ds:
        train_step(images, labels)
    template = 'Epoch {}, Loss: {}, Accuracy: {}'
    print(template.format(epoch + 1,
                          train_loss.result(),
                          train_accuracy.result() * 100))

警告消息:

........
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2. ........

我想使用数据集 API(带有 prefetch )从流输入将数据提供给模型。尽管在当前的 alpha 版本中仍然可以实现，但以后会被删除吗？

tensorflow 会将生成器数据集中使用的 tf.py_func 替换为新的内容，还是删除整个 dataset_from 生成器 API？

Answer 1

不，tf.data.Dataset.from_generator TensorFlow 2.0 中不会弃用。您看到的是一条警告消息，它用于通知用户 future 的更改。如果您需要直接使用 py_func，最直接的方法是使用 tf.compat.v1.py_func。 TF2.0 有自己的包装器，称为 tf.py_function。