tensorflow - 使用rejection_resample() 和Dataset Api

Question

我很难尝试使用rejection_resample() 和Dataset API 进行一些平衡批处理。我使用图像和标签(整数)作为输入，你可以在代码中一目了然，但 拒绝重新采样()似乎没有按预期工作。

注意:我使用的是 Tensorflow v1.3

在这里我定义了数据集、数据集的分布和我想要的分布。

target_dist = [0.1, 0.0, 0.0, 0.0, 0.9]
initial_dist = [0.1061, 0.3213, 0.4238, 0.1203, 0.0282]

training_filenames = training_records
training_dataset = tf.contrib.data.TFRecordDataset(training_filenames)
training_dataset = training_dataset.map(tf_record_parser)  # Parse the record into tensors.
training_dataset = training_dataset.repeat()  # number of epochs
training_dataset = training_dataset.shuffle(buffer_size=1000)

training_dataset = tf.contrib.data.rejection_resample(training_dataset,
                                                      class_func=lambda _, c: c,
                                                      target_dist=target_dist,
                                                      initial_dist=initial_dist)

# Return to the same Dataset shape as was the original input
training_dataset = training_dataset.map(lambda _, data: (data))

training_dataset = training_dataset.batch(64)

handle = tf.placeholder(tf.string, shape=[])
iterator = tf.contrib.data.Iterator.from_string_handle(
    handle, training_dataset.output_types, training_dataset.output_shapes)
batch_images, batch_labels = iterator.get_next()
training_iterator = training_dataset.make_initializable_iterator()

当我运行这个东西时，我应该只从第 0 类和第 4 类中获取样本，但是我从所有类中获得了结果，就好像它不起作用一样。

with tf.Session() as sess:
    training_handle = sess.run(training_iterator.string_handle())
    sess.run(training_iterator.initializer)
    batch_faces_np, batch_label_np = sess.run([batch_images, batch_labels],feed_dict={handle: training_handle})

    ctr = Counter(batch_label_np)

Counter({2: 31, 3: 22, 4: 6, 1: 5})

我用一个基于这篇文章的例子进行了测试: Dataset API, Iterators and tf.contrib.data.rejection_resample来自 tensorflow repo 的原始测试代码，它可以工作。

initial_known = True
classes = np.random.randint(5, size=(20000,))  # Uniformly sampled
target_dist = [0.5, 0.0, 0.0, 0.0, 0.4]
initial_dist = [0.2] * 5 if initial_known else None

iterator = dataset_ops.Iterator.from_dataset(
    dataset_ops.rejection_resample(
        (dataset_ops.Dataset.from_tensor_slices(classes)
         .shuffle(200, seed=21)
         .map(lambda c: (c, string_ops.as_string(c)))),
        target_dist=target_dist,
        initial_dist=initial_dist,
        class_func=lambda c, _: c,
        seed=27))
init_op = iterator.initializer
get_next = iterator.get_next()
variable_init_op = variables.global_variables_initializer()

with tf.Session() as sess:
    sess.run(variable_init_op)
    sess.run(init_op)
    returned = []
    while True:
        returned.append(sess.run(get_next))

Counter({(0, (0, b'0')): 3873, (4, (4, b'4')): 3286})

你们能帮我解决这个问题吗？谢谢。

Answer 1

尝试使用种子值进行随机播放。
它对我来说具有种子值(value)。