python - Tensorflow:tf.image.resize 仍然没有对齐角吗？

Question

我正在读这篇文章blog post在 Hackernoon 中，关于 Tensorflow 的 tf.image.resize_area() 函数如何不是反射等变的。因此，如果我要在某些数据增强步骤中调整图像大小，这可能会真正搞乱模型训练。

作者接着说，用户不应使用任何 tf.image.resize 函数，因为可能存在不可预测的行为。这篇文章是 2018 年 1 月写的，所以不是很久以前的事。我实际上查看了文章的评论部分，没有人提到问题已解决。

我只是想知道这些问题是否仍然存在以及解决方法是什么？ tensorflow 后续版本中的任何更改。比如我可以使用 tf.keras 增强函数来避免这些问题吗？

Answer 1

在我最初阅读您引用的 Hackernoon 文章后，我还发现了 this article它很好地总结了 OpenCV、TF 1.X 和其他一些深度学习框架中双线性插值的不同实现。

我在 TF 2.0 文档中找不到任何相关内容，因此我重现了该文章中给出的示例来测试 2.0 中的双线性插值。当我使用 TensorFlow 2.0 运行以下代码时，测试通过，因此看起来迁移到 TF2.0 将为您提供与 OpenCV 实现相匹配的双线性插值实现(因此解决了 Hackernoon 文章中提出的问题):

def test_tf2_resample_upsample_matches_opencv_methodology():
    """
    According to the article below, the Tensorflow 1.x implementation of bilinear interpolation for resizing images did
    not reproduce the pixel-area-based approach adopted by OpenCV. The `align_corners` option was set to False by
    default due to some questionable legacy reasons but users were advised to set it to True in order to get a
    'reasonable' output: https://jricheimer.github.io/tensorflow/2019/02/11/resize-confusion/
    This appears to have been fixed in TF 2.0 and this test confirms that we get the results one would expect from a
    pixel-area-based technique.

    We start with an input array whose values are equivalent to their column indices:
    input_arr = np.array([
        [[0], [1], [2], [3], [4], [5]],
        [[0], [1], [2], [3], [4], [5]],
    ])

    And then resize this (holding the rows dimension constant in size, but increasing the column dimnesion to 12) to
    reproduce the OpenCV example from the article. We expect this to produce the following output:
    expected_output = np.array([
        [[0], [0.25], [0.75], [1.25], [1.75], [2.25], [2.75], [3.25], [3.75], [4.25], [4.75], [5]],
        [[0], [0.25], [0.75], [1.25], [1.75], [2.25], [2.75], [3.25], [3.75], [4.25], [4.75], [5]],
    ])

    """
    input_tensor = tf.convert_to_tensor(
        np.array([
            [[0], [1], [2], [3], [4], [5]],
            [[0], [1], [2], [3], [4], [5]],
        ]),
        dtype=tf.float32,
    )
    output_arr = tf.image.resize(
        images=input_tensor,
        size=(2,12),
        method=tf.image.ResizeMethod.BILINEAR).numpy()
    expected_output = np.array([
        [[0], [0.25], [0.75], [1.25], [1.75], [2.25], [2.75], [3.25], [3.75], [4.25], [4.75], [5]],
        [[0], [0.25], [0.75], [1.25], [1.75], [2.25], [2.75], [3.25], [3.75], [4.25], [4.75], [5]],
    ])
    np.testing.assert_almost_equal(output_arr, expected_output, decimal=2)