python - 如何在tensorflow中将hsv张量更改为rgb张量？

Question

我需要一个可以将 HSV 张量(形状:[batch_size, image_width, image_height, num_channels]，channels 表示 h, s, v)转换为 RGB 的操作张量([batch_size，image_width，image_height，num_channels]， channel 表示r，g，b])。我知道存在tf.image.hsv_to_rgb，虽然它似乎是一个图像预处理函数并且无法获得渐变。为了创建对话操作，我们是否需要使用一些元操作从头开始编写代码(例如tf.multiply()、tf.add()等)

new_image_hsv = tf.random_normal(shape=[32,32,3]) 
new_image_rgb = tf.image.hsv_to_rgb(new_image_hsv, name='hsv2rgb') 
hsv2rgb = tf.get_default_graph().get_operation_by_name('hsv2rgb') 
print('hsv2rgb: ', get_gradient_function(hsv2rgb)) #hsv2rgb: None

Answer 1

目前HSV到RGB的转换函数没有注册渐变，您可以考虑opening an issue关于它。然而，看看implementation of the kernel ，完全可以使用具有定义梯度的基本 TensorFlow 操作来复制计算:

import tensorflow as tf

def my_hsv_to_rgb(tensor):
    h = tensor[..., 0]
    s = tensor[..., 1]
    v = tensor[..., 2]
    c = s * v;
    m = v - c;
    dh = h * 6
    h_category = tf.cast(dh, tf.int32)
    fmodu = tf.mod(dh, 2)
    x = c * (1 - tf.abs(fmodu - 1))
    component_shape = tf.shape(tensor)[:-1]
    dtype = tensor.dtype
    rr = tf.zeros(component_shape, dtype=dtype)
    gg = tf.zeros(component_shape, dtype=dtype)
    bb = tf.zeros(component_shape, dtype=dtype)
    h0 = tf.equal(h_category, 0)
    rr = tf.where(h0, c, rr)
    gg = tf.where(h0, x, gg)
    h1 = tf.equal(h_category, 1)
    rr = tf.where(h1, x, rr)
    gg = tf.where(h1, c, gg)
    h2 = tf.equal(h_category, 2)
    gg = tf.where(h2, c, gg)
    bb = tf.where(h2, x, bb)
    h3 = tf.equal(h_category, 3)
    gg = tf.where(h3, x, gg)
    bb = tf.where(h3, c, bb)
    h4 = tf.equal(h_category, 4)
    rr = tf.where(h4, x, rr)
    bb = tf.where(h4, c, bb)
    h5 = tf.equal(h_category, 5)
    rr = tf.where(h5, c, rr)
    bb = tf.where(h5, x, bb)
    r = rr + m
    g = gg + m
    b = bb + m
    return tf.stack([r, g, b], axis=-1)

测试它:

import tensorflow as tf
import numpy as np

img = tf.placeholder(tf.float32, (None, None, 3))
# Compute builtin conversion to check that our conversion is correct
tf_conversion = tf.image.hsv_to_rgb(img)
my_conversion = my_hsv_to_rgb(img)
# Difference between the builtin conversion and ours
error = tf.losses.mean_squared_error(tf_conversion, my_conversion)
# Take gradients of the conversion
my_conversion_grad = tf.gradients(my_conversion, img)[0]
# Test it
with tf.Session() as sess:
    np.random.seed(100)
    random_img = np.random.rand(10, 10, 3)
    error_val, grad_val = sess.run([error, my_conversion_grad],
                                    feed_dict={img: random_img})
    print(error_val)
    print(grad_val)

输出:

Error:
1.914486e-16
Gradient:
[[[-7.0903623e-01 -5.3507453e-01  2.6491349e+00]
  [-3.4420034e-03 -1.2991568e-01  2.9949572e+00]
  [ 6.7739707e-01 -2.7006465e-01  1.3685231e+00]
  [-1.1187987e+00 -3.0346024e-01  1.7070839e+00]
  [-1.4286079e-01 -2.4429685e-01  2.8794882e+00]
  [-8.3736974e-01 -3.2182935e-01  1.4807379e+00]
  [ 7.0991272e-01 -4.7601932e-01  2.6977921e+00]
  [-1.6489303e+00 -5.5128366e-01  1.6589090e+00]
  [-1.2725148e-02 -5.9869420e-03  2.6076081e+00]
  [-7.2826855e-02 -2.3104541e-02  1.7949229e+00]]
#...

但是，请注意，HSV 到 RGB 的转换不是连续函数 ( see here )，这可能是它没有定义梯度的原因。这意味着，由于转换定义中的这种“跳跃”，梯度可能并不总是指示正确的优化方向。