python - 如何在 Keras 中将张量的 2D 子集分配给另一个 2D 张量？

Question

如果我有两个 3-D 张量 img 和 gen。如何将 img 的 2D 子集分配给 gen 的 2D 子集？以下内容不起作用，因为 tensorflow 不允许直接分配张量。

img[96:160 , 144:240 , :] = gen[96:160 , 144:240 , :]

编辑:

这是周围的代码。所以我使用自定义的 keras 层。该层必须接收输入图像img和生成图像x。它必须用 x 替换部分 img，并且必须返回修改后的 img。

def patcher(tensors):
    img = tensor[1]
    gen = tensor[0]
    #This is where the slicing must happen
    img[96:160 , 144:240 , :] = gen[96:160 , 144:240 , :]
    return [img]

img = Input( .. )
x = Conv( .. )(img)
out = Lambda(patcher,lambda a : [a[1]] )([x , img])
model = Model(img, out)

Answer 1

目前，您无法以直接的方式替换张量的切片。我其实opened an issue about it因为这是人们一直要求的东西。使用当前的 API，您必须找出构建所需张量的最佳方法。在本例中，假设 img 和 gen 具有相同的形状，这是您可以实现的一种方法:

import tensorflow as tf
import numpy as np

# Input
img = tf.placeholder(tf.float32, [None, None, None])
gen = tf.placeholder(tf.float32, [None, None, None])
row_start = tf.placeholder(tf.int32, [])
row_end = tf.placeholder(tf.int32, [])
col_start = tf.placeholder(tf.int32, [])
col_end = tf.placeholder(tf.int32, [])
# Masks rows and columns to be replaced
shape = tf.shape(img)
rows = shape[0]
cols = shape[1]
channels = shape[2]
i = tf.range(rows)
row_mask = (row_start <= i) & (i < row_end)
j = tf.range(cols)
col_mask = (col_start <= j) & (j < col_end)
# Full mask of replaced elements
mask = row_mask[:, tf.newaxis] & col_mask
# Select elements from flattened arrays
img_flat = tf.reshape(img, [-1, channels])
gen_flat = tf.reshape(gen, [-1, channels])
mask_flat = tf.reshape(mask, [-1])
result_flat = tf.where(mask_flat, gen_flat, img_flat)
# Reshape back
result = tf.reshape(result_flat, shape)

这是一个小测试:

with tf.Session() as sess:
    # img is positive and gen is negative
    img_val = np.arange(60).reshape((4, 5, 3))
    gen_val = -img_val
    # Do img[2:4, 0:3, :] = gen[2:4, 0:3, :]
    result_val = sess.run(result, feed_dict={
        img: img_val,
        gen: gen_val,
        row_start: 2,
        row_end: 4,
        col_start: 0,
        col_end: 3,
    })
    # Print one channel only for clarity
    print(result_val[:, :, 0])

输出:

[[  0.   3.   6.   9.  12.]
 [ 15.  18.  21.  24.  27.]
 [-30. -33. -36.  39.  42.]
 [-45. -48. -51.  54.  57.]]

<小时/>

编辑:

这是您发布的代码的可能实现。我在这里使用基于乘法的稍微不同的方法，我认为当您有很多图像时，这种方法更好。

import tensorflow as tf

def replace_slices(img, gen, row_start, row_end, col_start, col_end):
    # Masks rows and columns to be replaced
    shape = tf.shape(img)
    rows = shape[1]
    cols = shape[2]
    i = tf.range(rows)
    row_mask = (row_start <= i) & (i < row_end)
    j = tf.range(cols)
    col_mask = (col_start <= j) & (j < col_end)
    # Full mask of replaced elements
    mask = row_mask[:, tf.newaxis] & col_mask
    # Add channel dimension to mask and cast
    mask = tf.cast(mask[:, :, tf.newaxis], img.dtype)
    # Compute result
    result = img * (1 - mask) + gen * mask
    return result

def patcher(tensors):
    img = tensor[1]
    gen = tensor[0]
    img = replace_slices(img, gen, 96, 160, 144, 240)
    return [img]

img = Input( .. )
x = Conv( .. )(img)
out = Lambda(patcher, ambda a: [a[1]])([x , img])
model = Model(img, out)