python - 将 vgg19 层的输出显示为图像

Question

我正在读这篇论文:Neural Style Transfer 。在本文中，作者根据 vgg19 层的输出重建图像。我正在使用 Keras。 block1_conv1层的输出大小为(1, 400, 533, 64)。这里 1 是输入图像的数量，400 是行数，533 列数和 64 channel 数。当我尝试将其重建为图像时，出现错误，因为图像大小为 13644800，它不是 3 的倍数，因此我无法在三个 channel 中显示图像。我怎样才能重建这个图像？

我想从图层重建图像，如下所示: 下面是相同的代码:

from keras.preprocessing.image import load_img, img_to_array
from scipy.misc import imsave
import numpy as np
from keras.applications import vgg19
from keras import backend as K

CONTENT_IMAGE_FN = store image as input here

def preprocess_image(image_path):
    img = load_img(image_path, target_size=(img_nrows, img_ncols))
    img = img_to_array(img)
    img = np.expand_dims(img, axis=0)
    img = vgg19.preprocess_input(img)
    return img

width, height = load_img(CONTENT_IMAGE_FN).size
img_nrows = 400
img_ncols = int(width * img_nrows / height)
base_image = K.variable(preprocess_image(CONTENT_IMAGE_FN))

RESULT_DIR = "generated/"
RESULT_PREFIX = RESULT_DIR + "gen"
if not os.path.exists(RESULT_DIR):
  os.makedirs(RESULT_DIR)
result_prefix = RESULT_PREFIX

# this will contain our generated image
if K.image_data_format() == 'channels_first':
    combination_image = K.placeholder((1, 3, img_nrows, img_ncols))
else:
    combination_image = K.placeholder((1, img_nrows, img_ncols, 3))

x = preprocess_image(CONTENT_IMAGE_FN)

outputs_dict = dict([(layer.name, layer.output) for layer in model.layers])
feature_layers = ['block1_conv1', 'block2_conv1',
                  'block3_conv1', 'block4_conv1',
                  'block5_conv1']
outputs = []
for layer_name in feature_layers:
  outputs.append(outputs_dict[layer_name])
functor = K.function([combination_image], outputs )   # evaluation function

# Testing
test = x
layer_outs = functor([test])
print(layer_outs)

layer_outs[0].reshape(400, -1 , 3) //getting error here

我收到以下错误:

ValueError: cannot reshape array of size 13644800 into shape (400,newaxis,3)

Answer 1

您写道:

"The size of output of block1_conv1 layer is (1, 400, 533, 64). Here 1 is number of images as input, 400 is number of rows, 533 number of columns and 64 number of channels" But this is not correct. The block1_conv1 output corresponds 1 channel dimension(channel first), 400 * 533 image dimension and 64 filters.

当您尝试将具有 1 个 channel 的图像输入的 VGG19 输出向量 reshape 为 (400 * 533 * 64 = 13644800) 时，会发生错误对应于 3 channel 输出的向量。

此外，您还必须传递3 channel 输入:

来自VGG19代码:

input_shape: optional shape tuple, only to be specified if include_top is False (otherwise the input shape has to be (224, 224, 3) (with channels_last data format) or (3, 224, 224) (with channels_first data format). It should have exactly 3 inputs channels, and width and height should be no smaller than 32. E.g. (200, 200, 3) would be one valid value.

因此您的输入图像必须是 3 个 channel 。如果您甚至想将 1 channel (灰度)图像提供给 VGG19，则您应该执行以下操作(如果 channel 优先):

X = np.repeat(X, 3 , axis=0) 

或

X = np.repeat(X, 3 , axis=2) 

如果 channel 最后 没有批量维度或

X = np.repeat(X, 3 , axis=3) 

具有批处理维度。

如果您提供有关图像输入矩阵的实际尺寸及其类型(灰度、RGB)的更多信息，我可以在需要时为您提供更多帮助。