python - 使用预训练模型的 CNN 输出层可视化

Question

我训练了我的模型并保存了它:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing.image import img_to_array

new_model=tf.keras.models.load_model('the_model.h5')
new_model.summary()

img = load_img('e.jpg',target_size=(227,227))
img=img_to_array(img)

img = np.expand_dims(img,axis=0)
img=img/255.
print(img.shape)
#prints out (1,227,227,3) the expected shapes
 

所以我的模型架构如下，我使用的是预训练的 resnet50

backbone = ResNet50(input_shape=(227,227,3),weights='imagenet', include_top=False)
    model = Sequential()
    model.add(backbone)
    model.add(GlobalAveragePooling2D())
    model.add(Dropout(0.5))
    model.add(Dense(64,activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(1,activation='sigmoid'))

我尝试可视化隐藏层的输出，但是使用 keras 或 keract 我无法获得输出

与喀拉斯:

layer_outputs=[]
for layer in new_model.layers:
    if layer.name=='resnet50':
        temp = [l.output for l in layer.layers]
        layer_outputs=temp
    else:
        layer_outputs.append(layer.output)
    

activation_model = Model(inputs=new_model.input,  outputs=layer_outputs)

最后一行引起的错误:

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []

我觉得我的模型输入与 layer_outputs 匹配，所以我真的不明白这个错误，事实上，当我检查时:

print(new_model.layers[0].input)
#prints out    :Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32)

print(layer_outputs[0])
#prints out :  Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32)

使用 keract 时:

a = keract.get_activations(new_model, img)  # with just one sample.
keract.display_activations(a, directory='f', save=True)

tensorflow.python.framework.errors_impl.InvalidArgumentError:  You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,227,227,3]

知道如何修复它，或者使用预训练模型从隐藏层获取输出的其他可行解决方案吗？

谢谢，

Answer 1

好的，我找到了解决问题的便捷方法。

事实上，我认为出现这个问题是因为我的顺序模型本身是由另一个模型 (resnet) 组成的。

因为我没有在预训练的 resnet 模型上添加很多层，所以我只是决定可视化 resnet 模型的特征图

img = load_img('e.jpg',target_size=(227,227))
img=img_to_array(img)

img = np.expand_dims(img,axis=0)
img=img/255.
print(img.shape)



loaded=tf.keras.models.load_model('age_gender_train.h5')


layer_outputs=[ layer.output for layer in loaded.layers[0].layers]
res = loaded.layers[0]

activation_model = Model(inputs=res.input, outputs=layer_outputs)
activations=activation_model.predict(img)
img = np.squeeze(img,axis=0)

然后您可以使用 activations 变量轻松显示特征图。

Note that since you have the outputs of the resnet model, it might be possible to get the feature maps from the layers on top by repeating the process. Using the output of resnet as input and removing the resnet model from the layer_outputs.(i did not try this, could not work)

希望对大家有帮助