keras - 使用 Keras 和 Google Cloud ML 进行 Base64 图像

Question

我正在使用 Keras 预测图像类别。它适用于 Google Cloud ML (GCML)，但为了提高效率，需要将其更改为传递 base64 字符串而不是 json 数组。 Related Documentation

我可以轻松运行python代码将base64字符串解码为json数组，但是当使用GCML时，我没有机会运行预处理步骤(除非可能在Keras中使用Lambda层，但我不认为这是正确的做法)。

Another answer建议添加类型为 tf.string 的 tf.placeholder，这是有道理的，但如何将其合并到 Keras 模型中？

这里是训练模型和保存 GCML 导出模型的完整代码...

import os
import numpy as np
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.preprocessing import image
from tensorflow.python.platform import gfile

IMAGE_HEIGHT = 138
IMAGE_WIDTH = 106
NUM_CLASSES = 329

def preprocess(filename):
    # decode the image file starting from the filename
    # end up with pixel values that are in the -1, 1 range
    image_contents = tf.read_file(filename)
    image = tf.image.decode_png(image_contents, channels=1)
    image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
    image = tf.expand_dims(image, 0) # resize_bilinear needs batches
    image = tf.image.resize_bilinear(image, [IMAGE_HEIGHT, IMAGE_WIDTH], align_corners=False)
    image = tf.subtract(image, 0.5)
    image = tf.multiply(image, 2.0) # -1 to 1
    image = tf.squeeze(image,[0])
    return image



filelist = gfile.ListDirectory("images")
sess = tf.Session()
with sess.as_default():
    x = np.array([np.array(     preprocess(os.path.join("images", filename)).eval()      ) for filename in filelist])

input_shape = (IMAGE_HEIGHT, IMAGE_WIDTH, 1)   # 1, because preprocessing made grayscale

# in our case the labels come from part of the filename
y = np.array([int(filename[filename.index('_')+1:-4]) for filename in filelist])
# convert class labels to numbers
y = keras.utils.to_categorical(y, NUM_CLASSES)

########## TODO: something here? ##########
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
# scores = build_model(decoded)


model = Sequential()

# model.add(decoded)

model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.Adadelta(),
            metrics=['accuracy'])

model.fit(
    x,
    y,
    batch_size=64,
    epochs=20,
    verbose=1,
    validation_split=0.2,
    shuffle=False
    )

predict_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={'input_bytes':tf.saved_model.utils.build_tensor_info(model.input)},
    ########## TODO: something here? ##########
    # inputs={'input': image },    # input name must have "_bytes" suffix to use base64.
    outputs={'formId': tf.saved_model.utils.build_tensor_info(model.output)},
    method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)

builder = tf.saved_model.builder.SavedModelBuilder("exported_model")

builder.add_meta_graph_and_variables(
    sess=K.get_session(),
    tags=[tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: predict_signature
    },
    legacy_init_op=tf.group(tf.tables_initializer(), name='legacy_init_op')
)

builder.save()

这与我的 previous question 有关.

更新:

问题的核心是如何将调用decode的占位符合并到Keras模型中。换句话说，在创建将 base64 字符串解码为张量的占位符后，如何将其合并到 Keras 运行的内容中？我认为它需要是一个层。

image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
model = Sequential()

# Something like this, but this fails because it is a tensor, not a Keras layer.  Possibly this is where a Lambda layer comes in?
model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
...

更新2:

尝试使用 lambda 层来完成此操作...

import keras
from keras.models import Sequential
from keras.layers import Lambda
from keras import backend as K
import tensorflow as tf

image = K.placeholder(shape=(), dtype=tf.string)
model = Sequential()
model.add(Lambda(lambda image: tf.image.decode_jpeg(image, channels=3), input_shape=() ))

给出错误:TypeError:“DecodeJpeg”Op 的输入“内容”的类型为 float32，与预期的字符串类型不匹配。

Answer 1

首先我使用 tf.keras 但这应该不是一个大问题。下面是如何读取 Base64 解码的 jpeg 的示例:

def preprocess_and_decode(img_str, new_shape=[299,299]):
    img = tf.io.decode_base64(img_str)
    img = tf.image.decode_jpeg(img, channels=3)
    img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.BILINEAR, align_corners=False)
    # if you need to squeeze your input range to [0,1] or [-1,1] do it here
    return img
InputLayer = Input(shape = (1,),dtype="string")
OutputLayer = Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
base64_model = tf.keras.Model(InputLayer,OutputLayer)   

上面的代码创建了一个模型，该模型采用任意大小的 jpeg，将其大小调整为 299x299 并返回 299x299x3 张量。该模型可以直接导出到saved_model并用于Cloud ML Engine服务。它有点愚蠢，因为它唯一做的就是将 base64 转换为张量。

如果您需要将此模型的输出重定向到现有经过训练和编译的模型(例如 inception_v3)的输入，您必须执行以下操作:

base64_input = base64_model.input
final_output = inception_v3(base64_model.output)
new_model = tf.keras.Model(base64_input,final_output)

这个new_model可以保存。它采用 base64 jpeg 并返回由 inception_v3 部分标识的类。