android - 在运行 Flask 服务的 GCP App Engine 服务上接收图像时出现问题

Question

我正在开发一个应用程序，让用户可以拍照并将其发送到 Keras 模型进行预测。此模型已部署在 Google App Engine 服务中，其中包含一个 Python 脚本，该脚本使用 Flask 通过 POST 请求接收图像并调用模型进行预测。这是 Python 代码:

import numpy as np
import flask
import io
import logging
import tensorflow as tf
from keras.preprocessing.image import img_to_array
from keras.applications import imagenet_utils
from keras.models import load_model
from PIL import Image


# initialize our Flask application and the Keras model
app = flask.Flask(__name__)
app.config['PROPAGATE_EXCEPTIONS'] = True
model = None

def recortar(image):
    # Function that centers and crop image. Please, asume that it works properly. Return is a numpy array.
    return image

@app.route("/predict", methods=["POST"])
def predict():
    model = load_model('modelo_1.h5')
    graph = tf.get_default_graph()
    data = {"success": False}

    if flask.request.method == "POST":
        if flask.request.files.get("image"):
            # read the image in PIL format
            image = flask.request.files["image"].read()
            image = Image.open(io.BytesIO(image))

            image = recortar(image)
            app.logger.info('Tamaño: '+str(image.size))
            image = img_to_array(image)
            image = np.expand_dims(image, axis=0)

            with graph.as_default():
                preds = model.predict(image)

            data['predictions'] = str(np.squeeze(preds).tolist())

            data["success"] = True  
            return flask.jsonify(data)
        else:
            return "No se ha obtenido la imagen"
    else:
        return "El HTTP request no era POST"

# if this is the main thread of execution first load the model and
# then start the server
if __name__ == "__main__":
    print(("* Loading Keras model and Flask starting server..."
        "please wait until server has fully started"))
    app.debug = True
    app.run()

通过 curl 发送图像非常有效:正如预期的那样，我从服务器获得了包含预测的 JSON 响应。这是 CURL 命令和服务器响应:

>> curl -X POST -F image=@nevus.jpg 'https://example.com/predict'
{"predictions":"[0.7404708862304688, 0.25952914357185364]","success":true}

然后我尝试重复相同的过程，但通过 Android 应用程序，但我收到 500 错误作为响应。在检查 Stackdriver 错误报告的日志时，我看到以下堆栈跟踪:AttributeError:

'NoneType' object has no attribute 'size'
at predict (/home/vmagent/app/main.py:73)
at dispatch_request (/env/lib/python3.6/site-packages/flask/app.py:1799)
at full_dispatch_request (/env/lib/python3.6/site-packages/flask/app.py:1813)
at reraise (/env/lib/python3.6/site-packages/flask/_compat.py:35)
at handle_user_exception (/env/lib/python3.6/site-packages/flask/app.py:1718)
at full_dispatch_request (/env/lib/python3.6/site-packages/flask/app.py:1815)
at wsgi_app (/env/lib/python3.6/site-packages/flask/app.py:2292)
at reraise (/env/lib/python3.6/site-packages/flask/_compat.py:35)
at handle_exception (/env/lib/python3.6/site-packages/flask/app.py:1741)
at wsgi_app (/env/lib/python3.6/site-packages/flask/app.py:2295)
at __call__ (/env/lib/python3.6/site-packages/flask/app.py:2309)
at handle_request (/env/lib/python3.6/site-packages/gunicorn/workers/sync.py:176)
at handle (/env/lib/python3.6/site-packages/gunicorn/workers/sync.py:135)

这个错误是指图像对象，所以我假设，由于代码之前工作正常，错误一定是我通过 HTTP 请求发送图像的方式。回想一下，图像是在用户单击按钮时拍摄的，因为此按钮发送了拍摄照片的 Intent 。拍摄照片后，用户可以单击发送按钮，我在下面发布了其代码。请注意，orientedBitmap 对应于以位图格式拍摄的照片。

btn_enviarfoto.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                Log.d(TAG, "Botón \"enviar\" pulsado. Codificando imagen.");
                ByteArrayOutputStream stream = new ByteArrayOutputStream();
                orientedBitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
                byte[] byteArray = stream.toByteArray();
                orientedBitmap.recycle();
                uploadToServer(byteArray);
            }
        });

uploadToServer 只是调用 AsynchTask 类的执行方法，如下所示:

private void uploadToServer(byte[] data) {
        Bitmap bitmapOrg = BitmapFactory.decodeByteArray(data, 0, data.length);
        Log.d(TAG, "Imagen codificada. Enviando al servidor.");
        ObtenerPrediccionTask task = new ObtenerPrediccionTask();
        task.execute(bitmapOrg);    
    }

最后也是最重要的，这是 ObtenerPrediccionTask 类的代码:

public class ObtenerPrediccionTask extends AsyncTask<Bitmap, Void, String> {

    @Override
    protected String doInBackground(Bitmap... imagen) {
        ByteArrayOutputStream bao = new ByteArrayOutputStream();
        HttpURLConnection connection = null;
        DataOutputStream outputStream = null;
        String probabilidad_melanoma = "";
        JsonReader jsonReader = null;


        try {
            for (int i = 0; i < imagen.length; i++) {
                Bitmap imagen2 = imagen[i];
                imagen2.compress(Bitmap.CompressFormat.JPEG, 90, bao);
                byte[] ba = bao.toByteArray();
                InputStream fileInputStream = new ByteArrayInputStream(ba);


                URL url = new URL("https://example.com/predict"); // not the real URL

                String lineEnd = "\r\n";
                String twoHyphens = "--";
                String boundary = "xxxxxxxx";
                String str = twoHyphens + boundary + lineEnd;


                connection = (HttpURLConnection) url.openConnection();

                // Allow Inputs & Outputs
                connection.setDoInput(true);
                connection.setDoOutput(true);
                connection.setUseCaches(false);

                // Enable POST method
                connection.setRequestMethod("POST");

                connection.setRequestProperty("Connection", "Keep-Alive");
                connection.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);

                outputStream = new DataOutputStream(connection.getOutputStream());
                outputStream.writeBytes(twoHyphens + boundary + lineEnd);
                outputStream.writeBytes("Content-Disposition: form-data; name=\"" +
                        "image" + "\";filename=\"" +
                        "foto.jpg" + "\"" + lineEnd);
                outputStream.writeBytes(lineEnd);

                int bytesAvailable = fileInputStream.available();
                int bufferSize = Math.min(bytesAvailable, 1024);
                byte[] buffer = new byte[bufferSize];

                // Read file
                int bytesRead = fileInputStream.read(buffer, 0, bufferSize);

                while (bytesRead > 0) {
                    outputStream.write(buffer, 0, bufferSize);
                    bytesAvailable = fileInputStream.available();
                    bufferSize = Math.min(bytesAvailable, 1024);
                    bytesRead = fileInputStream.read(buffer, 0, bufferSize);
                }

                outputStream.writeBytes(lineEnd);
                outputStream.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);

                // Responses from the server (code and message)
                int responseCode = connection.getResponseCode();
                connection.getResponseMessage();

                fileInputStream.close();
                outputStream.flush();
                outputStream.close();

                if (responseCode == HttpURLConnection.HTTP_OK) {
                    InputStream responseStream = new
                            BufferedInputStream(connection.getInputStream());

                    BufferedReader responseStreamReader =
                            new BufferedReader(new InputStreamReader(responseStream));

                    String line = "";
                    StringBuilder stringBuilder = new StringBuilder();

                    while ((line = responseStreamReader.readLine()) != null) {
                        stringBuilder.append(line).append("\n");
                    }
                    responseStreamReader.close();

                    String response = stringBuilder.toString();
                    Log.d(TAG, "Imagen recibida por el servidor y pasada al modelo. Esta es la respuesta: " + response);

                    jsonReader = new JsonReader(new StringReader(response));
                    probabilidad_melanoma = readJson(jsonReader);
                } else {
                    Log.d(TAG, Integer.toString(responseCode));
                }
            }
            return probabilidad_melanoma;
        } catch (MalformedURLException malformedURLException) {
            Log.e(TAG, malformedURLException.toString());
            return null;
        } catch (IOException io) {
            Log.e(TAG, io.toString());
            return null;
        } finally {
            if (connection != null) {
                connection.disconnect();
            }
        }
    }

    protected void onPostExecute(String probabilidad_melanoma) {
        if (probabilidad_melanoma != null) {
            Log.d(TAG, "Probabilidad melanoma: " + probabilidad_melanoma);
        } else {
            Log.w(TAG, "La respuesta ha sido nula");
        }
    }
}

readJson 函数也正常工作，所以不要被它打扰。

这最后一段代码是在 SO 中广泛搜索正确发送图像的方法的结果，但由于还没有任何效果，我已经没有想法了。我的代码有什么问题？

Answer 1

崩溃回溯表明在这一行 image 是 None:

        app.logger.info('Tamaño: '+str(image.size))

这意味着 recortar() 返回 None，尽管您有评论:

# Function that centers and crop image. Please, asume that it works properly. Return is a numpy array.

所以您的错误一定是我通过 HTTP 请求发送图像的方式 假设可能是错误的。在花时间之前，我首先要添加检查以确保 recortar() 正常工作。