c++ - 树莓派上的 tensorflow lite 量化 ssd 对象检测

Question

在使用 C++ 的带有 tensorflow lite 的树莓派上，对象检测无法正常工作。我的代码编译并运行，但输出似乎从未得到正确填充。我是否会遗漏任何依赖项或错误地访问结果？

我遵循了以下教程: https://medium.com/tensorflow/training-and-serving-a-realtime-mobile-object-detector-in-30-minutes-with-cloud-tpus-b78971cf1193

并从以下位置获得 detect.tflite 模型: https://storage.googleapis.com/download.tensorflow.org/models/tflite/pets_ssd_mobilenet_v1_0.75_quant_2018_06_29.zip

我已经为树莓派编译了tensorflow lite和opencv，并修改了minimal.cc以读取图像并进行推理，如下所示:

/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include <cstdio>
#include "tensorflow/contrib/lite/interpreter.h"
#include "tensorflow/contrib/lite/kernels/register.h"
#include "tensorflow/contrib/lite/model.h"
#include "tensorflow/contrib/lite/optional_debug_tools.h"

#include <iostream>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"

// This is an example that is minimal to read a model
// from disk and perform inference.
//
// Usage: detect <tflite model> <image filepath>

using namespace tflite;

#define TFLITE_MINIMAL_CHECK(x)                              \
  if (!(x)) {                                                \
    fprintf(stderr, "Error at %s:%d\n", __FILE__, __LINE__); \
    exit(1);                                                 \
  }

int main(int argc, char* argv[]) {
  if(argc != 3) {
    fprintf(stderr, "minimal <tflite model filepath> <image filepath>\n");
    return 1;
  } else {
      fprintf(stdout, "Reading model from %s\n", argv[1]);
      fprintf(stdout, "Reading image from %s\n", argv[2]);
  }
  const char* model_filename = argv[1];
  const char* image_filename = argv[2];

  // Load model
  std::unique_ptr<tflite::FlatBufferModel> model =
      tflite::FlatBufferModel::BuildFromFile(model_filename);
  TFLITE_MINIMAL_CHECK(model != nullptr);

  // Build the interpreter
  tflite::ops::builtin::BuiltinOpResolver resolver;
  InterpreterBuilder builder(*model.get(), resolver);
  std::unique_ptr<Interpreter> interpreter;
  builder(&interpreter);
  TFLITE_MINIMAL_CHECK(interpreter != nullptr);

  // Allocate tensor buffers.
  TFLITE_MINIMAL_CHECK(interpreter->AllocateTensors() == kTfLiteOk);
  printf("=== Pre-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // Fill input buffers
  // TODO(user): Insert code to fill input tensors
  cv::Mat img = cv::imread(image_filename);
  //std::cout << "before: " << interpreter->typed_input_tensor<uchar>(0) << std::endl;
  //std::cout << "image: " << img.data << std::endl;
  memcpy(interpreter->typed_input_tensor<uchar>(0), img.data, img.total() * img.elemSize());
  //std::cout << "after: " << interpreter->typed_input_tensor<uchar>(0) << std::endl;

  // Run inference
  TFLITE_MINIMAL_CHECK(interpreter->Invoke() == kTfLiteOk);
  printf("\n\n=== Post-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // Read output buffers
  // TODO(user): Insert getting data out code.
  cv::Mat results0(10, 4, CV_8U);
  cv::Mat results1(1, 10, CV_8U);
  cv::Mat results2(1, 10, CV_8U);
  cv::Mat results3(1, 1, CV_8U);

  results0.data = interpreter->typed_output_tensor<uchar>(0);
  results1.data = interpreter->typed_output_tensor<uchar>(1);
  results2.data = interpreter->typed_output_tensor<uchar>(2);
  results3.data = interpreter->typed_output_tensor<uchar>(3);

  std::cout << "results 0: " << results0 << std::endl;
  std::cout << "results 1: " << results1 << std::endl;
  std::cout << "results 2: " << results2 << std::endl;
  std::cout << "results 3: " << results3 << std::endl;

  return 0;
}

我看到的结果是:

results 0: []               
results 1: []               
results 2: []               
results 3: []  

Answer 1

使用cout 打印cv::Mat 时似乎有些奇怪:您明确定义了指标的大小，但打印的结果为空。

您可以尝试不使用 cv:Mat 直接打印出这些值吗？您可以像这样进行调试:

const auto* output = interpreter->typed_output_tensor<unsigned char>(0);
for (int i = 0; i < 40; ++i) {
  printf("%d, ", static_cast<int>(output[i]);
}