opencv - 如何使用 Tensorflow C++ API 增加批量大小？

Question

我取了https://gist.github.com/kyrs/9adf86366e9e4f04addb中的代码(它将 opencv cv::Mat 图像作为输入并将其转换为张量)，我用它来标记带有模型 inception_v3_2016_08_28_frozen.pb 在 Tensorflow 教程 ( https://www.tensorflow.org/tutorials/image_recognition#usage_with_the_c_api ) 中说明的图像。使用 batchsize 为 1 时一切正常。但是，当我将 batchsize 增加到 2(或更大)时，finalOutput(属于 std::vector 类型)为零。

这是重现错误的代码:

// Only for VisualStudio
#define COMPILER_MSVC
#define NOMINMAX

#include <string>
#include <iostream>
#include <fstream>

#include <opencv2/opencv.hpp>
#include <opencv2/imgproc/imgproc.hpp>

#include "tensorflow/core/public/session.h"
#include "tensorflow/core/platform/env.h"
#include "tensorflow/core/framework/tensor.h"

int batchSize = 2;
int height = 299;
int width = 299;
int depth = 3;

int mean = 0;
int stdev = 255;

// Set image paths
cv::String pathFilenameImg1 = "D:/IMGS/grace_hopper.jpg";
cv::String pathFilenameImg2 = "D:/IMGS/lenna.jpg";

// Set model paths
std::string graphFile = "D:/Tensorflow/models/inception_v3_2016_08_28_frozen.pb";
std::string labelfile = "D:/Tensorflow/models/imagenet_slim_labels.txt";
std::string InputName = "input";
std::string OutputName = "InceptionV3/Predictions/Reshape_1";


void read_prepare_image(cv::String pathImg, cv::Mat &imgPrepared) {

       // Read Color image:
       cv::Mat imgBGR = cv::imread(pathImg);

       // Now we resize the image to fit Model's expected sizes:
       cv::Size s(height, width);
       cv::Mat imgResized;
       cv::resize(imgBGR, imgResized, s, 0, 0, cv::INTER_CUBIC);

       // Convert the image to float and normalize data:
       imgResized.convertTo(imgPrepared, CV_32FC1);
       imgPrepared = imgPrepared - mean;
       imgPrepared = imgPrepared / stdev;

}

int main()
{
       // Read and prepare images using OpenCV:
       cv::Mat img1, img2;
       read_prepare_image(pathFilenameImg1, img1);
       read_prepare_image(pathFilenameImg2, img2);

       // creating a Tensor for storing the data
       tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({ batchSize, height, width, depth }));
       auto input_tensor_mapped = input_tensor.tensor<float, 4>();

       // Copy images data into the tensor:
       for (int b = 0; b < batchSize; ++b) {

             const float * source_data;

             if (b == 0) 
                    source_data = (float*)img1.data;
             else 
                    source_data = (float*)img2.data;

             for (int y = 0; y < height; ++y) {

                    const float* source_row = source_data + (y * width * depth);
                    for (int x = 0; x < width; ++x) {

                           const float* source_pixel = source_row + (x * depth);
                           const float* source_B = source_pixel + 0;
                           const float* source_G = source_pixel + 1;
                           const float* source_R = source_pixel + 2;

                           input_tensor_mapped(b, y, x, 0) = *source_R;
                           input_tensor_mapped(b, y, x, 1) = *source_G;
                           input_tensor_mapped(b, y, x, 2) = *source_B;

                    }
             }
       }

       // Load the graph:
       tensorflow::GraphDef graph_def;
       ReadBinaryProto(tensorflow::Env::Default(), graphFile, &graph_def);

       // create a session with the graph
       std::unique_ptr<tensorflow::Session> session_inception(tensorflow::NewSession(tensorflow::SessionOptions()));
       session_inception->Create(graph_def);

       // run the loaded graph 
       std::vector<tensorflow::Tensor> finalOutput;
       session_inception->Run({ { InputName,input_tensor } }, { OutputName }, {}, &finalOutput);

       // Get Top 5 classes:
       std::cerr << "final output size = " << finalOutput.size() << std::endl;
       tensorflow::Tensor output = std::move(finalOutput.at(0));
       auto scores = output.flat<float>();
       std::cerr << "scores size=" << scores.size() << std::endl;

       std::ifstream label(labelfile);
       std::string line;

       std::vector<std::pair<float, std::string>> sorted;

       for (unsigned int i = 0; i <= 1000; ++i) {
             std::getline(label, line);
             sorted.emplace_back(scores(i), line);
       }

       std::sort(sorted.begin(), sorted.end());
       std::reverse(sorted.begin(), sorted.end());
       std::cout << "size of the sorted file is " << sorted.size() << std::endl;
       for (unsigned int i = 0; i< 5; ++i)
             std::cout << "The output of the current graph has category  " << sorted[i].second << " with probability " << sorted[i].first << std::endl;

}

我错过了什么吗？有什么想法吗？

提前致谢!

Answer 1

我遇到了同样的问题。当我更改为 https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/benchmark 中使用的模型时(开始的不同版本)更大的批量大小可以正常工作。

请注意，您需要将输入大小从 299,299,3 更改为 224,224,3，并将输入和输出层名称更改为:input:0 和 output:0