opencv - 'copyTo' 断言失败错误。检查图像大小和类型

Question

我尝试过在这个论坛上搜索、阅读文档和谷歌搜索。我找不到问题的答案。

下面简要介绍一下我正在尝试做的事情:

1. 从网络摄像头读取Mat对象frame(在父类OpenCVVideoWorker中声明为私有(private)成员)。
2. 使用 Rect_ 对象 ROIbox 给出的位置和坐标在框架上绘制一个矩形。
3. 使用 inRange 对感兴趣区域设置阈值。
4. 感兴趣区域中的阈值图像被转换为​​ RGB 并复制回输出帧的相同位置。

我使用 Qt 作为 GUI 和 Visual Studio 作为我的开发环境。当我运行该程序时，尽管在使用 copyTo 时确保图像的行、列和类型相同，但我仍收到“断言失败”错误。错误在 frameROIProcessed.copyTo(outputFrame(ROIbox))(我定期输入控制台输出以找到它)。

你能帮我找出我哪里出错了吗？

void OpenCVVideoWorker::processFrame() {
    frame.copyTo(outputFrame);

    if (ROIbox.width != 0 && ROIbox.height != 0) {
        rectangle(outputFrame, Point(ROIbox.x, ROIbox.y), cvPoint(ROIbox.x + ROIbox.width, ROIbox.y + ROIbox.height), Scalar(0, 0, 255), 1);
    }

    Mat frameROI = frame(ROIbox);

    if (frameROI.rows != 0 && frameROI.cols != 0) {
        Mat frameROIProcessed;
        cvtColor(frameROI, frameROIProcessed, COLOR_BGR2HSV);
        inRange(frameROIProcessed, Scalar(hueLow, saturationLow, brightnessLow), Scalar(hueHigh, saturationHigh, brightnessHigh), frameROIProcessed);
        cvtColor(frameROIProcessed, frameROIProcessed, COLOR_GRAY2BGR);

        qInfo("1) outputFrame(ROIbox)");
        qInfo(getImageType(outputFrame(ROIbox)).c_str());
        qInfo("2) frameROIProcessed");
        qInfo(getImageType(frameROIProcessed).c_str());
        qInfo("\n");

        frameROIProcessed.copyTo(outputFrame(ROIbox));
    }
    cvtColor(outputFrame, outputFrame, COLOR_BGR2RGB);
}

在线程中每 16ms 运行一次的函数中这样调用:

void OpenCVVideoWorker::receiveTimerTick() {    
    // Capture video from the webcam
    videoCapture.read(frame);
    if (frame.empty()) {
        qInfo("ERROR! blank frame grabbed");
        return;
    }

    // Process the captured frame
    processFrame();

    // Send the processed frame
    QImage outputImage((const unsigned char *) outputFrame.data, outputFrame.cols, outputFrame.rows, outputFrame.step, QImage::Format_RGB888);
    emit sendImage(outputImage);
}

我在 frameROIProcessed.copyTo(outputFrame(ROIbox)) 行出现以下错误:

Initializing the worker in its constructor...
Opening camera...
1) outputFrame(ROIbox)
8UC3 rows:4 cols:5
2) frameROIProcessed
8UC3 rows:4 cols:5


1) outputFrame(ROIbox)
8UC3 rows:27 cols:22
2) frameROIProcessed
8UC3 rows:27 cols:22


OpenCV(4.0.0-pre) Error: Assertion failed (!fixedSize() || ((Mat*)obj)->size.operator()() == Size(_cols, _rows)) in cv::debug_build_guard::_OutputArray::create, file d:\applications\opencv\source\opencv-master\modules\core\src\matrix_wrap.cpp, line 1227
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.0.0-pre) Error: Assertion failed (!fixedSize() || ((Mat*)obj)->size.operator()() == Size(_cols, _rows)) in cv::debug_build_guard::_OutputArray::create, file d:\applications\opencv\source\opencv-master\modules\core\src\matrix_wrap.cpp, line 1227

更新:我在非多线程格式中尝试了相同的代码并且成功了。现在我不确定为什么即使执行以下操作该函数也不是线程安全的(我仍然遇到相同的错误，但我想我确保一次只有一个函数实例在运行):

    if (isFrameBeingProcessed) {
        return;
    }
    isFrameBeingProcessed = true;
    frame.copyTo(outputFrame);

    if (ROIbox.width != 0 && ROIbox.height != 0) {
        rectangle(outputFrame, Point(ROIbox.x, ROIbox.y), cvPoint(ROIbox.x + ROIbox.width, ROIbox.y + ROIbox.height), Scalar(0, 0, 255), 1);
    }

    Mat frameROI = frame(ROIbox);

    if (frameROI.rows != 0 && frameROI.cols != 0) {
        Mat frameROIProcessed;
        cvtColor(frameROI, frameROIProcessed, COLOR_BGR2HSV);
        inRange(frameROIProcessed, Scalar(hueLow, saturationLow, brightnessLow), Scalar(hueHigh, saturationHigh, brightnessHigh), frameROIProcessed);
        cvtColor(frameROIProcessed, frameROIProcessed, COLOR_GRAY2BGR);

        qInfo("1) outputFrame(ROIbox)");
        qInfo(getImageType(outputFrame(ROIbox)).c_str());
        qInfo("2) frameROIProcessed");
        qInfo(getImageType(frameROIProcessed).c_str());
        qInfo((string("Box x: ")+to_string(ROIbox.x) + string(", y: ") + to_string(ROIbox.y) + string(", width: ") + to_string(ROIbox.width) + string(", height: ") + to_string(ROIbox.height)).c_str());
        qInfo((string("Mat width: ") + to_string(outputFrame(ROIbox).cols) + string(", height: ") + to_string(outputFrame(ROIbox).rows)).c_str());
        qInfo("\n");
        frameROIProcessed.copyTo(outputFrame(ROIbox));
    }
    cvtColor(outputFrame, outputFrame, COLOR_BGR2RGB);
    isFrameBeingProcessed = false;

Answer 1

我每一步都在控制台上显示调试语句，找到了问题所在。问题是 ROIbox 不是线程安全的(感谢@zindarod 和@Micka)。提供感兴趣区域框的 GUI ROIbox在不同的线程和ROIbox在帧的处理过程中改变大小。简单的解决方法是添加 Rect_<int> box = ROIbox;在函数的开头 processFrame()并在函数中使用这个复制的框。