c++ - 在 Aruco 标记周围绘制图像

Question

我正在尝试使用 OpenCV 4.1.1 和 Aruco 构建一个简单的增强现实应用程序。目标是将图像叠加在标记之上，但图像超出标记的边缘。

我已经校准了我的相机并获得了相机矩阵和畸变系数。通过使用 OpenCV 的 warpPerspective，我可以在标记的顶部绘制图像，但我只能将它绑定(bind)到标记的角上，以便它保持在标记的边界内。

    std::vector<int> ids;
    std::vector<std::vector<Point2f>> corners;

    // detect markers
    aruco::detectMarkers(image, dictionary, corners, ids);

if (ids.size() > 0) {
    // file with image to draw
    auto file = "square.png";
    // image to draw on the marker
    Mat im_src = imread(file);

    if (im_src.data == NULL) {
        std::cout << file << ": File not found\n" << std::endl;
        continue;
        }

        // flip(im_src, im_src, 1);

        // points of corners of the image
        std::vector<Point2f> pts_src;
        pts_src.push_back(Point2f(0, 0));
        pts_src.push_back(Point2f(im_src.cols-1, 0));
        pts_src.push_back(Point2f(im_src.cols-1, im_src.rows-1));
        pts_src.push_back(Point2f(0, im_src.rows-1));

        // use aruco marker
        for (int i = 0; i < ids.size(); i++) {
            if (ids[i] == 69) {
                aruco::drawDetectedMarkers(imageCopy, corners, ids);

                std::vector<Point> pts_dst;
                pts_dst.push_back(corners[i][0]);
                pts_dst.push_back(corners[i][1]);
                pts_dst.push_back(corners[i][2]);
                pts_dst.push_back(corners[i][3]);

                Mat h = findHomography(pts_src, pts_dst);

                Mat im_out;
                warpPerspective(im_src, im_out, h, imageCopy.size()); 
                fillConvexPoly(imageCopy, pts_dst, 0, 16);
                imageCopy = imageCopy + im_out; 
            }
        }

这是我所拥有的和我想要的图像。我想我需要使用 3d 点来绘制图像，但我不确定该怎么做。任何帮助将不胜感激。

[

[

Answer 1

正如您在评论中所说，如果标记长度可用，比如 l0，您可以将所需正方形的长度定义为 l = l0 * 1.05 或一些东西。

for (int i = 0; i < ids.size(); i++) {
    if (ids[i] == 69) {
        aruco::drawDetectedMarkers(imageCopy, corners, ids);

        // Estimate the pose of the marker
        std::vector<cv::Vec3d> rvecs, tvecs;
        cv::aruco::estimatePoseSingleMarkers(
            corners, l0, camera_matrix, dist_coeffs, 
            rvecs, tvecs
        );

        drawSquare(
            image_copy, camera_matrix, dist_coeffs, rvecs[i], tvecs[i], 
            l0
        );
    }
}


void drawSquare(
    cv::InputOutputArray image, cv::InputArray cameraMatrix, 
    cv::InputArray distCoeffs, cv::InputArray rvec, cv::InputArray tvec, 
    float l0
)
{
    float l = l0 * 1.05;  // new square is 5% larger than the aruco marker
    float half_l = l / 2.0;

    // Define the square on the camera frame (this is 3D since the camera
    // frame is 3D).
    std::vector<cv::Point3f> squarePoints;
    squarePoints.push_back(cv::Point3f(half_l, half_l, 0));
    squarePoints.push_back(cv::Point3f(half_l, -half_l, 0));
    squarePoints.push_back(cv::Point3f(-half_l, -half_l, 0));
    squarePoints.push_back(cv::Point3f(-half_l, half_l, 0));

    // Project the square to the image.
    std::vector<cv::Point2f> imagePoints;
    projectPoints(
        squarePoints, rvec, tvec, cameraMatrix, distCoeffs, imagePoints
    );

    // Draw the square on the image.
    cv::line(image, imagePoints[0], imagePoints[1], cv::Scalar(255, 0, 0), 3);
    cv::line(image, imagePoints[1], imagePoints[2], cv::Scalar(255, 0, 0), 3);
    cv::line(image, imagePoints[2], imagePoints[3], cv::Scalar(255, 0, 0), 3);
    cv::line(image, imagePoints[3], imagePoints[0], cv::Scalar(255, 0, 0), 3);
}

我没有对此进行测试，但我为不同的项目使用了类似的代码。如果您遇到任何问题，请告诉我。我会更新上面的代码。