python - OpenCV + Python : Calculate stereo reprojection error

Question

我想做类似于 this question 的事情，但用于 stereoCalibrate() 而不是 calibrateCamera()。也就是说，计算立体相机校准的重投影误差。

我的简化示例如下所示:

import cv2
import numpy as np

def calibrate_stereo(w, h, objpoints, imgpoints_l, imgpoints_r):
    stereocalib_criteria = (cv2.TERM_CRITERIA_COUNT + cv2.TERM_CRITERIA_EPS , 1000, 1e-6)
    retval, A1, D1, A2, D2, R, T, E, F = cv2.stereoCalibrate(objpoints,imgpoints_l, imgpoints_r,None,None,None,None, (w,h), flags=0, criteria=stereocalib_criteria)

    return (retval, (A1,D1,A2,D2, R, T, E, F))

def calc_rms_stereo(objectpoints, imgpoints_l, imgpoints_r, A1, D1, A2, D2, R, T):
    tot_error = 0
    total_points = 0

    for i, objpoints in enumerate(objectpoints):
        # calculate world <-> cam1 transformation
        _, rvec_l, tvec_l,_ = cv2.solvePnPRansac(objpoints, imgpoints_l[i], A1, D1)

        # compute reprojection error for cam1
        rp_l, _ = cv2.projectPoints(objpoints, rvec_l, tvec_l, A1, D1)
        tot_error += np.sum(np.square(np.float64(imgpoints_l[i] - rp_l)))
        total_points += len(objpoints)

        # calculate world <-> cam2 transformation
        rvec_r, tvec_r  = cv2.composeRT(rvec_l,tvec_l,cv2.Rodrigues(R)[0],T)[:2]

        # compute reprojection error for cam2
        rp_r,_ = cv2.projectPoints(objpoints, rvec_r, tvec_r, A2, D2)
        tot_error += np.square(imgpoints_r[i] - rp_r).sum()
        total_points += len(objpoints)

    mean_error = np.sqrt(tot_error/total_points)

    return mean_error


if __name__ == "__main__":    
    # omitted: reading values for w,h, objectPoints, imgpoints_l, imgpoints_r from file (format as expected by the OpenCV functions)
    # [...]

    rms, (A1,D1,A2,D2,R,T,_,_) = calibrate_stereo(w, h, objectpoints, imgpoints_l, imgpoints_r)

    print("RMS (stereo calib): {}".format(rms))

    rms_2 = calc_rms_stereo(objectpoints, imgpoints_l, imgpoints_r, A1, D1, A2, D2, R, T)    
    print("RMS (custom calculation):", rms_2)

示例输出:

RMS (stereo calib): 0.14342257926694932
RMS (custom calculation): 0.356273345751

据我所知，stereoCalibrate() 源代码中的计算与我的非常相似。我错过了什么？

Ubuntu 上的 OpenCV 3.3.0

Answer 1

我在基于 OpenCV 实现的自定义立体校准算法后解决了这个问题。

cv2.stereoCalibrate()里面计算的重投影误差的区别|我的自定义计算源于外部参数的不同值 rvec_l和 tvec_l .这些向量描述了左摄像头与每个图像的校准图案之间的旋转和平移。 cv2.solvePnpRansac()仅根据左图像的重投影误差产生优化值，而在 cv2.stereoCalibrate() 中这些值与 R 一起优化和 T基于每个立体对的两个图像中的重投影误差。

如果想要精确复制 cv2.stereoCalibrate() 返回的 RMS 值, 必须修改 cv::stereoCalibrate() 的 C/C++ 源代码也返回优化的外部参数(cv::calibrateCamera() 已经为单目校准做了这件事)。