java - 拼接2张图片(OpenCV)

Question

我正在尝试使用 OpenCV Java API 将两个图像拼接在一起。但是，我得到了错误的输出，无法解决问题。我使用以下步骤:1.检测特征2.提取特征3.匹配功能。4.寻找单应性5.寻找透视变换6. 经线透视7. 将两张图片“拼接”成一张组合图片。

但是我哪里出错了。我认为这是我梳理两张图片的方式，但我不确定。我在 2 张图像之间得到 214 个良好的特征匹配，但无法拼接它们？

public class ImageStitching {

static Mat image1;
static Mat image2;

static FeatureDetector fd;
static DescriptorExtractor fe;
static DescriptorMatcher fm;

public static void initialise(){
    fd = FeatureDetector.create(FeatureDetector.BRISK); 
    fe = DescriptorExtractor.create(DescriptorExtractor.SURF); 
    fm = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE);

    //images
    image1 = Highgui.imread("room2.jpg");
    image2 = Highgui.imread("room3.jpg");

    //structures for the keypoints from the 2 images
    MatOfKeyPoint keypoints1 = new MatOfKeyPoint();
    MatOfKeyPoint keypoints2 = new MatOfKeyPoint();

    //structures for the computed descriptors
    Mat descriptors1 = new Mat();
    Mat descriptors2 = new Mat();

    //structure for the matches
    MatOfDMatch matches = new MatOfDMatch();

    //getting the keypoints
    fd.detect(image1, keypoints1);
    fd.detect(image1, keypoints2);

    //getting the descriptors from the keypoints
    fe.compute(image1, keypoints1, descriptors1);
    fe.compute(image2,keypoints2,descriptors2);

    //getting the matches the 2 sets of descriptors 
    fm.match(descriptors2,descriptors1, matches);

    //turn the matches to a list
    List<DMatch> matchesList = matches.toList();

    Double maxDist = 0.0; //keep track of max distance from the matches
    Double minDist = 100.0; //keep track of min distance from the matches

    //calculate max & min distances between keypoints
    for(int i=0; i<keypoints1.rows();i++){
        Double dist = (double) matchesList.get(i).distance;
        if (dist<minDist) minDist = dist;
        if(dist>maxDist) maxDist=dist;
    }

    System.out.println("max dist: " + maxDist );
    System.out.println("min dist: " + minDist);

    //structure for the good matches
    LinkedList<DMatch> goodMatches = new LinkedList<DMatch>();

    //use only the good matches (i.e. whose distance is less than 3*min_dist)
    for(int i=0;i<descriptors1.rows();i++){
        if(matchesList.get(i).distance<3*minDist){
            goodMatches.addLast(matchesList.get(i));
        }
    }

    //structures to hold points of the good matches (coordinates)
    LinkedList<Point> objList = new LinkedList<Point>(); // image1
    LinkedList<Point> sceneList = new LinkedList<Point>(); //image 2

    List<KeyPoint> keypoints_objectList = keypoints1.toList();
    List<KeyPoint> keypoints_sceneList = keypoints2.toList();

    //putting the points of the good matches into above structures
    for(int i = 0; i<goodMatches.size(); i++){
        objList.addLast(keypoints_objectList.get(goodMatches.get(i).queryIdx).pt);
        sceneList.addLast(keypoints_sceneList.get(goodMatches.get(i).trainIdx).pt);
    }

    System.out.println("\nNum. of good matches" +goodMatches.size());

    MatOfDMatch gm = new MatOfDMatch();
    gm.fromList(goodMatches);

    //converting the points into the appropriate data structure
    MatOfPoint2f obj = new MatOfPoint2f();
    obj.fromList(objList);

    MatOfPoint2f scene = new MatOfPoint2f();
    scene.fromList(sceneList);

    //finding the homography matrix
    Mat H = Calib3d.findHomography(obj, scene);

    //LinkedList<Point> cornerList = new LinkedList<Point>();
    Mat obj_corners = new Mat(4,1,CvType.CV_32FC2);
    Mat scene_corners = new Mat(4,1,CvType.CV_32FC2);

    obj_corners.put(0,0, new double[]{0,0});
    obj_corners.put(0,0, new double[]{image1.cols(),0});
    obj_corners.put(0,0,new double[]{image1.cols(),image1.rows()});
    obj_corners.put(0,0,new double[]{0,image1.rows()});

    Core.perspectiveTransform(obj_corners, scene_corners, H);

    //structure to hold the result of the homography matrix
    Mat result = new Mat();

    //size of the new image - i.e. image 1 + image 2
    Size s = new Size(image1.cols()+image2.cols(),image1.rows());

    //using the homography matrix to warp the two images
    Imgproc.warpPerspective(image1, result, H, s);
    int i = image1.cols();
    Mat m = new Mat(result,new Rect(i,0,image2.cols(), image2.rows()));

    image2.copyTo(m);

    Mat img_mat = new Mat();

    Features2d.drawMatches(image1, keypoints1, image2, keypoints2, gm, img_mat, new Scalar(254,0,0),new Scalar(254,0,0) , new MatOfByte(), 2);

    //creating the output file
    boolean imageStitched = Highgui.imwrite("imageStitched.jpg",result);
    boolean imageMatched = Highgui.imwrite("imageMatched.jpg",img_mat);
}


public static void main(String args[]){
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    initialise();
}

我不能嵌入图片或发布超过 2 个链接，因为声誉点数？所以我链接了错误拼接的图像和显示两张图像之间匹配特征的图像(以了解问题):

不正确的拼接图像:http://oi61.tinypic.com/11ac01c.jpg检测到的特征:http://oi57.tinypic.com/29m3wif.jpg

Answer 1

您似乎有很多离群值，使单应性估计不正确。因此，您可以使用递归拒绝这些异常值的 RANSAC 方法。

不需要太多努力，只需在 findHomography 函数中使用第三个参数即可:

Mat H = Calib3d.findHomography(obj, scene, CV_RANSAC);

编辑

然后尝试确保提供给检测器的图像是 8 位灰度图像，如前所述 here