java - 如何使用分水岭改进图像分割？

Question

我正在开发一个应用程序来检测病变区域，为此我使用抓取来检测 ROI 并从图像中移除背景。但是在某些图像中，它无法正常工作。他最终没有很好地识别感兴趣区域的边界。分水岭可以更好地识别此类工作的边缘，但是我在从抓斗过渡到分水岭时遇到了困难。在处理grabcut之前，用户使用touchevent在感兴趣的图像(伤口区域)周围标记一个矩形，以方便算法的工作。如下图。



但是，使用其他伤口图像，分割效果不好，显示出 ROI 检测的缺陷。

在应用程序中使用抓取的图像



在桌面中使用分水岭的图像



这是代码:

private fun extractForegroundFromBackground(coordinates: Coordinates, currentPhotoPath: String): String {
    // TODO: Provide complex object that has both path and extension

    val width = bitmap?.getWidth()!!
    val height = bitmap?.getHeight()!!
    val rgba = Mat()
    val gray_mat = Mat()
    val threeChannel = Mat()
    Utils.bitmapToMat(bitmap, gray_mat)
    cvtColor(gray_mat, rgba, COLOR_RGBA2RGB)
    cvtColor(rgba, threeChannel, COLOR_RGB2GRAY)
    threshold(threeChannel, threeChannel, 100.0, 255.0, THRESH_OTSU)

    val rect = Rect(coordinates.first, coordinates.second)
    val fg = Mat(rect.size(), CvType.CV_8U)
    erode(threeChannel, fg, Mat(), Point(-1.0, -1.0), 10)
    val bg = Mat(rect.size(), CvType.CV_8U)
    dilate(threeChannel, bg, Mat(), Point(-1.0, -1.0), 5)
    threshold(bg, bg, 1.0, 128.0, THRESH_BINARY_INV)
    val markers = Mat(rgba.size(), CvType.CV_8U, Scalar(0.0))
    Core.add(fg, bg, markers)

    val marker_tempo = Mat()
    markers.convertTo(marker_tempo, CvType.CV_32S)

    watershed(rgba, marker_tempo)
    marker_tempo.convertTo(markers, CvType.CV_8U)

    val imgBmpExit = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565)
    Utils.matToBitmap(markers, imgBmpExit)

    image.setImageBitmap(imgBmpExit)


    // Run the grab cut algorithm with a rectangle (for subsequent iterations with touch-up strokes,
    // flag should be Imgproc.GC_INIT_WITH_MASK)
    //Imgproc.grabCut(srcImage, firstMask, rect, bg, fg, iterations, Imgproc.GC_INIT_WITH_RECT)

    // Create a matrix of 0s and 1s, indicating whether individual pixels are equal
    // or different between "firstMask" and "source" objects
    // Result is stored back to "firstMask"
    //Core.compare(mark, source, mark, Core.CMP_EQ)

    // Create a matrix to represent the foreground, filled with white color
    val foreground = Mat(srcImage.size(), CvType.CV_8UC3, Scalar(255.0, 255.0, 255.0))

    // Copy the foreground matrix to the first mask
    srcImage.copyTo(foreground, mark)

    // Create a red color
    val color = Scalar(255.0, 0.0, 0.0, 255.0)
    // Draw a rectangle using the coordinates of the bounding box that surrounds the foreground
    rectangle(srcImage, coordinates.first, coordinates.second, color)

    // Create a new matrix to represent the background, filled with black color
    val background = Mat(srcImage.size(), CvType.CV_8UC3, Scalar(0.0, 0.0, 0.0))

    val mask = Mat(foreground.size(), CvType.CV_8UC1, Scalar(255.0, 255.0, 255.0))
    // Convert the foreground's color space from BGR to gray scale
    cvtColor(foreground, mask, Imgproc.COLOR_BGR2GRAY)

    // Separate out regions of the mask by comparing the pixel intensity with respect to a threshold value
    threshold(mask, mask, 254.0, 255.0, Imgproc.THRESH_BINARY_INV)

    // Create a matrix to hold the final image
    val dst = Mat()
    // copy the background matrix onto the matrix that represents the final result
    background.copyTo(dst)

    val vals = Mat(1, 1, CvType.CV_8UC3, Scalar(0.0))
    // Replace all 0 values in the background matrix given the foreground mask
    background.setTo(vals, mask)

    // Add the sum of the background and foreground matrices by applying the mask
    Core.add(background, foreground, dst, mask)

    // Save the final image to storage
    Imgcodecs.imwrite(currentPhotoPath + "_tmp.png", dst)

    // Clean up used resources
    firstMask.release()
    source.release()
    //bg.release()
    //fg.release()
    vals.release()
    dst.release()

    return currentPhotoPath
}

导出:



如何更新代码以使用分水岭而不是 Grabcut？

Answer 1

关于如何在 OpenCV 中应用分水岭算法的描述是 here ，尽管它是在 Python 中。 documentation还包含一些可能有用的示例。由于您已经有了二值图像，剩下的就是应用欧几里德距离变换 (EDT) 和分水岭函数。所以而不是Imgproc.grabCut(srcImage, firstMask, rect, bg, fg, iterations, Imgproc.GC_INIT_WITH_RECT) ， 你将会拥有:

Mat dist = new Mat();
Imgproc.distanceTransform(srcImage, dist, Imgproc.DIST_L2, Imgproc.DIST_MASK_3); // use L2 for Euclidean Distance 
Mat markers = Mat.zeros(dist.size(), CvType.CV_32S);
Imgproc.watershed(dist, markers); # apply watershed to resultant image from EDT
Mat mark = Mat.zeros(markers.size(), CvType.CV_8U);
markers.convertTo(mark, CvType.CV_8UC1);
Imgproc.threshold(mark, firstMask, 0, 255, Imgproc.THRESH_BINARY + Imgproc.THRESH_OTSU); # threshold results to get binary image

阈值化步骤描述 here .此外，可选地，在您申请 Imgproc.watershed 之前，您可能希望对 EDT 的结果应用一些形态学运算，即；膨胀，腐 eclipse :

Imgproc.dilate(dist, dist, Mat.ones(3, 3, CvType.CV_8U));

如果您在处理二值图像时不熟悉形态学运算， OpenCV documentation包含一些好的，快速的例子。

希望这可以帮助!