python - 如何使用 Python 检测图像中的矩形项目

Question

我发现了很多关于使用 openCV 等在图像中查找“事物”的问题。在 Python 中，但到目前为止，我无法将它们拼凑在一起以可靠地解决我的问题。

我正在尝试使用计算机视觉来帮助计数微小的表面贴装电子零件。这个想法是让我把零件倒在一张纯色纸上，拍张照片，然后让软件告诉我里面有多少元素。

“事物”从一张图片到下一张图片都不同，但在任何一张图片中总是相同的。我似乎能够手动调整特定部分的色调/饱和度等参数，但每次更换新部分时往往需要进行调整。

我当前的半功能代码贴在下面:

import imutils
import numpy
import cv2
import sys

def part_area(contours, round=10):
    """Finds the mode of the contour area.  The idea is that most of the parts in an image will be separated and that
    finding the most common area in the list of areas should provide a reasonable value to approximate by.  The areas
    are rounded to the nearest multiple of 200 to reduce the list of options."""
    # Start with a list of all of the areas for the provided contours.
    areas = [cv2.contourArea(contour) for contour in contours]
    # Determine a threshold for the minimum amount of area as 1% of the overall range.
    threshold = (max(areas) - min(areas)) / 100
    # Trim the list of areas down to only those that exceed the threshold.
    thresholded = [area for area in areas if area > threshold]
    # Round the areas to the nearest value set by the round argument.
    rounded = [int((area + (round / 2)) / round) * round for area in thresholded]
    # Remove any areas that rounded down to zero.
    cleaned = [area for area in rounded if area != 0]
    # Count the areas with the same values.
    counts = {}
    for area in cleaned:
        if area not in counts:
            counts[area] = 0
        counts[area] += 1
    # Reduce the areas down to only those that are in groups of three or more with the same area.
    above = []
    for area, count in counts.iteritems():
        if count > 2:
            for _ in range(count):
                above.append(area)
    # Take the mean of the areas as the average part size.
    average = sum(above) / len(above)
    return average

def find_hue_mode(hsv):
    """Given an HSV image as an input, compute the mode of the list of hue values to find the most common hue in the
    image.  This is used to determine the center for the background color filter."""
    pixels = {}
    for row in hsv:
        for pixel in row:
            hue = pixel[0]
            if hue not in pixels:
                pixels[hue] = 0
            pixels[hue] += 1
    counts = sorted(pixels.keys(), key=lambda key: pixels[key], reverse=True)
    return counts[0]


if __name__ == "__main__":
    # load the image and resize it to a smaller factor so that the shapes can be approximated better
    image = cv2.imread(sys.argv[1])

    # define range of blue color in HSV
    hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
    center = find_hue_mode(hsv)
    print 'Center Hue:', center

    lower = numpy.array([center - 10, 50, 50])
    upper = numpy.array([center + 10, 255, 255])
    # Threshold the HSV image to get only blue colors
    mask = cv2.inRange(hsv, lower, upper)
    inverted = cv2.bitwise_not(mask)

    blurred = cv2.GaussianBlur(inverted, (5, 5), 0)
    edged = cv2.Canny(blurred, 50, 100)
    dilated = cv2.dilate(edged, None, iterations=1)
    eroded = cv2.erode(dilated, None, iterations=1)

    # find contours in the thresholded image and initialize the shape detector
    contours = cv2.findContours(eroded.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    contours = contours[0] if imutils.is_cv2() else contours[1]

    # Compute the area for a single part to use when setting the threshold and calculating the number of parts within
    # a contour area.
    part_area = part_area(contours)
    # The threshold for a part's area - can't be too much smaller than the part itself.
    threshold = part_area * 0.5

    part_count = 0
    for contour in contours:
        if cv2.contourArea(contour) < threshold:
            continue

        # Sometimes parts are close enough together that they become one in the image.  To battle this, the total area
        # of the contour is divided by the area of a part (derived earlier).
        part_count += int((cv2.contourArea(contour) / part_area) + 0.1)  # this 0.1 "rounds up" slightly and was determined empirically

        # Draw an approximate contour around each detected part to give the user an idea of what the tool has computed.
        epsilon = 0.1 * cv2.arcLength(contour, True)
        approx = cv2.approxPolyDP(contour, epsilon, True)
        cv2.drawContours(image, [approx], -1, (0, 255, 0), 2)

    # Print the part count and show off the processed image.
    print 'Part Count:', part_count
    cv2.imshow("Image", image)
    cv2.waitKey(0)

这是我使用的输入图像类型的示例: 或这个:

我目前得到的结果是这样的:

结果清楚地表明，脚本在识别某些部分时遇到了困难，而且当部分相互接触时，这似乎是真正的致命弱点。

所以我的问题/挑战是，我可以做些什么来提高这个脚本的可靠性？

该脚本将集成到现有的 Python 工具中，因此我正在寻找使用 Python 的解决方案。该解决方案不需要是纯 Python，因为我愿意安装可能需要的任何第 3 方库。

Answer 1

如果所有对象都是相似类型，您可能会更成功地隔离图像中的单个示例，然后使用特征匹配来检测它们。

完整的解决方案超出了 Stack Overflow 的范围，但我对进步的建议是首先使用您当前的矩形检索方法以某种方式找到一个或多个“正确”示例。您可能会寻找所有具有预期大小或精确矩形的样本。

一旦您分离出一些正面示例，就可以使用一些特征匹配技术来找到其他示例。有一个lot of reading up you probably need to do on it但这是一个潜在的解决方案。

一般的总结是，你使用你的正面例子来找到你想要检测的对象的“特征”。这些“特征”通常是拐角或渐变变化之类的东西。 OpenCV 包含许多您可以使用的方法。

获得这些特征后，您可以查看 OpenCV 中的几种算法，这些算法将搜索图像以查找所有匹配的特征。您需要旋转不变性(可以检测以不同旋转排列的相同特征)，但您可能不需要尺度不变性(可以检测多个尺度的相同特征)。

我对这种方法的一个担心是您在图像中搜索的项目非常小。可能很难找到好的、一致的特征来匹配。