python-3.x - 使用opencv检测护照图像中的文本

Question

我正在使用 OpenCV 检测各种护照图像中的文本。任务是获取护照上出现的裁剪文本，如姓名、出生日期、国籍等。当前代码如下:

image = cv2.imread(image) # read image
image = imutils.resize(image, height=600)

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (3, 3), 0)

rectKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (13, 5))
sqKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (21, 21))
blackhat = cv2.morphologyEx(gray, cv2.MORPH_BLACKHAT, rectKernel)

gradX = cv2.Sobel(blackhat, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1)
gradX = np.absolute(gradX)
(minVal, maxVal) = (np.min(gradX), np.max(gradX))
gradX = (255 * ((gradX - minVal) / (maxVal - minVal))).astype("uint8")

gradX = cv2.morphologyEx(gradX, cv2.MORPH_CLOSE, rectKernel)
thresh = cv2.threshold(gradX, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]

if do_erosion: # do erosion if flag is True only
    thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, sqKernel)
    thresh = cv2.erode(thresh, None, iterations=4)
p = int(image.shape[1] * 0.05)
thresh[:, 0:p] = 0
thresh[:, image.shape[1] - p:] = 0   # thresholded image shown below

cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)   # finding contours from threshold image
cnts = imutils.grab_contours(cnts)
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
rois=[]
# loop over the contours
for c in cnts:
    # compute the bounding box of the contour and use the contour to
    # compute the aspect ratio and coverage ratio of the bounding box
    # width to the width of the image
    (x, y, w, h) = cv2.boundingRect(c)
    ar = w / float(h)
    crWidth = w / float(gray.shape[1])
    # check to see if the aspect ratio and coverage width are within
    # acceptable criteria
    
    if ar > 2 and crWidth > 0.05:
        # pad the bounding box since we applied erosions and now need
        # to re-grow it
        pX = int((x + w) * 0.03)
        pY = int((y + h) * 0.03)
        (x, y) = (x - pX, y - pY)
        (w, h) = (w + (pX * 2), h + (pY * 2))
        # extract the ROI from the image and draw a bounding box
        # surrounding the MRZ
        roi = original[y:y + h, x:x + w].copy()  # interested in finding all ROIs having text
        rois.append(roi)
        cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)

我正确检测了一些文本值，但解决方案并不通用。它有时只提取 DOB 的年份而不是完整的日期(有时它也会检测月份和日期，但不在同一个 ROI 中)。原始图像和阈值图像如下所示:

如您所见，在“找到的 ROI 图像”中，某些 ROI 未在同一图像中检测到，例如 DOB 和到期日期。从护照中提取所有文本区域的任何帮助将不胜感激。谢谢!

Answer 1

您需要调整纵横比和覆盖率阈值以获得所有需要的边界框。当我运行你的代码时，对于 05，ar 的值为 1.541 而 crWidth 的值为 0.03。由于这些值小于您指定的阈值，因此它们将被过滤掉。这就是为什么有些词在最终图像中没有边界框的原因。

但是，由于您想在单个边界框中获取整个 DOB，就在行 thresh[:, 0:p] = 0 之前，您可以应用膨胀操作:
thresh = cv2.morphologyEx(thresh, cv2.MORPH_DILATE, np.ones((1, 30), np.uint8))。

这将扩大像素并合并水平附近的 Blob 。预处理后的结果图像如下 -

最终图像 -