python - OpenCV 数字合并到周围的盒子中

Question

我有一堆日期正在尝试使用 tesseract 进行 OCR。但是，日期中的许多数字与日期框中的行合并，如下所示:

---

---

另外，这是一张我可以很好地进行镶嵌的好图片:

---

这是我的代码:

import os
import cv2
from matplotlib import pyplot as plt
import subprocess
import numpy as np
from PIL import Image

def show(img):
    plt.figure(figsize=(20,20))
    plt.imshow(img,cmap='gray')
    plt.show()

def sort_contours(cnts, method="left-to-right"):
    # initialize the reverse flag and sort index
    reverse = False
    i = 0

    # handle if we need to sort in reverse
    if method == "right-to-left" or method == "bottom-to-top":
        reverse = True

    # handle if we are sorting against the y-coordinate rather than
    # the x-coordinate of the bounding box
    if method == "top-to-bottom" or method == "bottom-to-top":
        i = 1

    # construct the list of bounding boxes and sort them from top to
    # bottom
    boundingBoxes = [cv2.boundingRect(c) for c in cnts]

    cnts, boundingBoxes = zip(*sorted(zip(cnts, boundingBoxes),
        key=lambda b:b[1][i], reverse=reverse))

    # return the list of sorted contours and bounding boxes
    return cnts, boundingBoxes


def tesseract_it(contours,main_img, label,delete_last_contour=False):
    min_limit, max_limit = (1300,1700)
    idx =0 
    roi_list = []
    slist= set()
    for cnt in contours:
        idx += 1
        x,y,w,h = cv2.boundingRect(cnt)
        if label=='boxes':
            roi=main_img[y+2:y+h-2,x+2:x+w-2]
        else:
            roi=main_img[y:y+h,x:x+w]

        if w*h > min_limit and w*h < max_limit and w>10 and w< 50 and h>10 and h<50:
            if (x,y,w,h) not in slist: # Stops from identifying repeted contours

                roi = cv2.resize(roi,dsize=(45,45),fx=0 ,fy=0, interpolation = cv2.INTER_AREA)
                roi_list.append(roi)
                slist.add((x,y,w,h))

    if not delete_last_contour:
        vis = np.concatenate((roi_list),1)
    else:
        roi_list.pop(-1)
        vis = np.concatenate((roi_list),1)

    show(vis)

    # Tesseract the final image here
    # ...


image = 'bad_digit/1.jpg'
# image = 'bad_digit/good.jpg'
specimen_orig = cv2.imread(image,0)


specimen = cv2.fastNlMeansDenoising(specimen_orig)
#     show(specimen)
kernel = np.ones((3,3), np.uint8)

# Now we erode
specimen = cv2.erode(specimen, kernel, iterations = 1)
#     show(specimen)
_, specimen = cv2.threshold(specimen, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
#     show(specimen)
specimen_canny = cv2.Canny(specimen, 0, 0)
#     show(specimen_canny)

specimen_blank_image = np.zeros((specimen.shape[0], specimen.shape[1], 3))
_,specimen_contours, retr = cv2.findContours(specimen_canny.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE )
# print(len(specimen_contours))
cv2.drawContours(specimen_blank_image, specimen_contours, -1, 100, 2)
#     show(specimen_blank_image)
specimen_blank_image = np.zeros((specimen.shape[0], specimen.shape[1], 3))

specimen_sorted_contours, specimen_bounding_box = sort_contours(specimen_contours)

output_string = tesseract_it(specimen_sorted_contours,specimen_orig,label='boxes',)
# return output_string

---

附加的好图像的输出是这样的:

---

Tesseracing 这张图片确实给了我准确的结果。

但是，对于那些行合并为数字的行，我的输出如下所示:

这些根本不适用于 Tesseract。我想知道是否有办法删除线条并只保留数字。

我也尝试了以下方法: https://docs.opencv.org/3.2.0/d1/dee/tutorial_moprh_lines_detection.html

这在我附上的图片上似乎并不是很好。

我也尝试过使用 imagemagick:

convert original.jpg \
\( -clone 0 -threshold 50% -negate -statistic median 200x1 \)  \
-compose lighten -composite                                    \
\( -clone 0 -threshold 50% -negate -statistic median 1x200 \)  \
-composite output.jpg

它的结果是公平的，但删除的行有点穿过数字，如下所示:

有没有更好的方法可以解决这个问题？我的最终目标是对数字进行曲面 segmentation ，因此最终图像确实需要非常清晰。

Answer 1

这里有一些看起来运行良好的代码。有两个阶段:

• 可以观察到数字比方框略粗。加上整个图像具有很强的水平性。因此，我们可以在水平方向应用更强的膨胀来消除大多数垂直线。
• 此时，OCR，例如 Google's one ，可以检测到大多数数字。不幸的是，它有点太好了，并且看到了其他东西，所以我添加了另一个更复杂且与您的特定上下文非常相关的阶段。

这是第一阶段后的一张图片的结果:

这是第二阶段之后的所有结果:

如您所见，8 并不完美，可以将其视为 B(好吧，即使是像我这样的人也将其视为 B……但如果您的世界中只有数字，则可以轻松解决)。还有像“:”字符(已删除的垂直线的遗留物)，我无法摆脱任何不调整代码太多...

C# 代码:

static void Unbox(string inputFilePath, string outputFilePath)
{
    using (var orig = new Mat(inputFilePath))
    {
        using (var gray = orig.CvtColor(ColorConversionCodes.BGR2GRAY))
        {
            using (var dst = orig.EmptyClone())
            {
                // this is what I call the "horizontal shake" pass.
                // note I use the Rect shape here, this is important
                using (var dilate = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(4, 1)))
                {
                    Cv2.Dilate(gray, dst, dilate);
                }

                // erode just a bit to get back some numbers to life
                using (var erode = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(2, 1)))
                {
                    Cv2.Erode(dst, dst, erode);
                }

                // at this point, good OCR will see most numbers
                // but we want to remove surrounding artifacts

                // find countours
                using (var canny = dst.Canny(0, 400))
                {
                    var contours = canny.FindContoursAsArray(RetrievalModes.List, ContourApproximationModes.ApproxSimple);

                    // compute a bounding rect for all numbers w/o boxes and artifacts
                    // this is the tricky part where we try to discard what's not related exclusively to numbers
                    var boundingRect = Rect.Empty;
                    foreach (var contour in contours)
                    {
                        // discard some small and broken polygons
                        var polygon = Cv2.ApproxPolyDP(contour, 4, true);
                        if (polygon.Length < 3)
                            continue;

                        // we want only numbers, and boxes are approx 40px wide,
                        // so let's discard box-related polygons, if any
                        // and some other artifacts that passed previous checks
                        // this quite depends on some context knowledge...
                        var rect = Cv2.BoundingRect(polygon);
                        if (rect.Width > 40 || rect.Height < 15)
                            continue;

                        boundingRect = boundingRect.X == 0 ? rect : boundingRect.Union(rect);
                    }

                    using (var final = dst.Clone(boundingRect))
                    {
                        final.SaveImage(outputFilePath);
                    }
                }
            }
        }
    }
}