python - 从图像中提取表格数据

Question

我们已经建立了一个检测表区域的模型。

下一步是解析检测到的表格图像并将其转换为 CSV/Dataframe。我们正面临着这个问题，我们已经尝试了一些技术，

尝试了 opencv reduce 方法来获得垂直的行或列分隔，但是当单词之间的距离更大时它会失败(下面共享示例)。
示例图像中的白框是 OCR 系统检测到的单词的实际位置。

下面的代码在图像上执行两次，
1. 图像被传递到 OCR 系统，它返回检测到的文本及其边界框。
2. 我们在黑色背景的图像上绘制边界框。
3.然后我们将图像传递给下面的代码两次，
第一 - 原始绘制的图像以获得水平线坐标
第二 - 绘制的图像旋转 90 度，然后再次传递给相同的代码以获得垂直线坐标。

通过使用坐标绘制线条，我们得到以下结果。这只是为了可视化。但在这种情况下它会失败。

代码也分享一下。

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
hist = cv2.reduce(gray, 1, cv2.REDUCE_AVG).reshape(-1)
th = 2
H, W = img.shape[:2]
lowers = [y for y in range(H - 1) if hist[y] > th and hist[y + 1] <= th]

for y in lowers:
   img=cv2.line(img, (0,y), (W, y), (0,255,0), 1)
cv2.imwrite("demo_img.png", img)

For more sample documents

感谢帮助

Answer 1

如果您尝试使用 OCR 检测图像中的文本，对图像进行预处理以去除噪声、过滤掉不需要的对象并在这种情况下去除网格线非常重要。这里有一个简单的方法，获取二值图像，修复水平网格线进行检测，去除水平表格线，去除垂直表格线，然后使用Pytesseract进行OCR。这是您的一些图像的结果。

之前 ->之后和OCR结果

ASSETS

Checking & Savings ACCOUNT BEGINNING BALANCE — ENDING BALANCE
THIS PERIOD THIS PERIOD

Chase Total Checking 000000629831256 $174.02 $5.28

Chase Savings 000003313056365 25.00 0.72

Total $199.02 $6.00

TOTAL ASSETS $199.02 $6.00

HIBACHI GRILL ASIAN ELK GROVE VIL IL 10/23 (...4719) Card -$34.00 $1,531.31
Oct 23,2018 SAMSCLUB #6464 DES PLAINES IL 10/23 (...4719) Card -$26.07 $1,565.31
Oct 15,2018 SAMS CLUB SAM'S Club DES PLAINES IL 10/14 (...4719) Card -$36.07 $1,591.38
Premier *Bankcard LLC 605-3573440 SD 10/14 (...4719) | Card -$70.00 $1,627.45
CANOPY-BUFFETT DES PLAINES IL 10/14 (...4719) Card -$33.24 $1,697.45
COMCAST CHICAGO CS 1X 800-266-2278 IL 10/14 (...4719) Card -$275.45 $1,730.69
ATM CHECK DEPOSIT 10/13 1590 LEE ST DES PLAINES IL ATM deposit $803.92 $2,006.14
Oct 12,2018 VILLAGE OF ROSEM DIRECT DEP PPD ID: 9111111103 ACH credit $604.60 $1,202.22
Oct 11,2018 DEPOSIT ID NUMBER 706989 Deposit $541.56 $597.62
Oct 10, 2018 AURORA UNIVERSITY 800-742-5281 IL 10/09 (...4719) Card -$450.00 $56.06
Oct 9, 2018 ATM CASH DEPOSIT 10/08 1590 LEE ST DES PLAINES IL ATM transaction $400.00 $506.06
Oct 2, 2018 Convenience Fee WEB PAY Vaughn WEB ID: 2364303385 ACH debit -$1.50 $106.06
Vaughn WEB PAY Vaughn WEB ID: 1364303385 ACH debit -$1,118.10 $107.56
AURORA UNIVERSITY 800-742-5281 IL 10/01 (...4719) Card -$550.00 $1,225.66
Oct 1, 2018 SPEEDWAY 04250 DES DES PLAINES IL 09/29 (...4719) Card -$35.08 $1,775.66
ATM CASH DEPOSIT 10/01 1590 LEE ST DES PLAINES IL ATM transaction $380.00 $1,810.74
Sep 28, 2018 VILLAGE OF ROSEM DIRECT DEP PPD ID: 9111111103 ACH credit $561.62 $1,430.74
ATM CHECK DEPOSIT 09/28 1590 LEE ST DES PLAINES IL ATM deposit $785.45 $869.12
Sep 24,2018 SPEEDWAY 04250 DES DES PLAINES IL 09/21 (...4719) Card -$14.93 $83.67

DATE DESCRIPTION AMOUNT
06/27 Card Purchase 06/26 Culinart 119 At Con Long Island C NY Card 0018 $3.43
06/27 Card Purchase 06/27 Tst* Slice - Long |s Long Island C NY Card 0018 7.50
06/28 Card Purchase 06/27 Paypal *Netflix.Com 402-935-7733 CA Card 0018 13.99
06/28 Card Purchase 06/27 Culinart 119 At Con Long Island C NY Card 0018 6.26
06/29 Card Purchase 06/27 Butcher Bar Astoria NY Card 0018 10.00
| 06/29 Card Purchase 06/28 Culinart 119 At Con Long Island C NY Card 0018 5.93
| 06/29 Card Purchase 06/28 Boston Market 1669 Woodside NY Card 0018 11.90
| 06/29 Card Purchase 06/29 Caridad& Louis Rest Bronx NY Card 0018 31.79
| 06/29 Card Purchase With Pin 06/29 Superior Deli Long Island C NY Card 0018 8.00
07/02 Card Purchase 06/29 Culinart 119 At Con Long Island C NY Card 0018 2.88
07/02 Card Purchase 06/29 Bel Aire Diner Astoria NY Card 0018 18.53
07/02 Card Purchase 06/30 Gulf Oil 92039469 Bronx NY Card 0018 30.00
07/02 Card Purchase 06/30 Front Street Pizza Brooklyn NY Card 0018 6.26
07/02 Card Purchase 06/30 Gulf Oil 92039469 Bronx NY Card 0018 63.22
07/02 Card Purchase With Pin 07/01 Four Brothers Discount Bronx NY Card 0018 19.54
07/02 Card Purchase 07/01 Medonald's F2658 Bronx NY Card 0018 44.98
07/03 Recurring Card Purchase 07/03 Spotify USA 646-8375380 NY Card 0018 9.99
07/05 Card Purchase 07/02 Eastside Mkt Corp New York NC Card 0018 9.26
07/05 Card Purchase 07/03 Salvo's Pizza Bar New York NY Card 0018 15.00
07/05 Card Purchase 07/03 Eastside Mkt Corp New York NC Card 0018 8.79
07/05 Card Purchase 07/04 3340 Dominos Pizza 734-930-3030 NY Card 0018 37.58
07/09 Card Purchase 07/05 Eastside Mkt Corp New York NC Card 0018 9.78
07/09 Card Purchase 07/06 Salvo's Pizza Bar New York NY Card 0018 8.68
07/09 Card Purchase 07/07 Medonald's F2658 Bronx NY Card 0018 18.05
| 07/09 Card Purchase 07/08 lhop 4634 Bronx NY Card 0018 34.70
07/09 Recurring Card Purchase 07/06 Ibi*Shoedazzle 888-5081888 CA Card 0018 39.95
07/10 Card Purchase 07/09 Culinart 119 At Con Long Island C NY Card 0018 2.88
07/10 Card Purchase 07/09 Paypal *Bioceutical 402-935-7733 CA Card 0018 65.75
107/10 Card Purchase 07/09 Mamas Fmnanadas Astoria NY Card 0018 1178
07/10 Card Purchase With Pin 07/10 Community Green Market Bronx NY Card 0018 55.98

代码

import cv2
import pytesseract
import numpy as np

pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

# Load image, grayscale, Otsu's threshold
image = cv2.imread('7.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

# Repair horizontal table lines 
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,1))
thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=1)

# Remove horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (55,2))
detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(image, [c], -1, (255,255,255), 9)

# Remove vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2,55))
detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(image, [c], -1, (255,255,255), 9)

# Perform OCR
data = pytesseract.image_to_string(image, lang='eng',config='--psm 6')
print(data)

cv2.imshow('image', image)
cv2.imwrite('image7.png', image)
cv2.waitKey()

注:网格移除步骤改编自 Removing Horizontal Lines in image (OpenCV, Python, Matplotlib) .根据图像，内核的大小会发生变化。例如，要检测更长的行，我们可以使用 (50,1)内核代替。如果我们想要更粗的线条，我们可以增加第二个参数说 (50,2) .