python-3.x - 修复了从PIL图像到OpenCV Mat的低效图像转换

Question

我在大小为800x600的实时屏幕截图上运行神经网络。由于我的速度仅为3fps，因此我进行了一些故障排除，发现每个步骤大约花费了多少时间:

屏幕截图:12ms

图像处理:280ms

对象检测和盒子可视化:16ms

显示图像:0.5ms

我正在使用mss来截取屏幕截图( documentation)。

这是没有对象检测部分的代码:

import numpy as np
import cv2
from PIL import Image
import mss
monitor = {"top": 40, "left": 0, "width": 800, "height": 600}

with mss.mss() as sct:
    while True:

        # # Screenshot:
        image = sct.grab(monitor)

        # # Image processing:
        image = Image.frombytes("RGB", image.size, image.bgra, "raw", "RGBX")
        (im_width, im_height) = image.size
        image_np = np.array(image.getdata()).reshape((im_height, im_width, 3)).astype(np.uint8)

        # # Object detection and box visualisation:
        # ...

        # # Displaying image:
        cv2.imshow("Object Detection", image_np)

关于如何使它更快的任何想法？

Answer 1

问题是您的方法从BGRA图像格式开始。那是很多数据，可能没有必要。可能会有更有效的方法来截取屏幕截图并将其转换为OpenCV图像。在我的慢速计算机上，大约需要 56ms 的Here's an approach:

import ctypes
import datetime
import cv2
import numpy as np

from PIL import ImageGrab


# workaround to allow ImageGrab to capture the whole screen
user32 = ctypes.windll.user32
user32.SetProcessDPIAware()

# measure running time
start_time = datetime.datetime.now()

# take a full screenshot of the desktop
image = np.array(ImageGrab.grab( bbox= (40, 0, 800, 600) ))

# convert from RGB to BGR order so that colors are displayed correctly
mat = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)

# compute elapsed time
delta = datetime.datetime.now() - start_time
elapsed_time_ms = int(delta.total_seconds() * 1000)
print('* Elapsed time:', elapsed_time_ms, 'ms')

cv2.imshow('mat', mat)
cv2.waitKey()