gpt4 book ai didi

python - 捕获屏幕并找到引用图像

转载 作者:行者123 更新时间:2023-11-28 18:32:57 24 4
gpt4 key购买 nike

我正在尝试在 python 中执行以下操作:

  1. 截屏
  2. 如果屏幕截图包含给定的引用图像(可以是 jpg 或 pgn),获取该图像在屏幕上的坐标

更多信息:

  • 引用图片不会很大(5x5 像素就足够了)
  • 它应该尽可能快,因为它应该不断扫描屏幕
  • 如果可能:在 Windows 和 Linux 上工作

什么是最好的方法,我可以在 python 中实现它?

编辑:感谢 Leo Antunes,我使以下解决方案有效:

def bitmap2brg(bmp):
w = bmp.width;
h = bmp.height;
a = np.empty((h, w, 3), dtype=np.uint8);

for r in xrange(h):
for c in xrange(w):
v = bmp.get_color(c, r);
a[r, c, 2] = (v >> 16) & 0xFF;
a[r, c, 1] = (v >> 8) & 0xFF;
a[r, c, 0] = v & 0xFF;
return a;

def grabScreen():
THRESHOLD = 1

# reference image
needle = cv2.imread('img_top_left.png')
needle_height, needle_width, needle_channels = needle.shape

# Grabbing with autopy
screen = autopy.bitmap.capture_screen()
haystack = bitmap2brg(screen)

# work through the frame looking for matches above a certain THRESHOLD
# and mark them with a green circle
matches = 0
for pt in np.transpose(np.where(cv2.matchTemplate(haystack, needle, cv2.TM_CCOEFF_NORMED) >= THRESHOLD)):
cv2.circle(haystack, (pt[1] + needle_width/2, pt[0] + needle_height/2), 10, (0,255,0))
matches += 1

# display the altered frame
print "Number of matches: {}".format(matches)
cv2.imshow('matches', haystack)
if cv2.waitKey(0) & 0xFF == ord('q'):
cv2.destroyAllWindows()

最佳答案

更新:有了 OP 自己的一个伟大发现 (autopy),“正确”的解决方案变得非常简单:

import autopy

needle = autopy.bitmap.Bitmap.open('needle.png')

while True:
haystack = autopy.bitmap.capture_screen()
found = haystack.find_every_bitmap(needle)
print(found)

下面的原始次优答案:


您可以使用 OpenCV 相对轻松地对视频执行此类操作.唯一的问题是以便携方式捕获屏幕。我能找到的唯一理论上可移植的库是 this one ,但我无法让它工作。

我没有使用真正可移植的库,而是围绕 ffmpeg 破解了一个适用于 linux 的解决方案,并且 theoretically could be made to work on Windows and OSX as well :

import cv2
import numpy as np
import subprocess

THRESHOLD = 0.7

# your reference image
needle = cv2.imread('/path/to/some/needle.jpg', cv2.CV_LOAD_IMAGE_GRAYSCALE)
needle_height, needle_width = needle.shape

width, height = (800, 600) # you could of course detect this

command = [ '/usr/bin/ffmpeg',
'-f', 'x11grab',
'-i', ':0.0+0,0',
'-r', '3', # lower frame-rate for testing
'-s', '%dx%d' % (width, height),
'-f', 'rawvideo',
'-vcodec', 'rawvideo',
'-pix_fmt', 'bgr24',
'-']

pipe = subprocess.Popen(command, stdout = subprocess.PIPE, bufsize=10**8)

while(True):
# get a frame
raw = pipe.stdout.read(width*height*3)
# format it into a matrix which can be worked on by openCV
original = np.fromstring(raw, dtype='uint8').reshape((height, width, 3))
pipe.stdout.flush()

# transform it to grayscale for the matching
haystack = cv2.cvtColor(original, cv2.COLOR_BGR2GRAY)

# work through the frame looking for matches above a certain THRESHOLD
# and mark them with a green circle
for pt in np.transpose(np.where(cv2.matchTemplate(haystack, needle, cv2.TM_CCOEFF_NORMED) >= THRESHOLD)):
cv2.circle(original, (pt[1] + needle_width/2, pt[0] + needle_height/2), 10, (0,255,0))

# display the altered frame
cv2.imshow('matches', original)

if cv2.waitKey(1) & 0xFF == ord('q'):
break

cap.release()
cv2.destroyAllWindows()

请注意,从管道读取期间使用的魔法“3”来自 ffmpeg 的输出像素格式:bgr24

关于python - 捕获屏幕并找到引用图像,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/35205565/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com