python - 如何将 OpenNI VideoFrame 转换为 OpenCV Mat 数据结构？

Question

我正在寻找关于 openni VideoFrame 对象格式的一般描述/规范，我试图将其转换为 opencv Mat。

是否有一些官方网站/文档可用？我在 ros.org 上找到了以下内容其中描述了 VideoFrameRef 和 OniFrame 类。但首先这似乎与 libfreenect 有关，其次我没有找到任何关于框架结构的具体信息(如尺寸、 channel 等)。

因此，我们将不胜感激任何链接或描述!

下面的代码是我正在试验的一个片段。 while 循环 中的转换部分只产生黑色视频流。

import sys
import numpy as np
import cv2
from openni import openni2
from openni import _openni2 as c_api

openni2.initialize("./Redist")

device = openni2.Device.open_any()

ir_stream = device.create_ir_stream()
ir_stream.set_video_mode(c_api.OniVideoMode(pixelFormat = c_api.OniPixelFormat.ONI_PIXEL_FORMAT_GRAY16, resolutionX = 320, resolutionY = 240, fps = 30))
ir_stream.start()

while(True):
    frame = ir_stream.read_frame()
    frame_data = frame.get_buffer_as_uint16()
    img = np.frombuffer(frame_data, dtype=np.uint16)
    img.shape = (1, 240, 320)
    img = np.concatenate((img, img, img), axis=0)
    img = np.swapaxes(img, 0, 2)
    img = np.swapaxes(img, 0, 1)
    cv2.imshow("image", img)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

ir_stream.stop()
openni2.unload()

Answer 1

我想回答我自己的问题，因为我相信其他人可以从这个工作示例中获益。

cv2.imshow() 理解 uint8 (0..255)、uint32(0..2**32-1) 和 float (0..1) 所以我们必须转换使用 img.astype(np.float)/1024

的数组

import numpy as np
import cv2
from openni import openni2
from openni import _openni2 as c_api

openni2.initialize("./Redist")
device = openni2.Device.open_any()

ir_stream = device.create_ir_stream()
ir_stream.set_video_mode(c_api.OniVideoMode(pixelFormat = c_api.OniPixelFormat.ONI_PIXEL_FORMAT_GRAY16, resolutionX = 320, resolutionY = 240, fps = 30))
ir_stream.start()

while(True):
    frame = ir_stream.read_frame()

    # returns a 1-dim c_ushort_array of len: 76'800 = 320x240 each pixel having an uint16 value 0..65535
    frame_data = frame.get_buffer_as_uint16()

    # converting to numpy.ndarray which is still 1-dimension only
    img = np.frombuffer(frame_data, dtype=np.uint16)

    # convert to 3-dimensional array
    img.shape = (240, 320)

    # normalize values 
    img = img.astype(np.float) / 1024

    # Display image
    cv2.imshow("image", img)

    # Wait for input
    if cv2.waitKey(1) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

ir_stream.stop()
openni2.unload()

相关的 stackoverflow 帖子:

• Getting data from ctypes array into numpy
• OpenNI and OpenCV: cv2.imshow() crashes with error: (-215:Assertion failed) dst.data == (uchar*)dst_ptr in function 'cvShowImage'