c# - 来自 PhotoCaptureFrame 的 IMFMediaBuffer

Question

我有一个在 Unity 5.2 上使用 Hololens 可定位相机的计算机视觉插件。我使用 UnityEngine.XR.WSA.WebCam.PhotoCaptureFrame 类，在最高分辨率下启用照片模式，即 2048x1156，采用 NV12 颜色格式，这是相机原生格式。

PhotoCaptureFrame 类:https://docs.unity3d.com/2017.2/Documentation/ScriptReference/XR.WSA.WebCam.PhotoCaptureFrame.html

目前，我这样做是为了让我的插件处理任何一张照片:

PhotoCaptureFrame photo; // created correctly through Unity API

List<byte> photoBuffer = new List<byte>();
photo.CopyRawImageDataIntoBuffer(photoBuffer);

// marshal photoBuffer to plugin (and it works fine for the computer vision part)

但是，它每次都会复制原始缓冲区。我想直接从我的插件中使用原始缓冲区。所以我尝试了这个:

IntPtr rawBufferPtr = photo.GetUnsafePointerToBuffer(); // COM pointer to IMFMediaBuffer is incremented
// send rawBufferPtr to plugin (native function described below)
Marshal.Release(rawBufferPtr);

IMFMediaBuffer 接口(interface):https://msdn.microsoft.com/en-us/library/windows/desktop/ms696261(v=vs.85).aspx

在我的 C++ 插件中:

#include <Mfidl.h> // for IMFMediaBuffer support

void process_photo(void *photo_buffer_wrapper, int photo_width, int photo_height)
{
    // photo_buffer_wrapper = rawBufferPtr in managed environment

    IMFMediaBuffer *media_buffer = reinterpret_cast<IMFMediaBuffer *>(photo_buffer_wrapper);
    BYTE *photo_buffer = NULL;
    HRESULT result = media_buffer->Lock(&photo_buffer, NULL, NULL);
    if (SUCCEEDED(result))
    {
        // do image processing stuff here (with OpenCV) using photo_buffer
        media_buffer->Unlock;
    }
}

我觉得还不错。它也编译得很好。但在运行时，我遇到了访问冲突，应用程序在 Hololens 上崩溃了。

异常代码:0xC0000005

异常信息:线程试图读取或写入它没有适当访问权限的虚拟地址。

有人看到问题了吗？与我将 IMFMediaBuffer 对象从托管环境传递到非托管环境的方式有关吗？

非常感谢!

Answer 1

我会回答我自己的问题。

photo_buffer_wrapper 不是我想的指向 IMFMediaBuffer 的指针，而是指向 IUnknown 的指针。这是按预期工作的修改后的 native 函数:

// UNMANAGED ENVIRONMENT
#include <Mfidl.h> // for IMFMediaBuffer support + other COM stuff

void process_photo(void *photo_buffer_unknown, int photo_width, int photo_height)
{
    // photo_buffer_unknown = photoCaptureFrame.GetUnsafePointerToBuffer() in managed environment which is an IntPtr

    IMFMediaBuffer *media_buffer;
    if (SUCCEEDED(reinterpret_cast<IUnknown *>(photo_buffer_unknown)->QueryInterface<IMFMediaBuffer>(&media_buffer)))
    {
        BYTE* photo_buffer = NULL;
        if (SUCCEEDED(media_buffer->Lock(&photo_buffer, NULL, NULL)))
        {
            // process photo_buffer with OpenCV (wrapped in a cv::Mat)

            media_buffer->Unlock();
            media_buffer->Release(); // QueryInterface on IUnknown has incremented reference count by one
        }
    }
}

注意:从 photoCaptureFrame.GetUnsafePointerToBuffer() 返回的指针仍然需要像我的问题一样在托管环境中释放:

// MANAGED ENVIRONMENT
IntPtr mediaBufferUnknownPtr = photoCaptureFrame.GetUnsafePointerToBuffer();
// send mediaBufferUnknownPtr to native function through extern DLL call
Marshal.Release(mediaBufferUnknownPtr)