c# - .NET 和 Direct2D 快速绘制崩溃

Question

通用

我正在开发可用于使用 Direct2D 进行多用途图形渲染的 .NET 用户控件。此控件基于带有 ID2D1BitmapRenderTarget 后台缓冲区/双缓冲区的 ID2D1HwndRenderTarget 接口(interface)。

背景

我已经使用 C++/CLI 编写了一个包装器，它公开了一组用于 Direct2D 的托管包装器对象，并且我为 Direct2D 位图设计了一个单例资源管理器，代码可以将二进制数据发送到该位图，并获取资源 key 返回，然后代码可以将消息传递给用户控件以呈现位图。

到目前为止的使用情况

我已经能够加载位图、JPEG、自定义图像格式等并将它们发送到 Direct2D 渲染控件，一切都没有问题。我编写了一个自定义视频解码器，可以从 15 帧/秒的视频中加载帧(在后台线程中解码)并从 Win32 多媒体计时器引发的事件中渲染它们，没问题。

问题

我的问题是，当我尝试从单一多媒体格式扩展为更灵活的格式(特别是连接到 LibAV/ffmpeg)时，Direct2D 渲染控件开始使显示驱动程序崩溃。当按顺序渲染帧时(使用按钮而不是计时器来渲染下一帧)，不会发生。它也不会立即发生。当我在 Win32 计时器回调内部阻塞时，它也不会发生，但当我使用 Mutex 引发线程以释放计时器回调并让临时线程指示控件呈现时，它会发生。

症状

如果我启动一个计时器并附加到将导致下一次渲染的经过的事件方法，它通常会正常播放大约 2-20 秒。使用较小的视频，我可以在问题开始之前获得更长的播放时间。如果我播放 1080p 视频，我通常可以在 5 秒内无竞争地获得崩溃。播放将开始正常，在 catch 之前可能会在这里或那里打嗝。

• 最终，我会开始看到闪烁，就好像渲染了背景颜色但没有渲染框架。其他帧可能会或可能不会稍后呈现，但这是非常短暂的。
• 闪烁后，如果我足够快地停止计时器，就不会发生崩溃。
• 如果我没有及时停下来:
  • 如果幸运的话(可能有 10% 的几率)，帧将停止渲染，卡在一个帧上，直到我调整控件的大小，此时整个控件只绘制黑色。 (我读到这可能表示 DirectX 渲染的设备上下文丢失，但没有看到更多。)
  • 如果我运气不好，显示驱动程序会崩溃，Windows 会恢复并重新启动驱动程序，之后 EndDraw 将最终告诉我发生了错误并返回 D2DERR_RECREATE_TARGET。
  • 如果我真的不走运，显示屏将开始看起来像一个万花筒，我将不得不关闭我的机器，我只是浪费了 5 分钟的时间来启动、登录、加载 Visual Studio 和我的解决方案、加载视频并丢失了我所有的调试数据。

我想认为我遗漏了某种竞争条件，但每次我运行渲染代码时，它似乎应该被正确锁定。

渲染控件的代码

using System;
using System.Drawing;
using System.Windows.Forms;

using Direct2D = Bardez.Projects.DirectX.Direct2D;
using ExternalPixelEnums = Bardez.Projects.FileFormats.MediaBase.Video.Pixels.Enums;
using Bardez.Projects.DirectX.Direct2D;
using Bardez.Projects.FileFormats.MediaBase.Video;
using Bardez.Projects.FileFormats.MediaBase.Video.Enums;
using Bardez.Projects.Win32;

namespace Bardez.Projects.Output.Visual
{
    /// <summary>Represents a rendering target for Direct2D.</summary>
    /// <remarks>
    ///     To use this control, an external component will need to supply bitmap data. This does not need to be a GDI+ Bitmap class.
    ///     However, the container for this control will need to push data into this control.
    ///     So, in the case of a movie player, we'd see the following model:
    ///         * decompress a frame.
    ///         * push frame to control
    ///         * invoke Invalidate
    ///             * control will render the bitmap
    ///         * sleep just a little bit
    ///         * go back to first step
    /// </remarks>
    public class Direct2dRenderControl : VisualRenderControl
    {
        /*
        *   Locking orientation:
        *       There are two rendering targets: a GDI display and a bitmap back buffer.
        *       There are 5 'real' locking operations:
        *           Rendering to the GDI display (OnPaint)
        *           Rendering to the back buffer (DrawBitmapToBuffer, DiscardCurrentBuffer)
        *           Setting the current displayed frame (SetRenderFrame)
        *           Resource Freeing (FreeFrameResource)
        *           Resizing (OnResize)
        *
        *       Briefly, the overarching effects of these five are:
        *           Rendering
        *               Utilizes the buffer and the display
        *           Back Buffer
        *               Utilizes the buffer and if mid-render could affect this display
        *           Set Frame
        *               Sets the buffer's displayed image
        *           Resource Freeing
        *               Affects the buffer if a resource is refernced
        *           Resizing
        *               Resizes the render control and the bitmap, uses the frame set by set
        *
        *       Locking plan:
        *           Resize should block set, free and back buffer
        *           Render should block back buffer, control
        *           Set frame should block resize and back buffer
        *           Free should block back buffer,
        *           Back buffer should block rendering, setting, resizing, freeing
        *
        *       Basically, lock everything at the process level, and not at the access level.
        */

        #region Fields
        /// <summary>Represents a Win32 HWND render target for Direct2D</summary>
        private ControlRenderTarget ctrlRenderTarget;

        /// <summary>Represents a drawing buffer</summary>
        private BitmapRenderTarget bmpRenderTarget;

        /// <summary>Represents a drawing buffer, used solely to create bitmaps</summary>
        private BitmapRenderTarget resourceBmpRenderTarget;

        /// <summary>Represents a buffer drawing command lock</summary>
        private Object controlBufferLock;

        /// <summary>Represents a paint/render drawing command lock</summary>
        private Object controlPaintRenderLock;

        /// <summary>Locking object reference for resource management (memory bitmaps, etc.)</summary>
        private Object resourceLock;

        /// <summary>Represents the key to accessing the currently set key</summary>
        protected Int32 currentFrameKey;
        #endregion


        #region Properties
        /// <summary>Indicates whether there is a frame set for this control</summary>
        protected Boolean HasFrameSet
        {
            get { return currentFrameKey > -1; }
        }

        /// <summary>Exposes a wrapper for the bitmap render target</summary>
        protected BitmapRenderTarget BmpRenderTarget
        {
            get { return this.bmpRenderTarget; }
            set
            {
                lock (this.controlBufferLock)
                {
                    if (this.bmpRenderTarget != null)
                        this.bmpRenderTarget.Dispose();

                    this.bmpRenderTarget = value;
                }
            }
        }

        /// <summary>Exposes a wrapper for the bitmap render target</summary>
        protected BitmapRenderTarget ResourceBmpRenderTarget
        {
            get { return this.resourceBmpRenderTarget; }
            set
            {
                lock (this.resourceLock)
                {
                    if (this.resourceBmpRenderTarget != null)
                        this.resourceBmpRenderTarget.Dispose();

                    this.resourceBmpRenderTarget = value;
                }
            }
        }

        /// <summary>Represents a Win32 HWND render target for Direct2D</summary>
        protected ControlRenderTarget CtrlRenderTarget
        {
            get { return this.ctrlRenderTarget; }
            set
            {
                lock (this.controlPaintRenderLock)
                {
                    if (this.ctrlRenderTarget != null)
                        this.ctrlRenderTarget.Dispose();

                    this.ctrlRenderTarget = value;
                }
            }
        }
        #endregion


        #region Construction
        /// <summary>Default constructor</summary>
        public Direct2dRenderControl() : base()
        {
            this.controlBufferLock = new Object();
            this.controlPaintRenderLock = new Object();
            this.resourceLock = new Object();

            this.currentFrameKey = -1;
            this.InitializeControlDirect2D();
        }

        /// <summary>Initializes the Direct2D</summary>
        protected void InitializeControlDirect2D()
        {
            //disable Windows background draw
            this.SetStyle(ControlStyles.UserPaint | ControlStyles.AllPaintingInWmPaint, true);

            // Build options on the Control render target
            PixelFormat format = new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.Unknown);  //32-bit color, pure alpha
            DpiResolution res = Direct2dResourceManager.Instance.Factory.GetDesktopDpi();
            RenderTargetProperties rtProp = new RenderTargetProperties(RenderTargetType.Default, format, res, RenderTargetUsage.GdiCompatible, DirectXVersion.DirectX9);

            //Build out control render target properties
            HwndRenderTargetProperties hwndProp = new HwndRenderTargetProperties(this.Handle, new SizeU(this.Size), PresentationOptions.RetainContents);

            lock (this.controlPaintRenderLock)
            {
                // populate the Control rendering target
                ResultCode result = Direct2dResourceManager.Instance.Factory.CreateHwndRenderTarget(rtProp, hwndProp, out this.ctrlRenderTarget);

                lock (this.controlBufferLock)
                {
                    // create a bitmap rendering targets
                    this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.bmpRenderTarget);

                    lock (this.resourceLock)
                        this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.resourceBmpRenderTarget);
                }
            }
        }
        #endregion


        #region Destruction
        /// <summary>Disposal code; releases unmanaged resources</summary>
        /// <param name="disposing">True indicates to dispose managed resources</param>
        protected override void Dispose(Boolean disposing)
        {
            this.ResourceBmpRenderTarget = null;    //property disposes
            this.BmpRenderTarget = null;            //property disposes
            this.CtrlRenderTarget = null;           //property disposes
            base.Dispose(disposing);
        }

        /// <summary>Disposal</summary>
        ~Direct2dRenderControl()
        {
            this.Dispose();
        }
        #endregion


        #region Event Raising
        /// <summary>Draws the output, then raises the paint event</summary>
        /// <param name="e">Painting Event arguments</param>
        protected override void OnPaint(PaintEventArgs e)
        {
            lock (this.controlPaintRenderLock)
            {
                lock (this.controlBufferLock)
                {
                    this.SuspendLayout();

                    this.OnPaintBackground(e);

                    Direct2D.Bitmap bmp;
                    ResultCode result;

                    result = this.BmpRenderTarget.GetBitmap(out bmp);

                    Direct2D.RectangleF rect = new Direct2D.RectangleF(e.ClipRectangle);

                    this.CtrlRenderTarget.BeginDraw();

                    this.CtrlRenderTarget.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);

                    result = this.CtrlRenderTarget.EndDraw();

                    bmp.Dispose();

                    if (result != ResultCode.Success_OK)
                        throw new ApplicationException(String.Format("Error encountered during draw: '{0}'", result.ToString()));

                    base.OnPaint(e);

                    this.ResumeLayout();
                }
            }
        }

        /// <summary>Overides the resize method</summary>
        /// <param name="e">Parameters for the resize event</param>
        protected override void OnResize(EventArgs e)
        {
            lock (this.controlPaintRenderLock)
            {
                lock (this.controlBufferLock)
                {
                    //Null check since resize fires before it is constructed, with the size event during parent's InitializeComponent
                    if (this.CtrlRenderTarget != null)
                    {
                        this.SuspendLayout();

                        //resize the existing control rendering target
                        this.CtrlRenderTarget.Resize(new SizeU(this.Size));

                        //I also need to resize the buffer, but can't. Instead, create a new one, then copy the existing one. Kind of lame.
                        this.ResizeBitmapRenderTarget();

                        base.OnResize(e);

                        //cause another draw
                        this.Invalidate(new Rectangle(new Point(0, 0), this.Size));

                        this.ResumeLayout();
                    }
                }
            }
        }

        /// <summary>Overridden Paint background method</summary>
        /// <param name="e">Paint event arguments</param>
        /// <remarks>
        ///     Made empty to avoid GDI and Direct2D writing to the same control; when moving the control around
        ///     or rendering at high speeds, the dreaded WinForms flicker was introduced.
        ///     The background painting can be found in the <see cref="FillBufferRenderTarget"/> method,
        ///     which fills the bitmap back buffer with the control's background color.
        /// </remarks>
        protected override void OnPaintBackground(PaintEventArgs e) {  }
        #endregion


        #region Drawing
        /// <summary>Resizes the bitmap rendering target buffer</summary>
        /// <remarks>Does not lock the GDI render target. The OnResize or other callers lock instead.</remarks>
        protected void ResizeBitmapRenderTarget()
        {
            lock (this.controlPaintRenderLock)
            {
                lock (this.controlBufferLock)
                {
                    using (BitmapRenderTarget bmpCurr = this.BmpRenderTarget)
                    {
                        BitmapRenderTarget bmpNew;

                        ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);

                        this.DuplicateDoubleBufferContents(bmpNew);

                        //Property disposes and locks
                        this.BmpRenderTarget = bmpNew;
                    }
                }
            }
        }

        /// <summary>Draws a Bitmap to the render buffer</summary>
        /// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
        protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp, Point2dF origin)
        {
            lock (this.controlBufferLock)
            {
                lock (this.resourceLock)
                {
                    Direct2D.SizeF bmpSize = bmp.GetSize();

                    Single width = bmpSize.Width > this.BmpRenderTarget.Size.Width ? this.BmpRenderTarget.Size.Width : bmpSize.Width;
                    Single height = bmpSize.Height > this.BmpRenderTarget.Size.Height ? this.BmpRenderTarget.Size.Height : bmpSize.Height;

                    Direct2D.RectangleF destRect = new Direct2D.RectangleF(origin.X, origin.X + width, origin.Y, origin.Y + height);
                    Direct2D.RectangleF srcRect = new Direct2D.RectangleF(0.0F, width, 0.0F, height);

                    this.BmpRenderTarget.BeginDraw();

                    this.FillBufferRenderTarget(this.BmpRenderTarget);

                    // do the actual draw
                    this.BmpRenderTarget.DrawBitmap(bmp, destRect, 1.0F, BitmapInterpolationMode.Linear, srcRect);

                    //tell Direct2D that a paint operation is ending
                    ResultCode result = this.BmpRenderTarget.EndDraw();
                }
            }
        }

        /// <summary>Draws a Bitmap to the render buffer</summary>
        /// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
        protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp)
        {
            this.DrawBitmapToBuffer(bmp, new Point2dF(0.0F, 0.0F));
        }

        /// <summary>Duplicates the bitmap behind the existing rendering target, and drawing it to a new one, discarding the current and setting the new.</summary>
        /// <remarks>Does not lock any references, as the outside method locks</remarks>
        protected void DuplicateDoubleBufferContents(BitmapRenderTarget bmpNew)
        {
            Direct2D.Bitmap bmp = null;
            ResultCode result = ResultCode.Success_OK;

            if (this.HasFrameSet)
                bmp = Direct2dResourceManager.Instance.GetBitmapResource(this.currentFrameKey);
            else
                result = this.BmpRenderTarget.GetBitmap(out bmp);

            bmpNew.BeginDraw();

            this.FillBufferRenderTarget(bmpNew);

            //calculate the size to copy
            Direct2D.SizeF bmpSize = bmp.GetSize();

            Single width = bmpSize.Width > this.CtrlRenderTarget.Size.Width ? this.CtrlRenderTarget.Size.Width : bmpSize.Width;
            Single height = bmpSize.Height > this.CtrlRenderTarget.Size.Height ? this.CtrlRenderTarget.Size.Height : bmpSize.Height;

            //Determine the copy rectangle
            Direct2D.RectangleF rect = new Direct2D.RectangleF(0, width, 0, height);

            //Copy
            bmpNew.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);

            //conditionally disose the bitmap, don't if it is in the manager
            if (!this.HasFrameSet)
                bmp.Dispose();

            result = bmpNew.EndDraw();
        }

        /// <summary>Discards the current buffer and replaces it with a blank one.</summary>
        protected void DiscardCurrentBuffer()
        {
            lock (this.controlBufferLock)
            {
                BitmapRenderTarget bmpNew;

                // create a bitmap rendering target
                ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);

                bmpNew.BeginDraw();
                this.FillBufferRenderTarget(bmpNew);
                result = bmpNew.EndDraw();

                //property locks, so no lock here
                this.BmpRenderTarget = bmpNew;  //replace the old buffer
            }
        }

        /// <summary>Fills the buffer render target with the background color</summary>
        /// <param name="renderTarget">Bitmap render target to fill</param>
        protected void FillBufferRenderTarget(BitmapRenderTarget renderTarget)
        {
            SolidColorBrush brush = null;

            ResultCode result = renderTarget.CreateSolidColorBrush(new ColorF(this.BackColor), out brush);

            renderTarget.FillRectangle(new Direct2D.RectangleF(new Rectangle(new Point(0, 0), this.Size)), brush);
            brush.Dispose();
        }
        #endregion


        #region Frame Resources
        /// <summary>Posts a Frame resource to the resource manager and returns a unique key to access it.</summary>
        /// <param name="resource">Frame to be posted.</param>
        /// <returns>A unique Int32 key</returns>
        public override Int32 AddFrameResource(Frame resource)
        {
            lock (this.resourceLock)
            {
                //create the bitmap
                BitmapProperties properties = new BitmapProperties(new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.PreMultiplied), Direct2dResourceManager.Instance.Factory.GetDesktopDpi());
                SizeU dimensions = new SizeU(Convert.ToUInt32(resource.Pixels.Metadata.Width), Convert.ToUInt32(resource.Pixels.Metadata.Height));

                Direct2D.Bitmap bmp = null;
                ResultCode result = this.ResourceBmpRenderTarget.CreateBitmap(dimensions, properties, out bmp);

                Byte[] data = resource.Pixels.GetPixelData(ExternalPixelEnums.PixelFormat.RGBA_B8G8R8A8, ScanLineOrder.TopDown, 0, 0);
                result = bmp.CopyFromMemory(new RectangleU(dimensions), data, Convert.ToUInt32(resource.Pixels.Metadata.Width * 4));

                return Direct2dResourceManager.Instance.AddFrameResource(bmp);
            }
        }

        /// <summary>Frees a Bitmap resource in the resource manager and Disposes of it.</summary>
        /// <param name="frameKey">Direct2D Bitmap key to be Disposed.</param>
        public override void FreeFrameResource(Int32 frameKey)
        {
            lock (this.resourceLock)
            {
                if (frameKey > -1)
                    Direct2dResourceManager.Instance.FreeFrameResource(frameKey);
            }
        }
        #endregion


        #region Frame Setting
        /// <summary>Sets the frame to be rendered to the User Control</summary>
        /// <param name="key">Frame key to set as current image</param>
        public override void SetRenderFrame(Int32 key)
        {
            this.SetRenderFrame(key, 0, 0);
        }

        /// <summary>Sets the frame to be rendered to the User Control</summary>
        /// <param name="key">Frame key to set as current image</param>
        /// <param name="originX">X coordinate to start drawing from</param>
        /// <param name="originY">Y coordinate to start drawing from</param>
        public override void SetRenderFrame(Int32 key, Int64 originX, Int64 originY)
        {
            lock (this.controlBufferLock)
            {
                if (key > -1)
                    this.DrawBitmapToBuffer(Direct2dResourceManager.Instance.GetBitmapResource(key), new Point2dF(Convert.ToSingle(originX), Convert.ToSingle(originY)));
                else
                    this.DiscardCurrentBuffer();

                this.currentFrameKey = key;
            }
        }

        /// <summary>This method invokes the rendering. For use by the appliation to tell the control to change images on demand.</summary>
        public void Render()
        {
            this.Invalidate();
        }

        /// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
        /// <param name="key">Frame key to set as current image</param>
        public virtual void SetRenderFrameAndRender(Int32 key)
        {
            this.SetRenderFrameAndRender(key, false);
        }

        /// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
        /// <param name="key">Frame key to set as current image</param>
        /// <param name="freePreviousFrame">
        ///     Flag indicating whether to dispose of the previous image set
        ///     (in the case of transient images, such as composite images for a game or high-frame video playback)
        /// </param>
        public virtual void SetRenderFrameAndRender(Int32 key, Boolean freePreviousFrame)
        {
            this.SetRenderFrameAndRender(key, 0, 0, freePreviousFrame);
        }

        /// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
        /// <param name="key">Frame key to set as current image</param>
        /// <param name="originX">X coordinate to start drawing from</param>
        /// <param name="originY">Y coordinate to start drawing from</param>
        public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY)
        {
            this.SetRenderFrameAndRender(key, originX, originY, false);
        }

        /// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
        /// <param name="key">Frame key to set as current image</param>
        /// <param name="originX">X coordinate to start drawing from</param>
        /// <param name="originY">Y coordinate to start drawing from</param>
        /// <param name="freePreviousFrame">
        ///     Flag indicating whether to dispose of the previous image set
        /// </param>
        public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY, Boolean freePreviousFrame)
        {
            lock (this.controlBufferLock)
            {
                Int32 previousFrameKey = this.currentFrameKey;

                this.SetRenderFrame(key, originX, originY);
                this.Render();

                if (freePreviousFrame)
                    this.FreeFrameResource(previousFrameKey);
            }
        }
        #endregion
    }
}

Win32 多媒体计时器回调的代码

排除了大小，但这里是相关代码，链接到“winmm.dll”，EntryPoint = “timeSetEvent”

        /// <summary>Callback method for WIn32 API</summary>
        protected virtual void Win32Callback(UInt32 timerId, UInt32 message, IntPtr user, IntPtr param1, IntPtr param2)
        {
            TimeSpan raise;
            lock (this.timerLock)
            {
                //get system time for start time
                UInt32 uptime = Win32MultimediaTimeFunctions.GetEnvironmentUptime();
                Int32 upTimeSec = (Int32)(uptime / 1000);
                Int32 upTimeMilliSec = (Int32)(uptime % 1000);
                TimeSpan WinUpTime = new TimeSpan(0, 0, 0, upTimeSec, upTimeMilliSec);
                raise = WinUpTime - this.StartTime;
                //this.elapsed(raise);
                this.RaiseTimer(raise);
            }

            //Note: outside the lock, this.elapse(raise) kills the display driver
            //inside, without threading, it is fine.
            //inside, with threading, the display driver will crash after a bit of playback
        }

        protected void RaiseTimer(TimeSpan elapsedTime)
        {
            Thread timerEvent = new Thread(() => { this.RunThread(elapsedTime); });
            timerEvent.IsBackground = true;
            timerEvent.Name = "Timer callback";
            timerEvent.Start();
        }

        /// <summary>Raises the timer on a separate thread</summary>
        protected void RunThread(TimeSpan elapsedTime)
        {
            if (this.threadLock.WaitOne(0))
            {
                this.elapsed(elapsedTime);
                this.threadLock.ReleaseMutex();
            }
        }

结论

3 周后，我毫不羞愧地说我对这种行为感到困惑，并寻找任何类型的输入来识别竞争条件，从架构上讲，我可能会遗漏一些关于 Direc2D 或我在哪里'只是显示我的无知。

Answer 1

代码中的所有锁语句似乎是一个过于复杂的解决方案。此外，从构造函数创建 COM 对象(如 Direct2D1)并从线程 UI 方法使用它们通常不是一个好习惯。

我会通过在同一个线程上执行所有设备/对象的创建和呈现来简化您的应用程序，同时将您的呈现代码保留在表单之外。理想情况下，这应该是完全独立的，因为您可以通过 HWND 或 DXGI 表面或 Windows 8 Metro 表面呈现任何类型的窗口/控件。

此外，正如 Aren 所建议的，您应该依赖现有的强大包装器，而不是使用您自己的 Direct2D1 包装器，例如 SharpDX .它将避免您维护昂贵的层并帮助您专注于您的特定应用程序。