c# - 在 C# : synchronisation issues 中记住和重放击键

Question

我是 PC 游戏的速通玩家(喜欢以尽可能快的方式完成游戏的人)，我想在玩游戏时记录我的输入，以便稍后自动重播。所以我创建了一个小 C# 程序来做到这一点:基本上，它会启动一个计时器，每次我按下/松开一个键时，它都会保存操作(keyup/keydown)、键以及我这样做的毫秒数。然后，当我想再次播放它时，它会启动一个计时器，当它达到发生击键的毫秒时，它会重现它。

它有效!嗯...事实上，它几乎可以工作:键复制得很好，但有时，它们有点不同，导致我之前成功的意外死亡。

这是显示问题的视频:
https://www.youtube.com/watch?v=4RPkcx68hpw&feature=youtu.be

上面的视频是重制键，下面的视频是原版播放。一切似乎都相似，直到第 3 个房间，原始戏剧击中“蜘蛛”并使其返回，而复制的键不接触它，因此它干扰了其余的进程。当然，这部分游戏是 100% 确定的，因此相同的输入会导致相同的结果。通过在我的视频编辑器中逐帧推进视频，当角色爬上第一个 crate 时，我清楚地看到了 2 帧的差距，并且这个差距继续扩大。

这是我的(大量评论)代码:

KeysSaver.cs，保存我输入的类

class KeysSaver
{
    public static IntPtr KEYUP = (IntPtr)0x0101; // Code of the "key up" signal
    public static IntPtr KEYDOWN = (IntPtr)0x0100; // Code of the "key down" signal
    private Stopwatch watch; // Timer used to trace at which millisecond each key have been pressed
    private Dictionary<long, Dictionary<Keys, IntPtr>> savedKeys; // Recorded keys activity, indexed by the millisecond the have been pressed. The activity is indexed by the concerned key ("Keys" type) and is associated with the activity code (0x0101 for "key up", 0x0100 for "key down").
    private IntPtr hookId; // Hook used to listen to the keyboard

    private delegate IntPtr LowLevelKeyboardProc(int nCode, IntPtr wParam, IntPtr lParam); // Imported type : LowLevelKeyboardProc. Now we can use this type.


    /*
     * Constructor 
     */
    public KeysSaver()
    {
        this.savedKeys = new Dictionary<long, Dictionary<Keys, IntPtr>>();
        this.watch = new Stopwatch();
    }

    /*
     * method Start()
     * Description : starts to save the keyboard inputs.
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms644990%28v=vs.85%29.aspx
     */

    public void Start()
    {
        using (Process curProcess = Process.GetCurrentProcess())
        using (ProcessModule curModule = curProcess.MainModule) // Get the actual thread
        {
            // Installs a hook to the keyboard (the "13" params means "keyboard", see the link above for the codes), by saying "Hey, I want the function 'onActivity' being called at each activity. You can find this function in the actual thread (GetModuleHandle(curModule.ModuleName)), and you listen to the keyboard activity of ALL the treads (code : 0)
            this.hookId = SetWindowsHookEx(13, onActivity, GetModuleHandle(curModule.ModuleName), 0);
        }
        this.watch.Start(); // Starts the timer
    }

    /*
     * method Stop()
     * Description : stops to save the keyboard inputs.
     * Returns : the recorded keys activity since Start().
     */
    public Dictionary<long, Dictionary<Keys, IntPtr>> Stop()
    {
        this.watch.Stop(); // Stops the timer
        UnhookWindowsHookEx(this.hookId); //Uninstalls the hook of the keyboard (the one we installed in Start())
        return this.savedKeys;
    }

    /*
     * method onActivity()
     * Description : function called each time there is a keyboard activity (key up of key down). Saves the detected activity and the time at the moment it have been done.
     * @nCode : Validity code. If >= 0, we can use the information, otherwise we have to let it.
     * @wParam : Activity that have been detected (keyup or keydown). Must be compared to KeysSaver.KEYUP and KeysSaver.KEYDOWN to see what activity it is.
     * @lParam : (once read and casted) Key of the keyboard that have been triggered.
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms644985%28v=vs.85%29.aspx (for this function documentation)
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms644974%28v=vs.85%29.aspx (for CallNextHookEx documentation)
     */
    private IntPtr onActivity(int nCode, IntPtr wParam, IntPtr lParam)
    {
        if (nCode >= 0) //We check the validity of the informations. If >= 0, we can use them.
        {
            long time = this.watch.ElapsedMilliseconds; //Number of milliseconds elapsed since we called the Start() method
            int vkCode = Marshal.ReadInt32(lParam); //We read the value associated with the pointer (?)
            Keys key = (Keys)vkCode; //We convert the int to the Keys type
            if (!this.savedKeys.ContainsKey(time))
            {
                // If no key activity have been detected for this millisecond yet, we create the entry in the savedKeys Dictionnary
                this.savedKeys.Add(time, new Dictionary<Keys, IntPtr>());
            }
            this.savedKeys[time].Add(key, wParam); //Saves the key and the activity
        }
        return CallNextHookEx(IntPtr.Zero, nCode, wParam, lParam); //Bubbles the informations for others applications using similar hooks
    }

    // Importation of native libraries
    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelKeyboardProc lpfn, IntPtr hMod, uint dwThreadId);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    private static extern bool UnhookWindowsHookEx(IntPtr hhk);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode,
        IntPtr wParam, IntPtr lParam);

    [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr GetModuleHandle(string lpModuleName);
}

KeysPlayer.cs，模拟关键事件的那个。

class KeysPlayer
    {
        private Dictionary<long, Dictionary<Keys, IntPtr>> keysToPlay; // Keys to play, with the timing. See KeysSaver.savedKeys for more informations.
        private Dictionary<long, INPUT[]> playedKeys; // The inputs that will be played. This is a "translation" of keysToPlay, transforming Keys into Inputs.
        private Stopwatch watch; // Timer used to respect the strokes timing.
        private long currentFrame; // While playing, keeps the last keysToPlay frame that have been played.

        /*
         * Constructor 
         */
        public KeysPlayer(Dictionary<long, Dictionary<Keys, IntPtr>> keysToPlay)
        {
            this.keysToPlay = keysToPlay;
            this.playedKeys = new Dictionary<long, INPUT[]>();
            this.watch = new Stopwatch();
            this.currentFrame = 0;
            this.loadPlayedKeys(); //Load the keys that will be played.
        }

        /*
         * method Start()
         * Description : starts to play the keyboard inputs.
         */
        public void Start()
        {
            this.currentFrame = 0;  //currentFrame is 0 at the beginning.
            this.watch.Reset(); //Resets the timer
            this.watch.Start(); //Starts the timer (yeah, pretty obvious)
            IEnumerator<long> enumerator = this.playedKeys.Keys.GetEnumerator(); //The playedKeys enumerator. Used to jump from one frame to another.
            long t; //Will receive the elapsed milliseconds, to track desync.
            while (enumerator.MoveNext()) //Moves the pointer of the playedKeys dictionnary to the next entry (so, to the next frame).
            {
                Thread.Sleep((int)(enumerator.Current - this.currentFrame - 1)); //The thread sleeps until the millisecond before the next frame. For exemple, if there is an input at the 42th millisecond, the thread will sleep to the 41st millisecond. Seems optionnal, since we have a "while" that waits, but it allows to consume less ressources. Also, in a too long "while", the processor tends to "forget" the thread for a long time, resulting in desyncs.
                while (this.watch.ElapsedMilliseconds < enumerator.Current) { } //We wait until the very precise millisecond that we want
                t = this.watch.ElapsedMilliseconds; //We save the actual millisecond
                uint err = SendInput((UInt32)this.playedKeys[enumerator.Current].Length, this.playedKeys[enumerator.Current], Marshal.SizeOf(typeof(INPUT))); //Simulate the inputs of the actual frame
                if (t != enumerator.Current) // We compare the saved time with the supposed millisecond. If they are different, we have a desync, so we log some infos to track the bug.
                {
                    Console.WriteLine("DESYNC : " + t + "/" + enumerator.Current + " - Inputs : " + err);
                }
                this.currentFrame = enumerator.Current; //Updates the currentFrame to the frame we just played.
            }
        }

        /*
         * method Stop()
         * Description : stops to play the keyboard inputs.
         */
        public void Stop()
        {
            this.watch.Stop(); //Stops the timer.
        }

        /*
         * method loadPlayedKeys()
         * Description : Transforms the keysToPlay dictionnary into a sequence of inputs. Also, pre-load the inputs we need (loading takes a bit of time that could lead to desyncs).
         */
        private void loadPlayedKeys()
        {
            foreach (KeyValuePair<long, Dictionary<Keys, IntPtr>> kvp in this.keysToPlay)
            {
                List<INPUT> inputs = new List<INPUT>(); //For each recorded frame, creates a list of inputs
                foreach (KeyValuePair<Keys, IntPtr> kvp2 in kvp.Value)
                {
                    inputs.Add(this.loadKey(kvp2.Key, this.intPtrToFlags(kvp2.Value))); //Load the key that will be played and adds it to the list. 
                }
                this.playedKeys.Add(kvp.Key, inputs.ToArray());//Transforms the list into an array and adds it to the playedKeys "partition".
            }
        }

        /*
         * method intPtrToFlags()
         * Description : Translate the IntPtr which references the activity (keydown/keyup) into input flags.
         */
        private UInt32 intPtrToFlags(IntPtr activity)
        {
            if (activity == KeysSaver.KEYDOWN) //Todo : extended keys
            {
                return 0;
            }
            if (activity == KeysSaver.KEYUP)
            {
                return 0x0002;
            }
            return 0;
        }

        /*
         * method loadKey()
         * Description : Transforms the Key into a sendable input (using the above structures).
         */
        private INPUT loadKey(Keys key, UInt32 flags)
        {
            return new INPUT
            {
                Type = 1, //1 = "this is a keyboad event"
                Data =
                {
                    Keyboard = new KEYBDINPUT
                    {
                        KeyCode = (UInt16)key,
                        Scan = 0,
                        Flags = flags,
                        Time = 0,
                        ExtraInfo = IntPtr.Zero
                    }
                }

            };
        }

        // Importation of native libraries
        [DllImport("user32.dll", SetLastError = true)]
        public static extern UInt32 SendInput(UInt32 numberOfInputs, INPUT[] inputs, Int32 sizeOfInputStructure);

        [DllImport("kernel32.dll")]
        static extern uint GetLastError();

    }
}

SendInput 使用的所有结构(从 InputSimulator 脚本复制的):

/*
     * Struct MOUSEINPUT
     * Mouse internal input struct
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms646273(v=vs.85).aspx
     */
    internal struct MOUSEINPUT
    {
        public Int32 X;
        public Int32 Y;
        public UInt32 MouseData;
        public UInt32 Flags;
        public UInt32 Time;
        public IntPtr ExtraInfo;
    }

    /*
     * Struct HARDWAREINPUT
     * Hardware internal input struct
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms646269(v=vs.85).aspx
     */
    internal struct HARDWAREINPUT
    {
        public UInt32 Msg;
        public UInt16 ParamL;
        public UInt16 ParamH;
    }

    /*
     * Struct KEYBDINPUT
     * Keyboard internal input struct (Yes, actually only this one is used, but we need the 2 others to properly send inputs)
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms646271(v=vs.85).aspx
     */
    internal struct KEYBDINPUT
    {
        public UInt16 KeyCode; //The keycode of the triggered key. See https://msdn.microsoft.com/en-us/library/windows/desktop/dd375731(v=vs.85).aspx
        public UInt16 Scan; //Unicode character in some keys (when flags are saying "hey, this is unicode"). Ununsed in our case.
        public UInt32 Flags; //Type of action (keyup or keydown). Specifies too if the key is a "special" key.
        public UInt32 Time; //Timestamp of the event. Ununsed in our case.
        public IntPtr ExtraInfo; //Extra information (yeah, it wasn't that hard to guess). Ununsed in our case.
    }

    /*
     * Struct MOUSEKEYBDHARDWAREINPUT
     * Union struct for key sending 
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms646270%28v=vs.85%29.aspx
     */

    [StructLayout(LayoutKind.Explicit)]
    internal struct MOUSEKEYBDHARDWAREINPUT
    {
        [FieldOffset(0)]
        public MOUSEINPUT Mouse;

        [FieldOffset(0)]
        public KEYBDINPUT Keyboard;

        [FieldOffset(0)]
        public HARDWAREINPUT Hardware;
    }

    /*
     * Struct INPUT
     * Input internal struct for key sending 
     * See : https://msdn.microsoft.com/en-us/library/windows/desktop/ms646270%28v=vs.85%29.aspx
     */

    internal struct INPUT
    {
        public UInt32 Type; //Type of the input (0 = Mouse, 1 = Keyboard, 2 = Hardware)
        public MOUSEKEYBDHARDWAREINPUT Data; //The union of "Mouse/Keyboard/Hardware". Only one is read, depending of the type.
    }

而我的主要形式:

public partial class Taslagrad : Form
    {
        private KeysSaver k;
        private KeysPlayer p;

        //Initialisation 
        public Taslagrad()
        {
            InitializeComponent();
            this.k = new KeysSaver();
        }

        /*
         * method launchRecording()
         * Description : Starts to record the keys. Called when the "record" button is triggered.
         */
        private void launchRecording(object sender, EventArgs e)
        {
            this.k.Start(); //Starts to save the keys
            startButton.Text = "Stop"; //Updates the button
            startButton.Click -= launchRecording;
            startButton.Click += stopRecording;
        }

        /*
         * method stopRecording()
         * Description : Stops to record the keys and logs the recorded keys in the console. Called when the "record" button is triggered.
         */
        private void stopRecording(object sender, EventArgs e)
        {
            startButton.Text = "Record";//Updates the button
            startButton.Click += launchRecording;
            startButton.Click -= stopRecording;
            Dictionary<long, Dictionary<Keys, IntPtr>> keys = this.k.Stop(); //Gets the recorded keys
            foreach (KeyValuePair<long, Dictionary<Keys, IntPtr>> kvp in keys)
            {
                foreach (KeyValuePair<Keys, IntPtr> kvp2 in kvp.Value)
                {
                    //Displays the recorded keys in the console
                    if (kvp2.Value == KeysSaver.KEYDOWN)
                    {
                        Console.WriteLine(kvp.Key + " : (down)" + kvp2.Key);
                    }
                    if (kvp2.Value == KeysSaver.KEYUP)
                    {
                        Console.WriteLine(kvp.Key + " : (up)" + kvp2.Key);
                    }
                }
            }
            this.p = new KeysPlayer(keys); //Creates a new player and gives it the recorded keys.
        }

        /*
         * method launchPlaying()
         * Description : Starts to play the keys. Called when the "play" button is triggered.
         */
        private void launchPlaying(object sender, EventArgs e)
        {
            this.p.Start(); //Starts to play the keys.
        }
}

当然，我所有的调试似乎都正常工作:录音机保存了所有输入(我通过键入长文本进行了测试)，当我比较录制键的毫秒数和播放键的毫秒数时，我没有区别。 ..

那么我录制/播放的方式有问题吗？秒表不够精确？有没有更精确/有效的方法？

Answer 1

碰巧欣赏MetalFoxDoS的工作。乍一看，它的边缘有点粗糙，但工作的要点就在那里。

诚实的答案是肯定的，如果用户提供输入出现延迟，他的精度有限并且运行的 CPU % 比 NO-OP 所需的要多一些。在采用异步方法的同时解决这个问题给了我微秒精度 - 远远超出人类所能提供的准确性。

我继续加强这项工作，试图进一步完善它，因为我意识到需要根据按键事件之间的延迟创建“框架”。我还纠正了过去几年差距所需的一些错误/更改。

https://github.com/houseofcat/WinKeyRecorder