java - 将麦克风输入转换为键盘输入

Question

我有一个键盘踏板(如钢琴键盘)，已插入麦克风插孔。我希望能够将输入转换为击键(例如，每当我踩踏板时，就按计算机键盘上的“V”)。我知道我可以使用 Java 的 AWT Robot 来执行击键，但我在处理麦克风输入时遇到问题(我没有音频处理经验)。

这是我踩下并释放踏板的信号:

对于有经验的用户来说，这似乎是一项非常简单的任务，有人知道我该怎么做吗？

到目前为止，我基本上是从周围复制粘贴代码来获得这个:

我正在使用 javax.sound API 将麦克风输入读取为字节。我试图检测踩踏板时振幅的跳跃...字节被转换为短裤，并将该值与任意高的数字进行比较。它似乎有效，但踩下踏板后，机器人只会永远按下该键(值> 32000)。

        while (true) {
            // Read the next chunk of data from the TargetDataLine.
            numBytesRead = microphone.read(data, 0, data.length);
            // Save this chunk of data.
            out.write(data, 0, numBytesRead);

            byte[] bytes = out.toByteArray();
            short[] shorts = new short[bytes.length/2];
            // to turn bytes to shorts as either big endian or little endian. 
            ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts);

            for (short s : shorts) {
                int value = Math.abs(s);
                if (value > 32000)
                {
                    robot.keyPress(KeyEvent.VK_V);
                    robot.keyRelease(KeyEvent.VK_V);
                    break;
                } else {
                    //robot.keyRelease(KeyEvent.VK_V);
                }
            }
        }

编辑:

它一直持续下去，因为我没有清除 ByteArrayOutputStream 缓冲区，所以随着时间的推移，我不断读取相同的字节集以及新的字节集。 out.reset() 为我解决了这个问题。

我现在的问题是关于我从踏板读取的输入，如果我在短时间内按下并释放踏板，我将无法正确解释按下或释放踏板。

红色圆圈显示踩下(并保持)踏板时得到的结果，黑色矩形显示释放踏板时得到的结果。

正如您所看到的，当它受到抑制时，它会先下降然后快速增加，然后再逐渐回到0。当它被释放时，它会猛烈上升，然后快速下降，然后再次回到 0。

我现在用来区分两者的方法是仅在两个帧/间隔之间存在较大差异时才注册按下/释放。根据图表，当它为负值时，我将其设置为按下，当它为正值时，我将其设置为释放。

我遇到的问题是，当我在信号处于上升状态时释放踏板时(或者当我在信号处于释放信号下降状态时踩下踏板时)，将没有足够的两个框架之间的差异让我使用这个方法。

这可能有助于可视化问题:

我不知道如何获得一种可靠的方法来检测新闻/发布。

这是我用于读取输入的固定代码，如果有人有兴趣尝试同样的事情(顺便说一下，我正在使用这个踏板:http://www.amazon.co.uk/Cherub-WTB-004-Keyboard-Sustain-Pedal/dp/B000UDVV6E)

import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

import javax.sound.sampled.*;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;

public class PedalToKeyboard {

    public static void main(String[] args) {

        AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);
        Robot robot = null;

        try {
            robot = new Robot();

        } catch (AWTException e) {
                e.printStackTrace();
        }

        try {
            TargetDataLine microphone = AudioSystem.getTargetDataLine(format);          
            System.out.println(microphone);

            microphone.open(format);

            ByteArrayOutputStream out = new ByteArrayOutputStream();
            int numBytesRead;
            byte[] data = new byte[microphone.getBufferSize()/5];

            // Begin audio capture.
            microphone.start();

            boolean keep_going = true;
            boolean keyPressed = false;         
            short previousShort = 0;
            while (keep_going) {

                // Read the next chunk of data from the TargetDataLine.
                numBytesRead = microphone.read(data, 0, data.length);

                // Reset the buffer (get rid of previous shit)
                out.reset();

                // Save this chunk of data.
                out.write(data, 0, numBytesRead);

                byte[] bytes = out.toByteArray();
                short[] shorts = new short[bytes.length/2];

                // to turn bytes to shorts as either big endian or little endian. 
                ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(shorts);

                for (short s : shorts) {

                    // Check if descending or ascending (pedal press is descending, release is ascending)
                    if (s < 0) { // descending                  
                        // make sure drop is large instantaneous drop
                        if (Math.abs(Math.abs(previousShort) - Math.abs(s)) > 10000 && s < -32700) {
                            if (!keyPressed) {
                                keyPressed = true;
                                //robot.keyPress(KeyEvent.VK_V);
                                System.out.println("Pressed: " + s);
                                break;
                            }
                        }
                    } else if (s > 0) { // ascending
                        // make sure increase is large instantaneous increase
                        if (Math.abs(Math.abs(previousShort) - Math.abs(s)) > 10000 && s > 32700) {
                            if (keyPressed) {
                                keyPressed = false;
                                //robot.keyRelease(KeyEvent.VK_V);
                                System.out.println("Released: " + s + "\n");
                                break;
                            }
                        }
                    }

                    previousShort = s;
                }
            }    

        } catch (LineUnavailableException e) {
            e.printStackTrace();
        }
    }
}

Answer 1

好吧，我实际上稍微调整了这些值并修复了一些逻辑(在比较差异时我错误地应用了 Math.abs)。

现在这对我来说效果很好，任何拥有 MIDI 踏板的人都可以尝试调整参数以供自己使用。

package pedal2keyboard;

import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

import javax.sound.sampled.*;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;

/***
 * Author: Dois Koh
 * Date: 27th October 2015
 * 
 * Gets your microphone signal and you can go do whatever you want with it.
 * Right now, it takes signals from my Cherub WTB-004 Keyboard Sustain Pedal, plugged into
 * my microphone jack, and converts it into key presses (holds down V when depressed,
 * releases V when released)
 */
public class PedalToKeyboard {

    // Robot for performing keyboard actions (pressing V)
    public static Robot robot = null;

    // Currently 8KHz, 16 bit signal (2 bytes), single channel, signed (+ and -) and BIG ENDIAN format
    public static AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);

    public static TargetDataLine microphone = null;
    public static boolean pedalPressed = false;

    public static void main(String[] args) {

        try {
            // Initialize robot for later use
            robot = new Robot();

            // Retrieve the line to from which to read in the audio signal
            microphone = AudioSystem.getTargetDataLine(format);

            // Open the line in the specified format -
            // Currently 8KHz, 16 bit signal (2 bytes), single channel, signed (+ and -) and BIG ENDIAN format      
            microphone.open(new AudioFormat(8000.0f, 16, 1, true, true));

            ByteArrayOutputStream out = new ByteArrayOutputStream();
            byte[] data = new byte[microphone.getBufferSize()/8];

            // Begin audio capture.
            microphone.start();

            int numBytesRead = 0;
            short previousShort = 0;

            // Continue until program is manually terminated
            while (true) {

                // Read the next chunk of data from the TargetDataLine.
                numBytesRead = microphone.read(data, 0, data.length);

                // Reset the buffer (get rid of previous data)
                out.reset();

                // Save this chunk of data.
                out.write(data, 0, numBytesRead);

                byte[] bytes = out.toByteArray();
                short[] shorts = new short[bytes.length/2];

                // to turn bytes to shorts as either big endian or little endian. 
                ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(shorts);

                // Iterate through retrieved 16 bit data (shorts)
                for (short s : shorts) {

                    // Check if descending or ascending (pedal press is descending, release is ascending)
                    if (s < 0) { // descending                  
                        // make sure drop is large instantaneous drop
                        if (Math.abs(previousShort - s) > 200 && s < -32700) {
                            if (!pedalPressed) {
                                PedalPressedAction();
                                previousShort = s;
                                break;
                            }
                        }
                    } else if (s > 0) { // ascending
                        // make sure increase is large instantaneous increase
                        if (Math.abs(previousShort - s) > 200 && s > 32700) {
                            if (pedalPressed) {
                                PedalReleasedAction();
                                previousShort = s;
                                break;
                            }
                        }
                    }

                    previousShort = s;
                }
            }    

        } catch (LineUnavailableException | AWTException e) {
            e.printStackTrace();
        } finally {
            if (microphone != null)
                microphone.close();
        }
    }

    /***
     * The action to perform when the pedal is depressed
     */
    public static void PedalPressedAction() {
        pedalPressed = true;
        robot.keyPress(KeyEvent.VK_V);
    }

    /***
     * The action to perform when the pedal is released
     */
    public static void PedalReleasedAction(){
        pedalPressed = false;
        robot.keyRelease(KeyEvent.VK_V);        
    }
}