java - 使用 SourceDataLine 时帧速率不稳定

Question

我使用简单的 Swing 图形进行声音编程已经有一段时间了，但由于某种原因我的帧速率不稳定。

通常我会在后台线程上执行以下操作:

for(;;) {
    // do some drawing
    aPanel.updateABufferedImage();
    // ask for asynchronous repaint
    aPanel.repaint();

    // write the sound
    aSourceDataLine.write(bytes, 0, bytes.length);
}

通过调试，我想我已经将问题追溯到 SourceDataLine#write 的阻塞行为。 。其文档如下:

If the caller attempts to write more data than can currently be written [...], this method blocks until the requested amount of data has been written.

所以，这似乎意味着 SourceDataLine 实际上有自己的自己的缓冲区，当我们将我们的缓冲区传递给时，它正在填充>写。它仅在其自己的缓冲区已满时阻塞。这似乎是一个阻碍:让它按预期阻塞。

为了演示该问题，这里有一个最小的示例:

• 将 0 写入 SourceDataLine(无声音)并计时。
• 绘制任意图形(翻转每个像素颜色)并计算重绘周期的时间。

import javax.swing.*;
import java.awt.*;
import java.awt.event.*;
import java.awt.image.*;
import javax.sound.sampled.*;

class FrameRateWithSound implements Runnable {
    public static void main(String[] args) {
        SwingUtilities.invokeLater(new FrameRateWithSound());
    }

    volatile boolean soundOn = true;
    PaintPanel panel;

    @Override
    public void run() {
        JFrame frame = new JFrame();
        JPanel content = new JPanel(new BorderLayout());

        final JCheckBox soundCheck = new JCheckBox("Sound", soundOn);
        soundCheck.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                soundOn = soundCheck.isSelected();
            }
        });

        panel = new PaintPanel();

        content.add(soundCheck, BorderLayout.NORTH);
        content.add(panel, BorderLayout.CENTER);

        frame.setContentPane(content);
        frame.pack();
        frame.setLocationRelativeTo(null);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);

        new Thread(new Worker()).start();
    }

    class Worker implements Runnable {
        @Override
        public void run() {
            AudioFormat fmt = new AudioFormat(
                AudioFormat.Encoding.PCM_SIGNED,
                44100f, 8, 1, 1, 44100f, true
            );

            // just 0's
            byte[] buffer = new byte[1000];

            SourceDataLine line = null;
            try {
                line = AudioSystem.getSourceDataLine(fmt);
                line.open(fmt);
                line.start();

                for(;;) {
                    panel.drawNextPixel();
                    panel.repaint();

                    if(soundOn) {
                        // time the write
                        long t = System.currentTimeMillis();

                        line.write(buffer, 0, buffer.length);

                        t = ( System.currentTimeMillis() - t );
                        System.out.println("sound:\t" + t);
                    }

                    // just so it doesn't fly off the handle
                    Thread.sleep(2);
                }
            } catch(Exception e) {
                // lazy...
                throw new RuntimeException(e);
            } finally {
                if(line != null) {
                    line.close();
                }
            }
        }
    }

    class PaintPanel extends JPanel {
        Dimension size = new Dimension(200, 100);

        BufferedImage img = new BufferedImage(
            size.width, size.height, BufferedImage.TYPE_INT_RGB);

        int x, y;

        int repaints;
        long begin, prev;
        String fps = "0";

        PaintPanel() {
            setPreferredSize(size);
            setOpaque(false);

            Graphics2D g = img.createGraphics();
            g.setColor(Color.LIGHT_GRAY);
            g.fillRect(0, 0, size.width, size.height);
            g.dispose();
        }

        synchronized void drawNextPixel() {
            img.setRGB(x, y, img.getRGB(x, y) ^ 0xFFFFFF); // flip

            if( ( ++x ) == size.width ) {
                x = 0;
                if( ( ++y ) == size.height ) {
                    y = 0;
                }
            }
        }

        @Override
        protected synchronized void paintComponent(Graphics g) {
            super.paintComponent(g);
            g.drawImage(img, 0, 0, size.width, size.height, null);

            long curr = System.currentTimeMillis();
            // time this cycle
            long cycle = ( curr - prev );
            System.out.println("paint:\t" + cycle);

            ++repaints;
            // time FPS every 1 second
            if(curr - begin >= 1000) {
                begin = curr;
                fps = String.valueOf(repaints);
                repaints = 0;
            }

            prev = curr;

            g.setColor(Color.RED);
            g.drawString(fps, 12, size.height - 12);
        }
    }
}

如果您对此感到好奇，我建议您实际运行该示例。

“播放”期间的典型 System.out feed 如下所示:

sound:  0
paint:  2
sound:  0
paint:  2
sound:  0
paint:  3
sound:  0
paint:  2
paint:  2
sound:  325 // <- 'write' seems to be blocking here
sound:  0
paint:  328
sound:  0
paint:  2

这非常清楚地显示了 write 的行为:它大部分时间都在旋转，然后阻塞很长一段时间，此时重绘也会突然发生。播放期间 FPS 表通常显示 ~45，但动画明显不稳定。

关闭声音时，FPS 会上升，动画也很流畅。

那么有办法解决吗？我究竟做错了什么？如何让 write 定期阻塞？

这种行为在 Windows 和 OSX 环境中都很明显。

我尝试过的一件事是使用 Thread.sleep 来调节它，但效果不是很好。还是很不稳定。

Answer 1

解决方案似乎是使用 open(AudioFormat, int)打开具有指定缓冲区大小的行。

line.open(fmt, buffer.length);

再次计时，我们可以看到 write 阻塞更加一致:

sound:  22
paint:  24
sound:  21
paint:  24
sound:  20
paint:  22
sound:  21
paint:  23
sound:  20
paint:  23

而且动画很流畅。