javascript - 使用 Web 音频工作集录制音频时出现咔嗒声

Question

我创建了一个 Electron 应用程序，它通过将 block 存储到一个数组中，然后将记录的 block 发送到主进程以写入一个 wav 文件，使用音频工作集节点记录音频。该工作集还计算计量并检查值是否被剪裁，但是这些计算发生在异步函数中，而进程函数没有等待它解决以防止缓冲区欠载。此外，为了直接监控，输入流连接到媒体流目的地节点。整个设置在大多数情况下都运行良好，但对于少量录制的音频文件，文件的随机部分会出现明显的点击噪音。奇怪的是，您在直接监听输出端听不到这些咔哒声。在查看文件的波形时，似乎有些样本只是缺少频谱图中也显示的内容:

waveform
spectrogram

我测量了处理方法每次运行所花费的时间，并记录了花费超过 2.9 毫秒(128 个样本/44100 kHz 单声道 => ~2.9 毫秒)的部分，有时花费的时间比这更长，但咔哒声不会出现在那些部位。是否有可能出现缓冲区欠载的 Web 音频 api，或者是否有一些内部缓冲区，并且发生这种情况时延迟会变得更糟？我只是无法弄清楚点击来自哪里。以下是代码的相关部分。

工作台代码:

const statsWindowSize = 1024 * 8; // ~5 stats per second for 44kHz
const clipThreshold = 0.98;

/* eslint-disable */
class RecordingWorkletProcessor extends AudioWorkletProcessor {

  constructor() {
    super();
    this.isRecording = false;
    this.clipping = false;
    this.sampleIndex = 0;
    this.sum = 0;
    this.recordedBuffers = [];
    this.writeIndex = 0;

    this.port.onmessage = ({ data }) => {
      if (data.type === 'startRecording') {
        this.writeIndex = 0;
        this.recordedBuffers = [];
        this.isRecording = true;
      } else if (data.type === 'stopRecording') {
        this.port.postMessage({
          type: 'recording',
          buffers: this.recordedBuffers,
        });
        this.isRecording = false;
      }
    };
  }

  async computeStats(buffer) {
    // ...removed to shorten the code snipped
  }

  process(inputs, outpus, parameters) {
    const t0 = Date.now();
    const writeIndex = this.writeIndex;
    this.writeIndex += 1;

    // Select the first input's first channel
    const buffer0 = inputs[0][0];
    // const { windowSize, clipThreshold, isRecording } = parameters;
    if (this.isRecording) {
      // Clone the data into a new Float32Array
      const f32Buffer = new Float32Array(buffer0.length);
      f32Buffer.set(buffer0);
      this.recordedBuffers.splice(writeIndex, 0, f32Buffer);
    }

    // Detach the stats computation to prevent underruns
    this.computeStats(buffer0);

    // this.lastRunFinished = true;
    if (this.isRecording) {
      const t1 = Date.now();
      const elapsedTime = t1 - t0;
      if (elapsedTime > (128 / 44100) * 1000) {
        const atPosition = (writeIndex * 128) / 44100;
        this.port.postMessage({ type: 'underrun', elapsedTime, atPosition });
      }
    }
    // Keep processor alive
    return true;
  }
}
/* eslint-enable */

registerProcessor('recording-worklet-processor', RecordingWorkletProcessor);

编写波形文件的代码:

// before these parts recordedBuffers will be send from the worklet via postMessage
// Merge all buffers from channel 1 into a single Float32Array
const totalByteLength = recordedBuffers.reduce(
  (total, buf) => total + buf.byteLength,
  0,
);
const header = Header({
  sampleRate: ctx.sampleRate,
  channels: 1,
  bitsPerSample: 32,
  audioFormat: IEEE_FLOAT,
  byteLength: totalByteLength,
});
const wstream = createWriteStream(audioFilePath);
wstream.write(header);
// RealBuffer is just an alias for the node Buffer type
const chunks = RealBuffer.allocUnsafe(totalByteLength);
let offset = 0;

for (let i = 0; i < recordedBuffers.length; i++) {
  const typedArray = recordedBuffers[i];
  for (let j = 0; j < typedArray.length; j++) {
    chunks.writeFloatLE(typedArray[j], offset);
    offset += typedArray.BYTES_PER_ELEMENT;
  }
}
wstream.write(chunks);
wstream.end();

创建 header 的模块:

import { RealBuffer } from 'utils/io'; // An alias for the node Buffer type

export const PCM = 1;
export const IEEE_FLOAT = 3;

export const Header = ({
  sampleRate,
  channels,
  bitsPerSample,
  byteLength,
  audioFormat,
}) => {
  let offset = 0;
  const buffer = RealBuffer.allocUnsafe(44);

  const writeString = (str) => {
    for (let i = 0; i < str.length; i += 1) {
      buffer.writeUInt8(str.charCodeAt(i), offset + i);
    }
    offset += str.length;
  };

  const writeUint32 = (value) => {
    buffer.writeUInt32LE(value, offset);
    offset += 4;
  };

  const writeUint16 = (value) => {
    buffer.writeUInt16LE(value, offset);
    offset += 2;
  };

  const blockAlign = channels * (bitsPerSample / 8);
  const byteRate = sampleRate * blockAlign;
  const chunkSize = (byteLength / 8) - 8;

  writeString('RIFF'); // ChunkID
  writeUint32(chunkSize); // ChunkSize
  writeString('WAVE'); // Format
  writeString('fmt '); // Subchunk1ID
  writeUint32(16); // Subchunk1Size
  writeUint16(audioFormat); // AudioFormat (PCM=1,IEEE Float=3,...)
  writeUint16(channels); // Channels
  writeUint32(sampleRate); // SampleRate
  writeUint32(byteRate); // ByteRate
  writeUint16(blockAlign); // BlockAlign
  writeUint16(bitsPerSample); // BitsPerSample
  writeString('data'); // Subchunk2ID
  writeUint32(byteLength); // Subchunk2Size
  return buffer;
};

export default Header;

Answer 1

在 process函数你总是在 f32buffer 中创建一个新数组每次调用该函数时。这最终需要收集，所以我猜测故障是由 GC 收集您创建的所有垃圾引起的。

您可以使用 chrome://tracing 获取更多详细信息以获取有关此信息。按记录，然后编辑类别并选择 blink_gc 和 webaudio，也许还有音频。然后记录跟踪并检查图表以查看发生了什么。