java - 在 Android 上创建视频文件

Question

在我的 Android 应用程序中，我尝试创建一个视频文件，在视频的给定时间位置添加音轨。我使用了 MediaMuxer 并更改了 presentationTimeUs 的值来移动音频。但显然这不是要走的路，因为视频的开始时间也发生了变化。另一个问题是 mp3 音频不起作用。到目前为止，这是我的尝试:

final long audioPositionUs = 10000000;
File fileOut = new File (Environment.getExternalStoragePublicDirectory (
  Environment.DIRECTORY_MOVIES) + "/output.mp4");
fileOut.createNewFile ();
MediaExtractor videoExtractor = new MediaExtractor ();
MediaExtractor audioExtractor = new MediaExtractor ();
AssetFileDescriptor videoDescriptor = getAssets ().openFd ("video.mp4");
// AssetFileDescriptor audioDescriptor = getAssets ().openFd ("audio.mp3"); // ?!
AssetFileDescriptor audioDescriptor = getAssets ().openFd ("audio.aac");
videoExtractor.setDataSource (videoDescriptor.getFileDescriptor (),
    videoDescriptor.getStartOffset (), videoDescriptor.getLength ());
audioExtractor.setDataSource (audioDescriptor.getFileDescriptor (),
    audioDescriptor.getStartOffset (), audioDescriptor.getLength ());
MediaFormat videoFormat = null;
for (int i = 0; i < videoExtractor.getTrackCount (); i++) {
  if (videoExtractor.getTrackFormat (i).getString (
      MediaFormat.KEY_MIME).startsWith ("video/")) {
    videoExtractor.selectTrack (i);
    videoFormat = videoExtractor.getTrackFormat (i);
    break;
  }
}
audioExtractor.selectTrack (0);
MediaFormat audioFormat = audioExtractor.getTrackFormat (0);
MediaMuxer muxer = new MediaMuxer (fileOut.getAbsolutePath (),
    MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
int videoTrack = muxer.addTrack (videoFormat);
int audioTrack = muxer.addTrack (audioFormat);
boolean end = false;
int sampleSize = 256 * 1024;
ByteBuffer videoBuffer = ByteBuffer.allocate (sampleSize);
ByteBuffer audioBuffer = ByteBuffer.allocate (sampleSize);
MediaCodec.BufferInfo videoBufferInfo = new MediaCodec.BufferInfo ();
MediaCodec.BufferInfo audioBufferInfo = new MediaCodec.BufferInfo ();
videoExtractor.seekTo (0, MediaExtractor.SEEK_TO_CLOSEST_SYNC);
audioExtractor.seekTo (0, MediaExtractor.SEEK_TO_CLOSEST_SYNC);
muxer.start ();
while (!end) {
  videoBufferInfo.size = videoExtractor.readSampleData (videoBuffer, 0);
  if (videoBufferInfo.size < 0) {
    end = true;
    videoBufferInfo.size = 0;
  } else {
    videoBufferInfo.presentationTimeUs = videoExtractor.getSampleTime ();
    videoBufferInfo.flags = videoExtractor.getSampleFlags ();
    muxer.writeSampleData (videoTrack, videoBuffer, videoBufferInfo);
    videoExtractor.advance ();
  }
}
end = false;
while (!end) {
  audioBufferInfo.size = audioExtractor.readSampleData (audioBuffer, 0);
  if (audioBufferInfo.size < 0) {
    end = true;
    audioBufferInfo.size = 0;
  } else {
    audioBufferInfo.presentationTimeUs = audioExtractor.getSampleTime () +
        audioPositionUs;
    audioBufferInfo.flags = audioExtractor.getSampleFlags ();
    muxer.writeSampleData (audioTrack, audioBuffer, audioBufferInfo);
    audioExtractor.advance ();
  }
}
muxer.stop ();
muxer.release ();

能否请您提供详细信息(如果可能，请提供代码)以帮助我解决此问题？

Answer 1

将 AudioRecord 的样本发送到 MediaCodec + MediaMuxer 包装器。在 audioRecord.read(...) 中使用系统时间作为音频时间戳非常有效，前提是您经常轮询以避免填满 AudioRecord 的内部缓冲区(以避免您调用 read 的时间和 AudioRecord 记录样本的时间之间的漂移).太糟糕了 AudioRecord 不直接传达时间戳...

//设置录音

while (isRecording) {
    audioPresentationTimeNs = System.nanoTime();
    audioRecord.read(dataBuffer, 0, samplesPerFrame);
    hwEncoder.offerAudioEncoder(dataBuffer.clone(), audioPresentationTimeNs);
}

请注意，AudioRecord 仅保证支持 16 位 PCM 样本，尽管 MediaCodec.queueInputBuffer 将输入作为 byte[]。将 byte[] 传递给 audioRecord.read(dataBuffer,...) 将为您将 16 位样本截断为 8 位。

我发现以这种方式轮询仍然偶尔会生成 timestampUs XXX < lastTimestampUs XXX 用于音频轨道错误，因此我包含了一些逻辑来跟踪 mediaCodec.dequeueOutputBuffer(bufferInfo, timeoutMs) 报告的 bufferInfo.presentationTimeUs 并调整是否在调用 mediaMuxer.writeSampleData(trackIndex, encodedData, bufferInfo) 之前是必要的。