Android，将音轨添加到 Muxer

Question

我正在尝试将音频添加到由以下开源项目创建的视频

特别是https://github.com/madisp/trails/blob/master/app/src/main/java/com/madisp/trails/CaptureService.java

我需要从 MIC 获取音频并将其作为音轨写入编码文件。目前，使用 Muxer 编码的文件只有视频轨道。

我可以毫无问题地从 MIC 获取音频

int nChannels = 1;
int minBufferSize = AudioRecord.getMinBufferSize(44100, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT) * 2;
AudioRecord aRecorder = new AudioRecord(MediaRecorder.AudioSource.MIC, 44100, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, minBufferSize);

short[] buffer = new short[44100 * nChannels];
aRecorder.startRecording();
int readSize = 0;

while (recording) {
    readSize = aRecorder.read(buffer, 0, minBufferSize);
    if (readSize < 0) {
        break;
    } else if (readSize > 0) {
        // do stuff with buffer
    }
}
aRecorder.stop();
aRecorder.release();

但我不确定如何将它合并到 ( https://github.com/madisp/trails/blob/master/app/src/main/java/com/madisp/trails/CaptureService.java )

while (running) {
    int index = avc.dequeueOutputBuffer(info, 10000);
    if (index == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        if (track != -1) {
            throw new RuntimeException("format changed twice");
        }
        track = muxer.addTrack(avc.getOutputFormat());
        muxer.start();
    } else if (index >= 0) {
        if ((info.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
            // ignore codec config
            info.size = 0;
        }
        if (track != -1) {
            ByteBuffer out = avc.getOutputBuffer(index);
            out.position(info.offset);
            out.limit(info.offset + info.size);
            muxer.writeSampleData(track, out, info);
            avc.releaseOutputBuffer(index, false);
        }
    }
}

是的，请理解我实际上是在要求您编写代码，但我对此并不具备专业知识

感谢任何帮助

谢谢

Answer 1

首先，使用 byte[] 而不是 short[] 作为与 AudioRecord 一起使用的缓冲区 - 这会稍微简化一些事情。

然后，为了对接收到的缓冲区进行编码，像这样的东西应该可以工作(未经测试):

while (recording) {
    readSize = aRecorder.read(buffer, 0, minBufferSize);
    if (readSize < 0) {
        break;
    } else if (readSize > 0) {
        boolean done = false;
        while (!done) {
            int index = avc.dequeueInputBuffer(10000);
            if (index >= 0) { // In case we didn't get any input buffer, it may be blocked by all output buffers being full, thus try to drain them below if we didn't get any
                ByteBuffer in = avc.getIndexBuffer(index);
                in.clear();
                in.put(buffer, 0, readSize);
                avc.queueInputBuffer(index, 0, readSize, System.nanoTime()/1000, 0);
                done = true; // Done passing the input to the codec, but still check for available output below
            }
            index = avc.dequeueOutputBuffer(info, 10000);
            if (index == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
                if (track != -1) {
                    throw new RuntimeException("format changed twice");
                }
                track = muxer.addTrack(avc.getOutputFormat());
                muxer.start();
            } else if (index >= 0) {
                if ((info.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
                    // ignore codec config
                    info.size = 0;
                }
                if (track != -1 && info.size > 0) {
                    ByteBuffer out = avc.getOutputBuffer(index);
                    out.position(info.offset);
                    out.limit(info.offset + info.size);
                    muxer.writeSampleData(track, out, info);
                    avc.releaseOutputBuffer(index, false);
                }
            }
        }
    }
}

我认为普通的 SW AAC 编码器应该可以将任意数量的音频字节传递给它，但如果编码器很挑剔，您需要将记录的数据以 1024 个样本 block 的形式传递给它(2048单声道字节，立体声 4096 字节)。