swift - CMSampleBuffer 样本值

Question

我正在尝试从 CMSampleBuffer 访问样本以进行进一步处理(主要是视觉处理)，这是我的代码:

 func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    var audioBufferList = AudioBufferList()
    var blockBuffer : CMBlockBuffer?

    CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, nil, &audioBufferList, MemoryLayout<AudioBufferList>.size, nil, nil, 0, &blockBuffer)

    let buffers = UnsafeBufferPointer<AudioBuffer>(start: &audioBufferList.mBuffers, count: Int(audioBufferList.mNumberBuffers))

    var data  =  audioBufferList.mBuffers.mData

    let pointer = data?.assumingMemoryBound(to: Float.self)

    let floatPointer = data?.bindMemory(to: Float.self, capacity: 1024)
    let floatBuffer = UnsafeBufferPointer(start: floatPointer, count: 1024)
    let outputArray = Array(floatBuffer)
    print(outputArray[0..<20])

}

我得到了结果，但它们看起来像是(对我而言)一种奇怪的形式，就像这样(打印语句的输出):

[-snan(0x1dff94), -nan(0x15ffba), 2.93871626e-39, 2.29594486e-39, 8.26537682e-40, -nan(0x1affef), -nan(0xeffe0), 2.84690739e-39, 4.95919386e-39, 4.86737658e-39, 1.19389788e-39, -nan(0x8ffe4), -snan(0xaffa6), -3.29645785e+38, -snan(0x17ff85), -nan(0x5ffae), 2.66320417e-39, 2.11228928e-39, -nan(0x2fff6), -nan(0x19ffd6)]
[-2.45903224e+38, -3.13694095e+38, -snan(0xdff85), -nan(0xbffb4), -snan(0x11ffba), -2.83122967e+38, -2.41915276e+38, -2.67170487e+38, -2.83122724e+38, -2.73817357e+38, -2.39257084e+38, -1.50864781e+38, -1.34249056e+38, -1.71464408e+38, -2.53878186e+38, -2.48561761e+38, -2.64512051e+38, -3.19011393e+38, -snan(0x12ff96), -snan(0x1bffb7)]

问题是我如何以“正常”AVAudioPCMBuffer 的方式访问CMSampleBuffer 样本？

FloatChannelData 或者我在这里做错了什么？

Answer 1

当我创建和 AVCaptureAudioDataOutput 时，我得到的样本是 Int16(不确定您是否可以配置Floats)，当打印出你修改后的代码时，它们非常有意义:

let intPointer = data?.bindMemory(to: Int16.self, capacity: 1024)

音频 Int16 是否在任何地方记录？如果不是，您始终可以通过查看其 AudioStreamBasicDescription

来检查示例缓冲区的格式

let asbd = CMAudioFormatDescriptionGetStreamBasicDescription(CMSampleBufferGetFormatDescription(sampleBuffer)!)!.pointee

或者开始使用 swift 4 以获得稍微更好的可读性

let asbd = sampleBuffer.formatDescription!.audioStreamBasicDescription!.pointee