swift - 由于缓冲区不足，从 AVCaptureDataOutputSynchronizerDelegate 中适当释放缓冲区

Question

我正在使用 AVCaptureDataOutputSynchronizerDelegate 来处理视频、深度和元数据的捕获数据

    private let videoDataOutput = AVCaptureVideoDataOutput()
    private let depthDataOutput = AVCaptureDepthDataOutput()
    private let metadataOutput = AVCaptureMetadataOutput()

因此，使用下面的代码，我能够在 AVCaptureDataOutputSynchronizerDelegate 使用的委托(delegate)方法中获取特定的视频数据。

func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection) {

    guard let syncedVideoData = synchronizedDataCollection.synchronizedData(for: self.videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { return }

问题是，当我尝试将 videoData 保存到如下数组中时，我得到了 OutOfBuffers错误。如果我尝试保存视频数据/相关图像/与此数据相关的任何内容，此问题仍然存在。

let array:[CMSampleBuffer] = []

...

array.append(syncedVideoData)
//Gets to about 5-6 sets of data, then it runs out of buffers. 
//I think the buffer is being retained permanently since I am saving to a global variable here.
//Leading to out of buffer error

所以，我想发生的事情是，因为我将任何相关数据保存到一个数组中，它会将数据保留在内存中的缓冲区中，而它通常会被释放。

webpage linked earlier for OutOfBuffers表示我可以

If you need to perform extended processing of captured data, copy that data into buffers whose lifetimes you manage instead of relying on buffers vended by the capture output.

我试图创建一个新的 CMSampleBuffer

extension VideoCapture: AVCaptureDataOutputSynchronizerDelegate {

func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection) {

    var newData:CMSampleBuffer?

    guard let syncedVideoData = synchronizedDataCollection.synchronizedData(for: self.videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { return }
    guard !syncedVideoData.sampleBufferWasDropped else {
        print(syncedVideoData.droppedReason.rawValue)
        return
    }
    let videoSampleBuffer = syncedVideoData.sampleBuffer

    CMSampleBufferCreateCopy(allocator: kCFAllocatorDefault, sampleBuffer: videoSampleBuffer, sampleBufferOut: &newData)
    if(newData != nil) {
        self.buffer.append(newData!)
    }
}

但这会导致相同的问题——videoData 仍保留在缓冲区中。我获得了大约 5-6 组 videoData，然后就没有更多的数据了。

有关如何“将数据复制到您管理其生命周期的缓冲区，而不是依赖于捕获输出提供的缓冲区”的任何指导。，如 the outOfBuffers website 中所示？

Answer 1

我能够在 this buffer 之后创建缓冲区和 this guide以及 Apple 文档中的其他一些内容。

...
guard let imagePixelBuffer = CMSampleBufferGetImageBuffer(videoSampleBuffer) else { fatalError() }


//First lock buffer
CVPixelBufferLockBaseAddress(imagePixelBuffer,
                                     CVPixelBufferLockFlags.readOnly)

//Do something with buffer
self.buffer = createMyBuffer(pixelBuffer: imagePixelBuffer)

//Unlock buffer
CVPixelBufferUnlockBaseAddress(imagePixelBuffer,
                                       CVPixelBufferLockFlags.readOnly)
self.doSomething(self.buffer)

...

func createMyBuffer(pixelBuffer: CVPixelBuffer) -> CVPixelBuffer? {
    let scaleWidth:Int = CVPixelBufferGetWidth(pixelBuffer)
    let scaleHeight:Int = CVPixelBufferGetHeight(pixelBuffer)

    let flags = CVPixelBufferLockFlags(rawValue: 0)
    guard kCVReturnSuccess == CVPixelBufferLockBaseAddress(pixelBuffer, flags) else {
        return nil
    }

    defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, flags) }

    guard let srcData = CVPixelBufferGetBaseAddress(pixelBuffer) else {
        print("Error: could not get pixel buffer base address")
        return nil
    }

    let srcBytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
    var srcBuffer = vImage_Buffer(data: srcData,
                                      height: vImagePixelCount(CVPixelBufferGetHeight(pixelBuffer)),
                                      width: vImagePixelCount(CVPixelBufferGetWidth(pixelBuffer)),
                                      rowBytes: srcBytesPerRow)

    let destBytesPerRow = scaleWidth*4
    guard let destData = malloc(scaleHeight*destBytesPerRow) else {
        print("Error: out of memory")
        return nil
    }

    var destBuffer = vImage_Buffer(data: destData,
                                       height: vImagePixelCount(scaleHeight),
                                       width: vImagePixelCount(scaleWidth),
                                       rowBytes: destBytesPerRow)

    let error = vImageScale_ARGB8888(&srcBuffer, &destBuffer, nil, vImage_Flags(kvImageLeaveAlphaUnchanged))
    if error != kvImageNoError {
        print("Error:", error)
        free(destData)
        return nil
    }

    let releaseCallback: CVPixelBufferReleaseBytesCallback = { _, ptr in
        if let ptr = ptr {
            free(UnsafeMutableRawPointer(mutating: ptr))
        }
    }

    let pixelFormat = CVPixelBufferGetPixelFormatType(pixelBuffer)
    var dstPixelBuffer: CVPixelBuffer?
    let status = CVPixelBufferCreateWithBytes(nil, scaleWidth, scaleHeight,
                                                  pixelFormat, destData,
                                                  destBytesPerRow, releaseCallback,
                                                  nil, nil, &dstPixelBuffer)
    if status != kCVReturnSuccess {
        print("Error: could not create new pixel buffer")
        free(destData)
        return nil
    }
    return dstPixelBuffer
}

这有效 - 但似乎多余。我正在使用一个函数，我发现它可以“缩放”缓冲区，但我只是将它缩放到与当前缓冲区完全相同的大小，它会返回一个新的缓冲区，当我选择删除它时。它是重复的，但功能有效。