swift - 添加覆盖时捕获的视频非常滞后

Question

我正在使用 AVAssetWriter 构建一个应用程序，我在其中向视频添加叠加层。当我不尝试添加覆盖时，它工作得很好。但是当我添加叠加层时，视频看起来被裁掉了一半(如您在屏幕截图中所见)。

这是我的 addOverlayToImage 函数:

func addOverlayToImage(from filteredImage: UIImage) -> UIImage {
   UIGraphicsBeginImageContextWithOptions(self.imageView.frame.size, false, 1.0);
   self.imageView.layer.render(in: UIGraphicsGetCurrentContext()!)
   let imageWithText = UIGraphicsGetImageFromCurrentImageContext()
   UIGraphicsEndImageContext();
   return imageWithText! 
}

我在 captureOutput 中调用函数:

func captureOutput(_ captureOutput: AVCaptureOutput, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection) {
   self.bufferVideoQueue.async() {
     let imageWithOverlay = self.addOverlayToImage(from: self.filteredImage)
     let buffer = self.imageToBuffer(from: imageWithOverlay)
     self.assetWriterPixelBufferInput?.append(buffer!, withPresentationTime: self.currentTime)
   }
}

和 imageToBuffer 函数:

func imageToBuffer(from image: UIImage) -> CVPixelBuffer? {
    let attrs = [
        String(kCVPixelBufferCGImageCompatibilityKey) : kCFBooleanTrue,
        String(kCVPixelBufferCGBitmapContextCompatibilityKey) : kCFBooleanTrue
    ] as [String : Any]
    var buffer : CVPixelBuffer?
    let status = CVPixelBufferCreate(kCFAllocatorDefault, Int(image.size.width), Int(image.size.height), kCVPixelFormatType_32ARGB, attrs as CFDictionary, &buffer)
    guard (status == kCVReturnSuccess) else {
        return nil
    }

    CVPixelBufferLockBaseAddress(buffer!, CVPixelBufferLockFlags(rawValue: 0))
    let pixelData = CVPixelBufferGetBaseAddress(buffer!)

    let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
    let context = CGContext(data: pixelData, width: Int(image.size.width), height: Int(image.size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(buffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)

    context?.translateBy(x: 0, y: image.size.height)
    context?.scaleBy(x: 1.0, y: -1.0)

    UIGraphicsPushContext(context!)
    image.draw(in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height))
    UIGraphicsPopContext()
    CVPixelBufferUnlockBaseAddress(buffer!, CVPixelBufferLockFlags(rawValue: 0))

    return buffer
}

还有视频截图:

Answer 1

您是否尝试过配置 AVAssetWriterInput 以指定数据是实时的(将 expectsMediaDataInRealTime 设置为 true)？这在写入实时(实时摄像头)数据时会产生很大的不同，如果设置不当，可能会导致输出滞后。

https://developer.apple.com/documentation/avfoundation/avassetwriterinput

https://developer.apple.com/documentation/avfoundation/avassetwriterinput/1387827-expectsmediadatainrealtime