ios - CMSampleBuffer 每次都会自动旋转

Question

在这里，我尝试从照片库中选择视频并将其作为samplebuffer逐帧读取，以便稍后可以裁剪或旋转。但问题是CMSampleBuffer默认旋转。我用于初始化的变量是

 var asset:AVAsset!     //load asset from url
 var assetReader:AVAssetReader!   
 var assetVideoOutput: AVAssetReaderTrackOutput!  // add assetreader for video
 var assetAudioOutput: AVAssetReaderTrackOutput!  // add assetreader for audio
 var readQueue: DispatchQueue!

之前变量的设置如下所示。

func resetRendering() {

    do{
        assetReader = try! AVAssetReader(asset: asset)
    }catch {
        print(error)
    }

    var tracks = asset.tracks(withMediaType: .video)
    var track : AVAssetTrack!
    if tracks.count > 0 {
        track = tracks[0]
    }

    let decompressionvideoSettings:[String:Any] = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]

    assetVideoOutput = AVAssetReaderTrackOutput(track: track, outputSettings: decompressionvideoSettings)

    tracks = asset.tracks(withMediaType: .audio)
    if tracks.count > 0 {
        track = tracks[0]
    }

    let audioReadSettings = [AVFormatIDKey: kAudioFormatLinearPCM]
    assetAudioOutput = AVAssetReaderTrackOutput(track: track, outputSettings: audioReadSettings)

    if (assetAudioOutput.responds(to: #selector(getter: AVAssetReaderOutput.alwaysCopiesSampleData))) {
        assetAudioOutput.alwaysCopiesSampleData = false
    }
    if assetReader.canAdd(assetAudioOutput) {
        assetReader.add(assetAudioOutput)
    }

    if (assetVideoOutput.responds(to: #selector(getter: AVAssetReaderOutput.alwaysCopiesSampleData))) {
        assetVideoOutput.alwaysCopiesSampleData = false
    }
    if assetReader.canAdd(assetVideoOutput) {
        assetReader.add(assetVideoOutput)
    }

}

现在，当我尝试逐帧读取视频并将其转换为图像时，它会自动将帧旋转 90 度。从 samplebuffer 到 uiimage 的转换扩展如下所示

extension CMSampleBuffer {
    var uiImage: UIImage? {
        guard let imageBuffer = CMSampleBufferGetImageBuffer(self) else { return nil }

        CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags(rawValue: 0))
        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
        let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
        let width = CVPixelBufferGetWidth(imageBuffer)
        let height = CVPixelBufferGetHeight(imageBuffer)
        let colorSpace = CGColorSpaceCreateDeviceRGB()
        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue)
        guard let context = CGContext(data: baseAddress,
                                      width: width,
                                      height: height,
                                      bitsPerComponent: 8,
                                      bytesPerRow: bytesPerRow,
                                      space: colorSpace,
                                      bitmapInfo: bitmapInfo.rawValue) else { return nil }
        guard let cgImage = context.makeImage() else { return nil }

        CVPixelBufferUnlockBaseAddress(imageBuffer,CVPixelBufferLockFlags(rawValue: 0));

        return UIImage(cgImage: cgImage)
    }
} 

结果照片是

实际照片是这张

Answer 1

更新

CGImage 不具有方向属性。所以转换为UIImage时必须手动设置。我们从视频轨道获取方向并将其设置为 UIImage。

下面的答案的右/左或上/下可能是相反的。请尝试检查。

//pls get orientation property outside of image processing to avoid redundancy
let videoTrack = videoAsset.tracks(withMediaType: AVMediaType.video).first!
let videoSize = videoTrack.naturalSize
let transform = videoTrack.preferredTransform

var orientation = UIImage.Orientation.right
switch (transform.tx, transform.ty) {
case (0, 0): orientation = UIImage.Orientation.up
case (videoSize.width, videoSize.height): UIImage.Orientation.right
case (0, videoSize.width): UIImage.Orientation.left
default: orientation = UIImage.Orientation.down
}


// then set orientation in image processing such as captureOutput
let uiImage = UIImage.init(cgImage: cgImage, scale: 1.0, orientation: orientation)

希望这有帮助。