avfoundation - 如何从具有正确方向的 AVCapturePhoto 生成 UIImage？

Question

我调用 AVFoundation的委托(delegate)方法来处理照片捕获，但我在转换 AVCapturePhoto 时遇到了困难它生成一个UIImage方向正确。虽然下面的例程是成功的，但我总是得到一个右向的UIImage (UIImage.imageOrientation = 3)。使用 UIImage(data: image) 时，我无法提供方向。并尝试首先使用 photo.cgImageRepresentation()?.takeRetainedValue()也无济于事。请协助。

图像方向在这里至关重要，因为生成的图像将被输入到 Vision Framework 工作流程中。

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    // capture image finished
    print("Image captured.")
    if let imageData = photo.fileDataRepresentation() {
        if let uiImage = UIImage(data: imageData){
            // do stuff to UIImage
        }
    }
}

更新 1:
阅读苹果的 Photo Capture Programming Guide (对于 iOS11 已过时)，我确实设法找到了我做错的一件事:

在每次捕获调用 (self.capturePhotoOutput.capturePhoto) 时，都必须建立与 PhotoOutput 的连接。对象并更新其方向以匹配拍摄照片时设备的方向。为此，我创建了 UIDeviceOrientation 的扩展名。并在 snapPhoto() 上使用它我创建的函数用于调用捕获例程并等待 didFinishProcessingPhoto要执行的委托(delegate)方法。我添加了代码的快照，因为这里的代码示例分隔符似乎没有正确显示它们。

更新 2
GitHub 上完整项目的链接: https://github.com/agu3rra/Out-Loud

Answer 1

最后更新:
我对该应用程序进行了一些实验，得出以下结论:

kCGImagePropertyOrientation似乎不会影响应用程序内捕获图像的方向，如果您更新 photoOutput，它只会随设备方向而变化每次你要调用capturePhoto方法。所以:

func snapPhoto() {
    // prepare and initiate image capture routine

    // if I leave the next 4 lines commented, the intented orientation of the image on display will be 6 (right top) - kCGImagePropertyOrientation
    let deviceOrientation = UIDevice.current.orientation // retrieve current orientation from the device
    guard let photoOutputConnection = capturePhotoOutput.connection(with: AVMediaType.video) else {fatalError("Unable to establish input>output connection")}// setup a connection that manages input > output
    guard let videoOrientation = deviceOrientation.getAVCaptureVideoOrientationFromDevice() else {return}
    photoOutputConnection.videoOrientation = videoOrientation // update photo's output connection to match device's orientation

    let photoSettings = AVCapturePhotoSettings()
    photoSettings.isAutoStillImageStabilizationEnabled = true
    photoSettings.isHighResolutionPhotoEnabled = true
    photoSettings.flashMode = .auto
    self.capturePhotoOutput.capturePhoto(with: photoSettings, delegate: self) // trigger image capture. It appears to work only if the capture session is running.
}

在调试器上查看生成的图像向我展示了它们是如何生成的，因此我可以推断出所需的旋转 (UIImageOrientation) 以使其垂直显示。换句话说:更新 UIImageOrientation告诉您应该如何旋转图像以便您以正确的方向看到它。于是我来到了下表:

我必须更新我的 UIDeviceOrientation扩展到一个相当不直观的形式:

extension UIDeviceOrientation {
    func getUIImageOrientationFromDevice() -> UIImageOrientation {
        // return CGImagePropertyOrientation based on Device Orientation
        // This extented function has been determined based on experimentation with how an UIImage gets displayed.
        switch self {
        case UIDeviceOrientation.portrait, .faceUp: return UIImageOrientation.right
        case UIDeviceOrientation.portraitUpsideDown, .faceDown: return UIImageOrientation.left
        case UIDeviceOrientation.landscapeLeft: return UIImageOrientation.up // this is the base orientation
        case UIDeviceOrientation.landscapeRight: return UIImageOrientation.down
        case UIDeviceOrientation.unknown: return UIImageOrientation.up
        }
    }
}

这就是我的最终委托(delegate)方法现在的样子。它以预期的方向显示图像。

func photoOutput(_ output: AVCapturePhotoOutput,
                                 didFinishProcessingPhoto photo: AVCapturePhoto,
                                 error: Error?)
{
    // capture image finished
    print("Image captured.")

    let photoMetadata = photo.metadata
    // Returns corresponting NSCFNumber. It seems to specify the origin of the image
    //                print("Metadata orientation: ",photoMetadata["Orientation"])

    // Returns corresponting NSCFNumber. It seems to specify the origin of the image
    print("Metadata orientation with key: ",photoMetadata[String(kCGImagePropertyOrientation)] as Any)

    guard let imageData = photo.fileDataRepresentation() else {
        print("Error while generating image from photo capture data.");
        self.lastPhoto = nil; self.controller.goToProcessing();
        return

    }

    guard let uiImage = UIImage(data: imageData) else {
        print("Unable to generate UIImage from image data.");
        self.lastPhoto = nil; self.controller.goToProcessing();
        return

    }

    // generate a corresponding CGImage
    guard let cgImage = uiImage.cgImage else {
        print("Error generating CGImage");self.lastPhoto=nil;return

    }

    guard let deviceOrientationOnCapture = self.deviceOrientationOnCapture else {
        print("Error retrieving orientation on capture");self.lastPhoto=nil;
        return

    }

    self.lastPhoto = UIImage(cgImage: cgImage, scale: 1.0, orientation: deviceOrientationOnCapture.getUIImageOrientationFromDevice())

    print(self.lastPhoto)
    print("UIImage generated. Orientation:(self.lastPhoto.imageOrientation.rawValue)")
    self.controller.goToProcessing()
}


func photoOutput(_ output: AVCapturePhotoOutput, 
                   willBeginCaptureFor resolvedSettings: AVCaptureResolvedPhotoSettings) 
                   {
    print("Just about to take a photo.")
    // get device orientation on capture
    self.deviceOrientationOnCapture = UIDevice.current.orientation
    print("Device orientation: \(self.deviceOrientationOnCapture.rawValue)")
}