ios - Metal View (MTKView) 绘图尺寸问题

Question

这里我有一个 MTKView 并在摄像头源上实时运行一个简单的 CIFilter。这很好用。

问题

在旧设备的自拍相机上，例如 iPhone 5、iPad Air，提要绘制在较小的区域。 更新:发现发生这种情况时，馈送到 MTKView 的 CMSampleBuffer 尺寸较小。我想每次更新中的纹理都需要按比例放大？

---

 import UIKit
 import MetalPerformanceShaders
 import MetalKit
 import AVFoundation

    final class MetalObject: NSObject, MTKViewDelegate {

        private var metalBufferView            : MTKView?
        private var metalDevice                = MTLCreateSystemDefaultDevice()
        private var metalCommandQueue          : MTLCommandQueue!
        private var metalSourceTexture         : MTLTexture?
        private var context                    : CIContext?
        private var filter                     : CIFilter?


        init(with frame: CGRect, filterType: Int, scaledUp: Bool) {
            super.init()

            self.metalCommandQueue = self.metalDevice!.makeCommandQueue()
            self.metalBufferView = MTKView(frame: frame, device: self.metalDevice)
            self.metalBufferView!.framebufferOnly = false
            self.metalBufferView!.isPaused = true

            self.metalBufferView!.contentScaleFactor = UIScreen.main.nativeScale
            self.metalBufferView!.delegate = self
            self.context = CIContext()


        }


        final func update (sampleBuffer: CMSampleBuffer) {

            var textureCache : CVMetalTextureCache?
            CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, self.metalDevice!,  nil, &textureCache)
            var cameraTexture: CVMetalTexture?

            guard
                let cameraTextureCache = textureCache,
                let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
                    return
            }

            let cameraTextureWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 0)
            let cameraTextureHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 0)
            CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                      cameraTextureCache,
                                                      pixelBuffer,
                                                      nil,
                                                      MTLPixelFormat.bgra8Unorm,
                                                      cameraTextureWidth,
                                                      cameraTextureHeight,
                                                      0,
                                                      &cameraTexture)

            if let cameraTexture = cameraTexture,
                let metalTexture = CVMetalTextureGetTexture(cameraTexture) {
                self.metalSourceTexture = metalTexture
                self.metalBufferView!.draw()
            }


        }

        //MARK: - Metal View Delegate
        final func draw(in view: MTKView) {

            guard let currentDrawable = self.metalBufferView!.currentDrawable,
                let sourceTexture = self.metalSourceTexture
                else {  return  }

            let commandBuffer = self.metalCommandQueue!.makeCommandBuffer()
            var inputImage = CIImage(mtlTexture: sourceTexture)!.applyingOrientation(self.orientationNumber)

            if self.showFilter {
                self.filter!.setValue(inputImage, forKey: kCIInputImageKey)
                inputImage = filter!.outputImage!
            }


            self.context!.render(inputImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: inputImage.extent, colorSpace: self.colorSpace!)

            commandBuffer.present(currentDrawable)
            commandBuffer.commit()

        }


        final func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {

        }
    }

---

观察

• 只发生在旧设备的自拍相机上
• 较新设备上的自拍相机没问题出现问题时，新内容会绘制在较小的区域(偏向左上角)，后置摄像头的旧内容仍保留在新内容之外。
• Metal View 的约束和大小/位置很好。
• self.metalBufferView!.contentScaleFactor = UIScreen.main.nativeScale解决了 Plus 设备上奇怪的缩放问题。

Answer 1

看起来旧设备上前置(自拍)摄像头的分辨率较低，因此如果您想让视频使用全宽或全高，则需要按比例放大视频。由于您已经在使用 CIContext 和 Metal，您可以简单地指示渲染调用将图像绘制到您喜欢的任何矩形。

在您的draw 方法中，您执行

self.context!.render(inputImage,
                     to: currentDrawable.texture,
                     commandBuffer: commandBuffer,
                     bounds: inputImage.extent,
                     colorSpace: self.colorSpace!)

bounds 参数是将在其中呈现图像的目标 矩形。当前，您正在使用图像范围，这意味着图像不会被缩放。

要放大视频，请改用显示矩形。您可以简单地使用您的 metalBufferView.bounds 因为这将是您的显示 View 的大小。你最终会得到

 self.context!.render(inputImage,
                      to: currentDrawable.texture,
                      commandBuffer: commandBuffer,
                      bounds: self.metalBufferView.bounds,
                      colorSpace: self.colorSpace!)

如果图像和 View 具有不同的纵横比(宽度/高度是纵横比)，那么您必须计算正确的尺寸以保持图像的纵横比。为此，您最终会得到如下代码:

CGRect dest = self.metalBufferView.bounds;
CGSize imageSize = inputImage.extent.size;
CGSize viewSize = dest.size; 
double imageAspect = imageSize.width / imageSize.height;
double viewAspect = viewSize.width / viewSize.height;
if (imageAspect > viewAspect) {
    // the image is wider than the view, adjust height
    dest.size.height = 1/imageAspect * dest.size.width;
} else {
    // the image is taller than the view, adjust the width
    dest.size.width = imageAspect * dest.size.height;

    // center the tall image
    dest.origin.x = (viewSize.width - dest.size.width) / 2;
}

希望这有用，如果有任何问题或澄清会有所帮助，请告诉我。