ios - 将非常大的图像缩小到 MTLTexture

Question

我关注了this example 从 Apple 缩小非常大的图像我是 downloading from a remote location .我已经用 Swift 重写了代码。它显然有效，但是当我调用 MTKTextureLoader.newTexture 时，应用程序崩溃并在 _loadCGImage 中出现 EXC_BAD_ACCESS。没有其他提示，但我怀疑图像数据已经发布或其他...

任何提示为什么它会在没有任何正确错误消息的情况下崩溃？

这是顶层代码，

// this is an extension of MTKTextureLoader
// [...]
if let uiImage = UIImage(contentsOfFile: cachedFileURL.path) {
    let maxDim : CGFloat = 8192
    if uiImage.size.width > maxDim || uiImage.size.height > maxDim {
        let scale = uiImage.size.width > maxDim ? maxDim / uiImage.size.width : maxDim / uiImage.size.height
        if let cgImage = MTKTextureLoader.downsize(image: uiImage, scale: scale) {
            return self.newTexture(with: cgImage, options: options, completionHandler: completionHandler)
        } else {
            anError = TextureError.CouldNotDownsample
        }
    } else {
        return self.newTexture(withContentsOf: cachedFileURL, options: options, completionHandler: completionHandler)
    }
}

这是缩小尺寸的方法，

private static func downsize(image: UIImage, scale: CGFloat) -> CGImage? {
    let destResolution = CGSize(width: Int(image.size.width * scale), height: Int(image.size.height * scale))
    let kSourceImageTileSizeMB : CGFloat = 40.0 // The tile size will be (x)MB of uncompressed image data
    let pixelsPerMB = 262144
    let tileTotalPixels = kSourceImageTileSizeMB * CGFloat(pixelsPerMB)
    let destSeemOverlap : CGFloat = 2.0 // the numbers of pixels to overlap the seems where tiles meet.
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)
    guard let destContext = CGContext(data: nil, width: Int(destResolution.width), height: Int(destResolution.height), bitsPerComponent: 8, bytesPerRow: 0, space: colorSpace, bitmapInfo: bitmapInfo.rawValue) else {
        NSLog("failed to create the output bitmap context!")
        return nil
    }
    var sourceTile = CGRect()
    sourceTile.size.width = image.size.width
    sourceTile.size.height = floor( tileTotalPixels / sourceTile.size.width )
    print("source tile size: \(sourceTile.size)")
    sourceTile.origin.x = 0.0
    // the output tile is the same proportions as the input tile, but
    // scaled to image scale.
    var destTile = CGRect()
    destTile.size.width = destResolution.width
    destTile.size.height = sourceTile.size.height * scale
    destTile.origin.x = 0.0
    print("dest tile size: \(destTile.size)")
    // the source seem overlap is proportionate to the destination seem overlap.
    // this is the amount of pixels to overlap each tile as we assemble the output image.
    let sourceSeemOverlap : CGFloat = floor( ( destSeemOverlap / destResolution.height ) * image.size.height )
    print("dest seem overlap: \(destSeemOverlap), source seem overlap: \(sourceSeemOverlap)")
    // calculate the number of read/write operations required to assemble the
    // output image.
    var iterations = Int( image.size.height / sourceTile.size.height )
    // if tile height doesn't divide the image height evenly, add another iteration
    // to account for the remaining pixels.
    let remainder = Int(image.size.height) % Int(sourceTile.size.height)
    if remainder > 0 {
        iterations += 1
    }
    // add seem overlaps to the tiles, but save the original tile height for y coordinate calculations.
    let sourceTileHeightMinusOverlap = sourceTile.size.height
    sourceTile.size.height += sourceSeemOverlap
    destTile.size.height += destSeemOverlap
    print("beginning downsize. iterations: \(iterations), tile height: \(sourceTile.size.height), remainder height: \(remainder)")
    for y in 0..<iterations {
        // create an autorelease pool to catch calls to -autorelease made within the downsize loop.
        autoreleasepool {
            print("iteration \(y+1) of \(iterations)")
            sourceTile.origin.y = CGFloat(y) * sourceTileHeightMinusOverlap + sourceSeemOverlap
            destTile.origin.y = ( destResolution.height ) - ( CGFloat( y + 1 ) * sourceTileHeightMinusOverlap * scale + destSeemOverlap )
            // create a reference to the source image with its context clipped to the argument rect.
            if let sourceTileImage = image.cgImage?.cropping( to: sourceTile ) {
                // if this is the last tile, its size may be smaller than the source tile height.
                // adjust the dest tile size to account for that difference.
                if  y == iterations - 1 && remainder > 0 {
                    var dify = destTile.size.height
                    destTile.size.height = CGFloat( sourceTileImage.height ) * scale
                    dify -= destTile.size.height
                    destTile.origin.y += dify
                }
                // read and write a tile sized portion of pixels from the input image to the output image.
                destContext.draw(sourceTileImage, in: destTile, byTiling: false)
            }
            // !!! In the original LargeImageDownsizing code, it released the source image here !!!
            // [image release];
            // !!! I assume I don't need to do that in Swift?? !!!
            /* while CGImageCreateWithImageInRect lazily loads just the image data defined by the argument rect,
             that data is finally decoded from disk to mem when CGContextDrawImage is called. sourceTileImageRef
             maintains internally a reference to the original image, and that original image both, houses and
             caches that portion of decoded mem. Thus the following call to release the source image. */
            // http://en.swifter.tips/autoreleasepool/
            // drain will be called
            // to free all objects that were sent -autorelease within the scope of this loop.
        }
        // !!! Original code reallocated the image here !!!
        // we reallocate the source image after the pool is drained since UIImage -imageNamed
        // returns us an autoreleased object.
    }
    print("downsize complete.")
    // create a CGImage from the offscreen image context
    return destContext.makeImage()
}

编辑:

每次我尝试用 CGImage 初始化 MTLTexture 时它都会崩溃，即使图像很小并且适合内存。所以它似乎与调整大小无关......这段代码也会崩溃，

func newTexture(with uiImage: UIImage, options: [String : NSObject]? = nil, completionHandler: @escaping MTKTextureLoaderCallback) {
    if let cgImage = uiImage.cgImage {
        return self.newTexture(with: cgImage, options: options, completionHandler: completionHandler)
    } else {
        completionHandler(nil, TextureError.CouldNotBeCreated)
    }
}

在 ImageIO copyImageBlockSetWithOptions。

编辑2:基于 warrenm 答案的解决方法:调用 newTexture(with: cgImage) 同步而不是异步。例如，上面的函数变成了，

func newTexture(with uiImage: UIImage, options: [String : NSObject]? = nil, completionHandler: MTKTextureLoaderCallback) {
    if let cgImage = uiImage.cgImage {
        let tex = try? self.newTexture(with: cgImage, options: options)
        completionHandler(tex, tex == nil ? TextureError.CouldNotBeCreated : nil)
    } else {
        completionHandler(nil, TextureError.CouldNotGetCGImage)
    }
}

(请注意，我已经删除了 @escaping，因为现在我实际上调用了 sync 方法。)

缩小代码是正确的。它现在可以使用此解决方法:)

Answer 1

这似乎是用于创建纹理的 CGImage 生命周期的问题，也可能是 MetalKit 中的错误。

本质上，上下文返回的 CGImage 只能保证在创建它的自动释放池范围结束之前一直有效。当您调用异步 newTexture 方法时，MTKTextureLoader 将创建纹理的工作移至后台线程，并在 CGImage 外部进行操作在其后备存储已被释放后，其封闭自动释放池的范围。

您可以通过在完成处理程序中捕获图像(这将导致 ARC 延长其生命周期)或使用相应的同步纹理创建方法 newTexture(with:options:)，它将保留在相关范围内，直到纹理完全初始化。