ios - 如何在 MPSNNGraph 的输入处使用两个 MPSImage

Question

我正在尝试在 MPSNNGraph 上输入两个图像。

但是，即使我在“withSourceImages”上输入像[input1, input2]这样的数组，我也只能输入“input1”作为输入图像。理想情况下，当创建如下图时，我想将“inputImage1”设置为“input1”，将“inputImage2”设置为“input2”。

实际上，当我像这样运行它并查看“concat”的结果时，我能够看到连接的是“input1”，而不是“input2”。

图表如下:

let inputImage1 = MPSNNImageNode(handle: nil)
let inputImage2 = MPSNNImageNode(handle: nil)
let scale = MPSNNBilinearScaleNode(source: inputImage1,
                                   outputSize: MTLSize(width:256,
                                                       height: 256,
                                                       depth: 3))
let scale2 = MPSNNBilinearScaleNode(source: inputImage1,
                                   outputSize: MTLSize(width:64,
                                                       height: 64,
                                                       depth: 3))
...

let concat = MPSNNConcatenationNode(sources: [conv3.resultImage, scale2.resultImage])

...

if let graph = MPSNNGraph(device: commandQueue.device,
                          resultImage: tanh.resultImage,
                          resultImageIsNeeded: true){
        self.graph = graph
}

编码图的一部分如下所示:

let input1 = MPSImage(texture: texture, ...)
let input2 = MPSImage(texture: texture2, ...)
graph.executeAsync(withSourceImages: [input1, input2]) { outputImage, error in
    ...
}

如何输入第二个输入并让图表接收它？

你能给我一些建议吗？

Answer 1

您提供的代码实际上看起来是正确的。在此处引用 MPSNNGraph.h header :

*  @param  sourceImages    A list of MPSImages to use as the source images for the graph.
*                          These should be in the same order as the list returned from
*                          MPSNNGraph.sourceImageHandles. They should be allocated against
*                          the same MTLDevice. There must be at least one source image.
*                          Note: this array is intended to handle the case where multiple
*                          input images are required to generate a single graph result.
*                          That is, the graph itself has multiple inputs.  If you need to
*                          execute the graph multiple times, then call this API multiple
*                          times, or better yet use [encodeToCommandBuffer:sourceImages:]
*                          multiple times.

但我想指出，MPSNNConcatenationNode 的行为方式非常独特。它总是在深度( channel )维度上连接。当连接不同空间尺寸的图像时，它将尊重较小的图像(即 2x2x10 连接 4x4x15 -> 2x2x25)。也许这就是您的问题的根源。