android - 如何在 PyTorch Mobile 上将张量转换为位图？

Question

我找到了解决方案 ( https://itnext.io/converting-pytorch-float-tensor-to-android-rgba-bitmap-with-kotlin-ffd4602a16b6 )，但是当我尝试以这种方式转换时，我发现 inputTensor.dataAsFloatArray 的大小超过了 bitmap.width*bitmap.height。如何将张量转换为 float 组，或者是否有任何其他可能的方法将 pytorch 张量转换为位图？

val inputTensor = TensorImageUtils.bitmapToFloat32Tensor(
    bitmap,
    TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB
)

// Float array size is 196608 when width and height are 256x256 = 65536

val res = floatArrayToGrayscaleBitmap(inputTensor.dataAsFloatArray, bitmap.width, bitmap.height)


fun floatArrayToGrayscaleBitmap (
    floatArray: FloatArray,
    width: Int,
    height: Int,
    alpha :Byte = (255).toByte(),
    reverseScale :Boolean = false
) : Bitmap {

    // Create empty bitmap in RGBA format (even though it says ARGB but channels are RGBA)
    val bmp = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
    val byteBuffer = ByteBuffer.allocate(width*height*4)
    Log.d("App", floatArray.size.toString() + " " + (width * height * 4).toString())

    // mapping smallest value to 0 and largest value to 255
    val maxValue = floatArray.max() ?: 1.0f
    val minValue = floatArray.min() ?: 0.0f
    val delta = maxValue-minValue
    var tempValue :Byte

    // Define if float min..max will be mapped to 0..255 or 255..0
    val conversion = when(reverseScale) {
        false -> { v: Float -> ((v-minValue)/delta*255).toByte() }
        true -> { v: Float -> (255-(v-minValue)/delta*255).toByte() }
    }

    // copy each value from float array to RGB channels and set alpha channel
    floatArray.forEachIndexed { i, value ->
        tempValue = conversion(value)
        byteBuffer.put(4*i, tempValue)
        byteBuffer.put(4*i+1, tempValue)
        byteBuffer.put(4*i+2, tempValue)
        byteBuffer.put(4*i+3, alpha)
    }

    bmp.copyPixelsFromBuffer(byteBuffer)

    return bmp
}

Answer 1

没有一个答案能够产生我想要的输出，所以这就是我想出的 - 它基本上只是 TensorImageUtils.bitmapToFloat32Tensor() 中发生的事情的逆向工程版本。

请注意，仅当您在 TensorImageUtils.bitmapToFloat32Tensor() 中使用 MemoryFormat.CONTIGUOUS(默认)时，此函数才有效。

fun tensor2Bitmap(input: FloatArray, width: Int, height: Int, normMeanRGB: FloatArray, normStdRGB: FloatArray): Bitmap? {
    val pixelsCount = height * width
    val pixels = IntArray(pixelsCount)
    val output = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
    
    val conversion = { v: Float -> ((v.coerceIn(0.0f, 1.0f))*255.0f).roundToInt()}

    val offset_g = pixelsCount
    val offset_b = 2 * pixelsCount
    for (i in 0 until pixelsCount) {
        val r = conversion(input[i] * normStdRGB[0] + normMeanRGB[0])
        val g = conversion(input[i + offset_g] * normStdRGB[1] + normMeanRGB[1])
        val b = conversion(input[i + offset_b] * normStdRGB[2] + normMeanRGB[2])
        pixels[i] = 255 shl 24 or (r.toInt() and 0xff shl 16) or (g.toInt() and 0xff shl 8) or (b.toInt() and 0xff)
    }
    output.setPixels(pixels, 0, width, 0, 0, width, height)
    return output
}

示例用法如下:

tensor2Bitmap(outputTensor.dataAsFloatArray, bitmap.width, bitmap.height, TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB)