android - Camera X 捕捉不同旋转状态下的图像

Question

好吧，我浏览了不同的帖子，发现根据移动制造商的不同，可能会出现捕获图像旋转等复杂情况，因此您必须意识到这一点。我所做的是:

fun rotateBitmap(bitmap: Bitmap): Bitmap? {
    val matrix = Matrix()

    when (getImageOrientation(bitmap)) {
        ExifInterface.ORIENTATION_NORMAL -> return bitmap
        ExifInterface.ORIENTATION_FLIP_HORIZONTAL -> matrix.setScale(-1f, 1f)
        ExifInterface.ORIENTATION_ROTATE_270 -> matrix.setRotate(-90f)
        ExifInterface.ORIENTATION_ROTATE_180 -> matrix.setRotate(180f)
        ExifInterface.ORIENTATION_ROTATE_90 -> matrix.setRotate(90f)
        ExifInterface.ORIENTATION_FLIP_VERTICAL -> {
            matrix.setRotate(180f)
            matrix.postScale(-1f, 1f)
        }
        ExifInterface.ORIENTATION_TRANSPOSE -> {
            matrix.setRotate(90f)
            matrix.postScale(-1f, 1f)
        }
        ExifInterface.ORIENTATION_TRANSVERSE -> {
            matrix.setRotate(-90f)
            matrix.postScale(-1f, 1f)
        }

        else -> return bitmap
}

这成功了。但后来我注意到一些非常奇怪的事情，这可能与我如何配置 Camera X 配置有关。

使用相同的设备，我得到不同旋转的位图(好吧，这不应该发生。如果设备奇怪地旋转图像，它应该在两种模式下旋转图像 - 在 ImageAnalysesUseCase 和 ImageCaptureUseCase).

那么，为什么会发生这种情况，我该如何解决？

代码实现:

将相机 X 绑定(bind)到生命周期:

CameraX.bindToLifecycle(
            this,
            buildPreviewUseCase(),
            buildImageAnalysisUseCase(),
            buildImageCaptureUseCase()
)

预览用例:

private fun buildPreviewUseCase(): Preview {
    val previewConfig = PreviewConfig.Builder()
        .setTargetAspectRatio(config.aspectRatio)
        .setTargetResolution(config.resolution)
        .setTargetRotation(Surface.ROTATION_0)
        .setLensFacing(config.lensFacing)
        .build()

    return AutoFitPreviewBuilder.build(previewConfig, cameraTextureView)
}

捕获用例:

private fun buildImageCaptureUseCase(): ImageCapture {
    val captureConfig = ImageCaptureConfig.Builder()
        .setTargetAspectRatio(config.aspectRatio)
        .setTargetRotation(Surface.ROTATION_0)
        .setTargetResolution(config.resolution)
        .setCaptureMode(config.captureMode)
        .build()

    val capture = ImageCapture(captureConfig)

    manualModeTakePhotoButton.setOnClickListener {


        capture.takePicture(object : ImageCapture.OnImageCapturedListener() {
            override fun onCaptureSuccess(imageProxy: ImageProxy, rotationDegrees: Int) {
                viewModel.onManualCameraModeAnalysis(imageProxy, rotationDegrees)
            }

            override fun onError(useCaseError: ImageCapture.UseCaseError?, message: String?, cause: Throwable?) {
                //
            }
        })
    }

    return capture
}

分析用例:

private fun buildImageAnalysisUseCase(): ImageAnalysis {
    val analysisConfig = ImageAnalysisConfig.Builder().apply {
        val analyzerThread = HandlerThread("xAnalyzer").apply { start() }
        analyzerHandler = Handler(analyzerThread.looper)

        setCallbackHandler(analyzerHandler!!)
        setTargetAspectRatio(config.aspectRatio)
        setTargetRotation(Surface.ROTATION_0)
        setTargetResolution(config.resolution)
        setImageReaderMode(config.readerMode)
        setImageQueueDepth(config.queueDepth)
    }.build()

    val analysis = ImageAnalysis(analysisConfig)
    analysis.analyzer = ImageRecognitionAnalyzer(viewModel)

    return analysis
}

AutoFitPreviewBuilder:

class AutoFitPreviewBuilder private constructor(config: PreviewConfig,
                                            viewFinderRef: WeakReference<TextureView>) {
/** Public instance of preview use-case which can be used by consumers of this adapter */
val useCase: Preview

/** Internal variable used to keep track of the use-case's output rotation */
private var bufferRotation: Int = 0
/** Internal variable used to keep track of the view's rotation */
private var viewFinderRotation: Int? = null
/** Internal variable used to keep track of the use-case's output dimension */
private var bufferDimens: Size = Size(0, 0)
/** Internal variable used to keep track of the view's dimension */
private var viewFinderDimens: Size = Size(0, 0)
/** Internal variable used to keep track of the view's display */
private var viewFinderDisplay: Int = -1

/** Internal reference of the [DisplayManager] */
private lateinit var displayManager: DisplayManager
/**
 * We need a display listener for orientation changes that do not trigger a configuration
 * change, for example if we choose to override config change in manifest or for 180-degree
 * orientation changes.
 */
private val displayListener = object : DisplayManager.DisplayListener {
    override fun onDisplayAdded(displayId: Int) = Unit
    override fun onDisplayRemoved(displayId: Int) = Unit
    override fun onDisplayChanged(displayId: Int) {
        val viewFinder = viewFinderRef.get() ?: return
        if (displayId == viewFinderDisplay) {
            val display = displayManager.getDisplay(displayId)
            val rotation = getDisplaySurfaceRotation(display)
            updateTransform(viewFinder, rotation, bufferDimens, viewFinderDimens)
        }
    }
}

init {
    // Make sure that the view finder reference is valid
    val viewFinder = viewFinderRef.get() ?:
    throw IllegalArgumentException("Invalid reference to view finder used")

    // Initialize the display and rotation from texture view information
    viewFinderDisplay = viewFinder.display.displayId
    viewFinderRotation = getDisplaySurfaceRotation(viewFinder.display) ?: 0

    // Initialize public use-case with the given config
    useCase = Preview(config)

    // Every time the view finder is updated, recompute layout
    useCase.onPreviewOutputUpdateListener = Preview.OnPreviewOutputUpdateListener {
        val viewFinder =
            viewFinderRef.get() ?: return@OnPreviewOutputUpdateListener

        // To update the SurfaceTexture, we have to remove it and re-add it
        val parent = viewFinder.parent as ViewGroup
        parent.removeView(viewFinder)
        parent.addView(viewFinder, 0)

        viewFinder.surfaceTexture = it.surfaceTexture
        bufferRotation = it.rotationDegrees
        val rotation = getDisplaySurfaceRotation(viewFinder.display)
        updateTransform(viewFinder, rotation, it.textureSize, viewFinderDimens)
    }

    // Every time the provided texture view changes, recompute layout
    viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ ->
        val viewFinder = view as TextureView
        val newViewFinderDimens = Size(right - left, bottom - top)
        val rotation = getDisplaySurfaceRotation(viewFinder.display)
        updateTransform(viewFinder, rotation, bufferDimens, newViewFinderDimens)
    }

    // Every time the orientation of device changes, recompute layout
    displayManager = viewFinder.context
        .getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
    displayManager.registerDisplayListener(displayListener, null)

    // Remove the display listeners when the view is detached to avoid
    // holding a reference to the View outside of a Fragment.
    // NOTE: Even though using a weak reference should take care of this,
    // we still try to avoid unnecessary calls to the listener this way.
    viewFinder.addOnAttachStateChangeListener(object : View.OnAttachStateChangeListener {
        override fun onViewAttachedToWindow(view: View?) {
            displayManager.registerDisplayListener(displayListener, null)
        }
        override fun onViewDetachedFromWindow(view: View?) {
            displayManager.unregisterDisplayListener(displayListener)
        }

    })
}

/** Helper function that fits a camera preview into the given [TextureView] */
private fun updateTransform(textureView: TextureView?, rotation: Int?, newBufferDimens: Size,
                            newViewFinderDimens: Size) {
    // This should not happen anyway, but now the linter knows
    val textureView = textureView ?: return

    if (rotation == viewFinderRotation &&
        Objects.equals(newBufferDimens, bufferDimens) &&
        Objects.equals(newViewFinderDimens, viewFinderDimens)) {
        // Nothing has changed, no need to transform output again
        return
    }

    if (rotation == null) {
        // Invalid rotation - wait for valid inputs before setting matrix
        return
    } else {
        // Update internal field with new inputs
        viewFinderRotation = rotation
    }

    if (newBufferDimens.width == 0 || newBufferDimens.height == 0) {
        // Invalid buffer dimens - wait for valid inputs before setting matrix
        return
    } else {
        // Update internal field with new inputs
        bufferDimens = newBufferDimens
    }

    if (newViewFinderDimens.width == 0 || newViewFinderDimens.height == 0) {
        // Invalid view finder dimens - wait for valid inputs before setting matrix
        return
    } else {
        // Update internal field with new inputs
        viewFinderDimens = newViewFinderDimens
    }

    val matrix = Matrix()

    // Compute the center of the view finder
    val centerX = viewFinderDimens.width / 2f
    val centerY = viewFinderDimens.height / 2f

    // Correct preview output to account for display rotation
    matrix.postRotate(-viewFinderRotation!!.toFloat(), centerX, centerY)

    // Buffers are rotated relative to the device's 'natural' orientation: swap width and height
    val bufferRatio = bufferDimens.height / bufferDimens.width.toFloat()

    val scaledWidth: Int
    val scaledHeight: Int
    // Match longest sides together -- i.e. apply center-crop transformation
    if (viewFinderDimens.width > viewFinderDimens.height) {
        scaledHeight = viewFinderDimens.width
        scaledWidth = Math.round(viewFinderDimens.width * bufferRatio)
    } else {
        scaledHeight = viewFinderDimens.height
        scaledWidth = Math.round(viewFinderDimens.height * bufferRatio)
    }

    // Compute the relative scale value
    val xScale = scaledWidth / viewFinderDimens.width.toFloat()
    val yScale = scaledHeight / viewFinderDimens.height.toFloat()

    // Scale input buffers to fill the view finder
    matrix.preScale(xScale, yScale, centerX, centerY)

    // Finally, apply transformations to our TextureView
    textureView.setTransform(matrix)
}

companion object {
    /** Helper function that gets the rotation of a [Display] in degrees */
    fun getDisplaySurfaceRotation(display: Display?) = when(display?.rotation) {
        Surface.ROTATION_0 -> 0
        Surface.ROTATION_90 -> 90
        Surface.ROTATION_180 -> 180
        Surface.ROTATION_270 -> 270
        else -> null
    }

    /**
     * Main entrypoint for users of this class: instantiates the adapter and returns an instance
     * of [Preview] which automatically adjusts in size and rotation to compensate for
     * config changes.
     */
    fun build(config: PreviewConfig, viewFinder: TextureView) =
        AutoFitPreviewBuilder(config, WeakReference(viewFinder)).useCase
}
}

如果配置正确(对我来说没问题)，那么下一个想法是将捕获的图像对象转换为位图可能是错误的。您可以在下面看到实现。

捕捉模式使用这个函数:

fun imageProxyToBitmap(image: ImageProxy): Bitmap {
    val buffer: ByteBuffer = image.planes[0].buffer
    val bytes = ByteArray(buffer.remaining())
    buffer.get(bytes)
    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

分析模式使用这个函数:

fun toBitmapFromImage(image: Image?): Bitmap? {
    try {
        if (image == null || image.planes[0] == null || image.planes[1] == null || image.planes[2] == null) {
            return null
        }

        val yBuffer = image.planes[0].buffer
        val uBuffer = image.planes[1].buffer
        val vBuffer = image.planes[2].buffer

        val ySize = yBuffer.remaining()
        val uSize = uBuffer.remaining()
        val vSize = vBuffer.remaining()

        val nv21 = ByteArray(ySize + uSize + vSize)

        /* U and V are swapped */
        yBuffer.get(nv21, 0, ySize)
        vBuffer.get(nv21, ySize, vSize)
        uBuffer.get(nv21, ySize + vSize, uSize)

        val yuvImage = YuvImage(nv21, ImageFormat.NV21, image.width, image.height, null)
        val out = ByteArrayOutputStream()
        yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 50, out)
        val imageBytes = out.toByteArray()
        return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
    } catch (e: IllegalStateException) {
        Log.e("IllegalStateException", "#ImageUtils.toBitmapFromImage(): Can't read the image file.")
        return null
    }
}

所以，奇怪的是，在少数设备上 toBitmapFromImage() 有时会向上出现，但同时(同一设备)imageProxyToBitmap() 以正确的旋转方式返回图像 -一定是图像到位图功能故障，对吧？为什么会发生这种情况(因为捕获模式正常返回图像)以及如何解决这个问题？

Answer 1

在 onImageCaptureSuccess 中，获取 rotationDegrees 并将位图旋转该度数以获得正确的方向。

override fun onImageCaptureSuccess(image: ImageProxy) {
       
       val capturedImageBitmap = image.image?.toBitmap()?.rotate(image.imageInfo.rotationDegrees.toFloat())
        mBinding.previewImage.setImageBitmap(capturedImageBitmap)
        showPostClickViews()
        mCurrentFlow = FLOW_CAMERA
    }

toBitmap() 和 rotate() 是扩展函数。

fun Image.toBitmap(): Bitmap {
    val buffer = planes[0].buffer
    buffer.rewind()
    val bytes = ByteArray(buffer.capacity())
    buffer.get(bytes)
    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

fun Bitmap.rotate(degrees: Float): Bitmap =
    Bitmap.createBitmap(this, 0, 0, width, height, Matrix().apply { postRotate(degrees) }, true)