android - Firebase ML Kit 人脸检测不准确

Question

我正在尝试使用 firebase ml 套件+来自 ('com.otaliastudios:cameraview:2.0.0-beta04') 的cameraview +我制作的自定义叠加 View 来制作一个过滤器应用程序。问题是它非常不准确。它确实在相机 View 内绘制了一个框，但不在我的脸周围。这是我的代码:

private void startFaceProcessor() {
    FirebaseVisionFaceDetectorOptions options =
            new FirebaseVisionFaceDetectorOptions.Builder().setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
            .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS).build();
    FirebaseVisionFaceDetector detector = FirebaseVision.getInstance()
            .getVisionFaceDetector(options);
    cameraView.addFrameProcessor(new FrameProcessor() {
        @Override
        public void process(@NonNull Frame frame) {
            int rotation = frame.getRotation() / 90;
            FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
                    .setWidth(frame.getSize().getWidth())   // 480x360 is typically sufficient for
                    .setHeight(frame.getSize().getHeight())  // image recognition
                    .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
                    .setRotation(rotation)
                    .build();
            FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(frame.getData(), metadata);
            detector.detectInImage(image).addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionFace>>() {
                @Override
                public void onSuccess(List<FirebaseVisionFace> firebaseVisionFaces) {
                    if (firebaseVisionFaces.size() > 0) {
                        FirebaseVisionFace face = firebaseVisionFaces.get(0);
                        Rect faceRect = face.getBoundingBox();
                        overlayView.setRect(faceRect);
                    } 
                }
            }).addOnFailureListener(new OnFailureListener() {
                @Override
                public void onFailure(@NonNull Exception e) {
                    Toast.makeText(getApplicationContext(), e.getMessage(), Toast.LENGTH_SHORT).show();
                }
            });
        }
    });
}

OverlayView 绘制框

public class OverlayView extends View {

    private Paint mBoxPaint;
    private Rect rect;
    private Frame frame;
    private FirebaseVisionFace face;

    private float widthScaleFactor = 1f;
    private float heightScaleFactor = 1f;

    public OverlayView(Context context, AttributeSet attrs) {
        super(context, attrs);
        mBoxPaint = new Paint();
        mBoxPaint.setStrokeWidth(2);
        mBoxPaint.setColor(Color.WHITE);
        mBoxPaint.setStyle(Paint.Style.STROKE);
    }

    @Override
    protected void onDraw(Canvas canvas) {
        super.onDraw(canvas);
        if(rect != null) {
            canvas.drawRect(rect, mBoxPaint);
        }

        postInvalidate();
    }

    public void setFrame(Frame frame) {
        this.frame = frame;
    }

    public void setRect(Rect rect) {
        this.rect = rect;
        //this.face = face;
    }

    private float translateX(float x) {
        return frame.getSize().getWidth() - scaleX(x);
    }

    private float translateY(float y) {
        return scaleY(y);
    }

    private float scaleX(float x) {
        return x * widthScaleFactor;
    }

    private float scaleY(float y) {
        return y * heightScaleFactor;
    }

}

XML 文件:

 <FrameLayout
        android:id="@+id/frameLayout"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_weight="1.5">

        <com.otaliastudios.cameraview.CameraView
            android:id="@+id/camera_view"
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:adjustViewBounds="true"
            android:keepScreenOn="true"
            app:cameraFacing="front" />

        <com.gbcapps.camerafilter.OverlayView
            android:id="@+id/overlay_view"
            android:layout_width="match_parent"
            android:layout_height="match_parent" />

    </FrameLayout>

我从昨天开始就一直在尝试，但就是无法让它发挥作用。

Answer 1

我不知道这是否有用，但我设法从捕获的帧中裁剪检测到的脸部的位图。当我在 ImageView 上显示faceBmp 时，它会正确裁剪框架。

detectedFace = face.getBoundingBox();
Bitmap faceBmp = Bitmap.createBitmap(frameBmp, detectedFace.left, detectedFace.top, detectedFace.width(), detectedFace.height());

以及示例快速入门应用程序:

float x = translateX(face.getBoundingBox().centerX());
float y = translateY(face.getBoundingBox().centerY());

float xOffset = scaleX(face.getBoundingBox().width() / 2.0f);
float yOffset = scaleY(face.getBoundingBox().height() / 2.0f);
float left = x - xOffset;
float top = y - yOffset;
float right = x + xOffset;
float bottom = y + yOffset;

canvas.drawRect(left, top, right, bottom, boxPaint);