android - 在 Android 屏幕录制中 - 如何获取每一帧？

Question

我正在使用 MediaRecorder 和 MediaProjection Api 在 Android 应用程序中录制屏幕。我无法获得可以通过 rtmp 流发送的每一帧。为了通过 RTMP 流发布，我使用的是 JAVACV Android 库。

例如 - 如果通过摄像头进行直播，我们可以在 onPreviewFrame() 回调中获取每一帧。获取每一帧后，我只需使用 JAVACV 库的 FFmpegFrameRecorder 将每一帧流式传输到 rtmp url。

如何通过屏幕录制实现相同的效果？

任何帮助将不胜感激。提前致谢!

Answer 1

首先，MediaProjection 没有回调接口(interface)，可以为您提供屏幕捕获帧的缓冲区，但 SurfaceTexture 很有可能.这是一种实现ScreenCapturerAndroid使用 SurfaceTexture 进行屏幕捕获。

An implementation of VideoCapturer to capture the screen content as a video stream. Capturing is done by MediaProjection on a SurfaceTexture. We interact with this SurfaceTexture using a SurfaceTextureHelper. The SurfaceTextureHelper is created by the native code and passed to this capturer in VideoCapturer.initialize(). On receiving a new frame, this capturer passes it as a texture to the native code via CapturerObserver.onFrameCaptured(). This takes place on the HandlerThread of the given SurfaceTextureHelper. When done with each frame, the native code returns the buffer to the SurfaceTextureHelper

但是，如果您想发送您的应用程序 View 流，那么下面的屏幕截图将对您有所帮助。

public class ScreenCapturer {
    private boolean capturing = false;
    private int fps=15;
    private int width,height;
    private Context context;
    private int[] frame;
    private Bitmap bmp;
    private Canvas canvas;
    private View contentView;
    private Handler mHandler = new Handler();
    public ScreenCapturer(Context context, View view) {
        this.context = context;
        this.contentView = view;
    }

    public void startCapturing(){
        capturing = true;

        mHandler.postDelayed(newFrame, 1000 / fps);
    }
    public void stopCapturing(){
        capturing = false;
        mHandler.removeCallbacks(newFrame);
    }
    private Runnable newFrame = new Runnable() {
        @Override
        public void run() {
            if (capturing) {
                int width = contentView.getWidth();
                int height = contentView.getHeight();
                if (frame == null ||
                        ScreenCapturer. this.width != width ||
                        ScreenCapturer.this.height != height) {

                    ScreenCapturer.this.width = width;
                    ScreenCapturer.this.height = height;

                    if (bmp != null) {
                        bmp.recycle();
                        bmp = null;
                    }
                    bmp = Bitmap.createBitmap(width,
                            height, Bitmap.Config.ARGB_8888);

                    canvas = new Canvas(bmp);
                    frame = new int[width * height];
                }
                canvas.saveLayer(0, 0, width, height, null);
                canvas.translate(-contentView.getScrollX(), - contentView.getScrollY());
                contentView.draw(canvas);

                bmp.getPixels(frame, 0, width, 0, 0, width, height);


               //frame[] is a rgb pixel array compress it to YUV if want and send over RTMP

                canvas.restore();
                mHandler.postDelayed(newFrame, 1000 / fps);

            }
        }
    };


}

用法

...
//Use this in your activity 

private View parentView;

parentView = findViewById(R.id.parentView);
capturer = new ScreenCapturer(this,parentView);

//To start capturing
 capturer.startCapturing();


//To Stop capturer
capturer.stopCapturing();

使用它您可以将 View 内容发送到 RTMP 流，您可以使用 Activity 的父 View 来捕获 Activity 的所有内容。