android - 使用 OpenGL 和 MediaCodec 播放视频

Question

我试图在两个不同的纹理 View 中同时播放同一个视频。我使用了 grafika 的代码(MoviePlayer 和 ContinuousCaptureActivity)来尝试让它工作(感谢 fadden)。为了使问题更简单，我尝试先用一个 TextureView 来解决。

目前我已经创建了一个 TextureView，一旦它获得了 SurfaceTexture，我就创建了一个 WindowSurface 并将其设置为当前。然后我生成一个使用 FullFrameRect 对象生成的 TextureID。

@Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int
        width, int height) {
    mSurfaceTexture = surface;
    mEGLCore = new EglCore(null, EglCore.FLAG_TRY_GLES3);
    Log.d("EglCore", "EGL core made");
    mDisplaySurface = new WindowSurface(mEGLCore, mSurfaceTexture);
    mDisplaySurface.makeCurrent();
    Log.d("DisplaySurface", "mDisplaySurface made");
    mFullFrameBlit = new FullFrameRect(new Texture2dProgram(Texture2dProgram.ProgramType.TEXTURE_EXT));
    mTextureID = mFullFrameBlit.createTextureObject();

    //mSurfaceTexture.attachToGLContext(mTextureID);
    clickPlayStop(null);
}

然后我得到一个屏幕外的 SurfaceTexture，将它与我上面得到的 TextureID 链接起来，然后创建一个表面以传递给 MoviePlayer:

public void clickPlayStop(@SuppressWarnings("unused") View unused) {
    if (mShowStopLabel) {
        Log.d(TAG, "stopping movie");
        stopPlayback();
        // Don't update the controls here -- let the task thread do it after the movie has
        // actually stopped.
        //mShowStopLabel = false;
        //updateControls();
    } else {
        if (mPlayTask != null) {
            Log.w(TAG, "movie already playing");
            return;
        }
        Log.d(TAG, "starting movie");
        SpeedControlCallback callback = new SpeedControlCallback();
        callback.setFixedPlaybackRate(24);

        MoviePlayer player = null;
        MovieTexture = new SurfaceTexture(mTextureID);
        MovieTexture.setOnFrameAvailableListener(this);
        Surface surface = new Surface(MovieTexture);
        try {
            player = new MoviePlayer(surface, callback, this);//TODO
        } catch (IOException ioe) {
            Log.e(TAG, "Unable to play movie", ioe);

            return;
        }

        adjustAspectRatio(player.getVideoWidth(), player.getVideoHeight());

        mPlayTask = new MoviePlayer.PlayTask(player, this);
        mPlayTask.setLoopMode(true);


        mShowStopLabel = true;
        mPlayTask.execute();
    }
}

想法是 SurfaceTexture 获得一个原始帧，我可以将其用作 OES_external 纹理以使用 OpenGL 从中进行采样。然后我可以在将我的 WindowSurface 设置为当前后从我的 EGLContext 调用 DrawFrame()。

private void drawFrame() {
    Log.d(TAG, "drawFrame");
    if (mEGLCore == null) {
        Log.d(TAG, "Skipping drawFrame after shutdown");
        return;
    }

    // Latch the next frame from the camera.
    mDisplaySurface.makeCurrent();
    MovieTexture.updateTexImage();
    MovieTexture.getTransformMatrix(mTransformMatrix);

    // Fill the WindowSurface with it.
    int viewWidth = mTextureView.getWidth();
    int viewHeight = mTextureView.getHeight();
    GLES20.glViewport(0, 0, viewWidth, viewHeight);
    mFullFrameBlit.drawFrame(mTextureID, mTransformMatrix);
    mDisplaySurface.swapBuffers();
}

如果我想用 2 个 TextureView 来做，我的想法是调用 makeCurrent() 并为每个 View 绘制到每个缓冲区，然后在绘制完成后调用 swapBuffers()。

这是我想做的，但我很确定这不是我的代码实际做的。有人可以帮助我了解我需要更改哪些内容才能使其正常运行吗？

@法登

更新:这很有趣。我将 onSurfaceTextureAvailable 中的代码更改为:

@Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int
        width, int height) {
    mSurfaceTexture = surface;
    TextureHeight = height;
    TextureWidth = width;
    //mEGLCore = new EglCore(null, EglCore.FLAG_TRY_GLES3);
    Log.d("EglCore", "EGL core made");
    //mDisplaySurface = new WindowSurface(mEGLCore, mSurfaceTexture);
    //mDisplaySurface.makeCurrent();
    Log.d("DisplaySurface", "mDisplaySurface made");
    //mFullFrameBlit = new FullFrameRect(new Texture2dProgram(Texture2dProgram.ProgramType.OPENGL_TEST));
    //mTextureID = mFullFrameBlit.createTextureObject();
    //clickPlayStop(null);


    // Fill the SurfaceView with it.
    //int viewWidth = width;
    //int viewHeight = height;
    //GLES20.glViewport(0, 0, viewWidth, viewHeight);
    //mFullFrameBlit.drawFrame(mTextureID, mTransformMatrix);
    //mFullFrameBlit.openGLTest();
    //mFullFrameBlit.testDraw(mDisplaySurface.getHeight(),mDisplaySurface.getWidth());
    //mDisplaySurface.swapBuffers();
}

所以，它不应该调用任何其他东西，只显示空的 TextureView - 这就是我所看到的......

Answer 1

感谢 Fadden 的帮助。

因此，当我使用新线程解码和生成帧时，似乎解决了一些未知问题。我还没有找出导致原始问题的原因，但我找到了解决方法。