android - MediaCodec.configure 因 1080p 视频的 IllegalStateException 而失败

Question

我一直在使用 grafika 的 MoviePlayer和 bigFlake 的 ExtractMpegFramesTest在我们的应用程序中实现搜索和提取帧功能。这些对我们的大多数用户来说工作正常，但是他们中的一些人在设置 ExtractMpegFramesTest 时在 MediaCodec.configure 上遇到 IllegalStateException(这发生在三星 Galaxy S4 mini、三星 Galaxy J7、三星 Galaxy A5、华为 Ascend G7)。相关代码如下:

电影播放器​​

public void prepareResources() throws IOException {
    // The MediaExtractor error messages aren't very useful.  Check to see if the input
    // file exists so we can throw a better one if it's not there.
    if (!mSourceFile.canRead()) {
        throw new FileNotFoundException("Unable to read " + mSourceFile);
    }

    try {
        mExtractor = new MediaExtractor();
        mExtractor.setDataSource(mSourceFile.toString());
        mTrackIndex = selectTrack(mExtractor);
        if (mTrackIndex < 0) {
            throw new RuntimeException("No video track found in " + mSourceFile);
        }
        mExtractor.selectTrack(mTrackIndex);

        MediaFormat format = mExtractor.getTrackFormat(mTrackIndex);

        // Create a MediaCodec decoder, and configure it with the MediaFormat from the
        // extractor.  It's very important to use the format from the extractor because
        // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
        String mime = format.getString(MediaFormat.KEY_MIME);
        mDecoder = MediaCodec.createDecoderByType(mime);
        mDecoder.configure(format, mOutputSurface, null, 0);
        mDecoder.start();
        mDecoderInputBuffers = mDecoder.getInputBuffers();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

ExtractMpegFramesTest

private void extractMpegFrames() throws IOException {
    MediaCodec decoder = null;
    CodecOutputSurface outputSurface = null;
    MediaExtractor extractor = null;
    /*int saveWidth = 640;
    int saveHeight = 480;*/

    try {
        long start1 = System.currentTimeMillis();

        File inputFile = new File(mInputVideoPath);   // must be an absolute path
        // The MediaExtractor error messages aren't very useful.  Check to see if the input
        // file exists so we can throw a better one if it's not there.
        if (!inputFile.canRead()) {
            throw new FileNotFoundException("Unable to read " + inputFile);
        }

        extractor = new MediaExtractor();
        extractor.setDataSource(inputFile.toString());
        int trackIndex = selectTrack(extractor);
        if (trackIndex < 0) {
            throw new RuntimeException("No video track found in " + inputFile);
        }
        extractor.selectTrack(trackIndex);

        MediaFormat format = extractor.getTrackFormat(trackIndex);
        if (VERBOSE) {
            Log.d(TAG, "Video size is " + format.getInteger(MediaFormat.KEY_WIDTH) + "x" +
                    format.getInteger(MediaFormat.KEY_HEIGHT));
        }

        // Could use width/height from the MediaFormat to get full-size frames.
        outputSurface = new CodecOutputSurface(format.getInteger(MediaFormat.KEY_WIDTH), format.getInteger(MediaFormat.KEY_HEIGHT));

        // Create a MediaCodec decoder, and configure it with the MediaFormat from the
        // extractor.  It's very important to use the format from the extractor because
        // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
        String mime = format.getString(MediaFormat.KEY_MIME);
        decoder = MediaCodec.createDecoderByType(mime);
        decoder.configure(format, outputSurface.getSurface(), null, 0); //fails right here
        decoder.start();

        long end1 = System.currentTimeMillis();
        Timber.d("extractMpegFrames(): FRAME_EXTRACT setup in: %d millis", (end1-start1));

        long start2 = System.currentTimeMillis();
        doExtract(extractor, trackIndex, decoder, outputSurface);
        long end2 = System.currentTimeMillis();
        Timber.d("extractMpegFrames(): FRAME_EXTRACT doExtract in: %d millis", (end2-start2));
    } finally {
        // release everything we grabbed
        if (outputSurface != null) {
            outputSurface.release();
            outputSurface = null;
        }
        if (decoder != null) {
            decoder.stop();
            decoder.release();
            decoder = null;
        }
        if (extractor != null) {
            extractor.release();
            extractor = null;
        }
    }
}

我知道这个question ，但是我不明白的是为什么 MediaCodec 可以在 MoviePlayer 中正常配置(视频运行正常)但在 ExtractMpegFramesTest 中却失败了.起初，我认为该设备在 OpenGL 设置方面存在一些问题，但后来证明是 MediaCodec 问题。

任何见解将不胜感激。

编辑:在获得 Galaxy S4 mini 测试设备后，我能够获得更有帮助的日志:

D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
D/MoviePlayer: Video size is 1920x1080
D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
I/OMXClient: Using client-side OMX mux.
E/ACodec: [OMX.qcom.video.decoder.avc] storeMetaDataInBuffers failed w/ err -2147483648
E/ACodec:  configureCodec multi window instance fail  appPid : 3283
E/ACodec: [OMX.qcom.video.decoder.avc] configureCodec returning error -38
E/MediaCodec: Codec reported an error. (omx error 0x80001001, internalError -38)
W/System.err: java.lang.IllegalStateException
W/System.err:     at android.media.MediaCodec.native_configure(Native Method)
W/System.err:     at android.media.MediaCodec.configure(MediaCodec.java:262)
W/System.err:     at co.test.testing.player.MoviePlayer.prepareResources(MoviePlayer.java:237)
W/System.err:     at co.test.testing.activities.VideoActivity.surfaceCreated(VideoActivity.java:276)
W/System.err:     at android.view.SurfaceView.updateWindow(SurfaceView.java:602)
W/System.err:     at android.view.SurfaceView.access$000(SurfaceView.java:94)
W/System.err:     at android.view.SurfaceView$3.onPreDraw(SurfaceView.java:183)
W/System.err:     at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:891)
W/System.err:     at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2201)
W/System.err:     at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1256)
W/System.err:     at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:6635)
W/System.err:     at android.view.Choreographer$CallbackRecord.run(Choreographer.java:813)
W/System.err:     at android.view.Choreographer.doCallbacks(Choreographer.java:613)
W/System.err:     at android.view.Choreographer.doFrame(Choreographer.java:583)
W/System.err:     at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:799)
W/System.err:     at android.os.Handler.handleCallback(Handler.java:733)
W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:95)
W/System.err:     at android.os.Looper.loop(Looper.java:146)
W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:5593)
W/System.err:     at java.lang.reflect.Method.invokeNative(Native Method)
W/System.err:     at java.lang.reflect.Method.invoke(Method.java:515)
W/System.err:     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
W/System.err:     at dalvik.system.NativeStart.main(Native Method)

这些是从有此问题的设备中检索到的编解码器信息:

• OMX.qcom.video.encoder.avc

• OMX.SEC.avc.enc

• OMX.Exynos.AVC.Encoder

奇怪的是 Galaxy S4 mini 可以很好地处理 720p 视频，它只是在处理 1080p 视频时遇到问题。

Answer 1

好的，这里再次回答我自己的问题:

• 首先，为什么我在使用MoviePlayer显示视频后无法使用ExtractMpegFramesTest提取帧:

  某些设备似乎无法同时处理 2 个 MediaCodec 实例以播放高分辨率视频(在 Galaxy S4 mini 上为 >720p)，因此您必须正确释放一个MediaCodec 的实例，然后再启动另一个实例，如下所示:

  public void releaseResources() {
      // release everything we grabbed
      if (mDecoder != null) {
          try {
              mDecoder.stop();
              mDecoder.release();
              mDecoder = null;
          } catch (Exception e) {
              Timber.d("releaseResources(): message %s cause %s", e.getMessage(),  e.getCause());
              e.printStackTrace();
          }
      }
      if (mExtractor != null) {
          try {
              mExtractor.release();
              mExtractor = null;
          } catch (Exception e) {
              Timber.d("releaseResources(): message %s cause %s", e.getMessage(),  e.getCause());
              e.printStackTrace();
          }
      }
  }
  

• 其次，为什么 MoviePlayer 无法配置 1080p 视频。在我的例子中，因为我从一个媒体选择器 Activity 开始这个 Activity ，其中有一个 VideoView 用于预览，它有一个 MediaPlayer.OnErrorListener() 注册来检测错误的视频。问题是 onError() 回调在 Galaxy S4 mini 上有一些非常奇怪的行为，而它在我的开发设备上工作得很好。我的疯狂猜测是 MediaPlayer.OnErrorListener() 使 MediaCodec 处于不良状态，这会在以后引起很多麻烦。

  因此解决方案是设置一个空白的 OnErrorListener() 并在使用 MediaCodec 执行任何其他操作之前正确释放 VideoView。像这样:

  mVideoView.setOnErrorListener(new MediaPlayer.OnErrorListener() {
          @Override
          public boolean onError(MediaPlayer mp, int what, int extra) {
              return true;
          }
  });
  .....
  mVideoView.stopPlayback();
  doOtherThingWithMediaCodec();
  

这解释了在不同设备上发生的许多奇怪的事情。我希望这可以帮助其他人调试他们的应用程序。