gpt4 book ai didi

android - 从 Android 中的 Exoplayer 获取视频像素数据

转载 作者:行者123 更新时间:2023-12-04 23:02:50 29 4
gpt4 key购买 nike

是否可以从 ExoPlayer 中正在运行的视频中获取像素数据(例如,作为 RGB 字节数组)?理想情况下是真实的视频分辨率,而不是显示 View 的大小。我想将该数据转发到 OpenCV 以进行图像处理。

或者,我正在寻找一个强大的(基于 ffmpeg 的)Android 框架来将视频输入 OpenCV,其中输入可能是 IP 摄像机、本地文件、在线文件和在线流。

任何帮助表示赞赏。

最佳答案

警告此解决方案目前仅适用于我的 Android 8 模拟器,因为 this .

好的,这是我的解决方案。 我正在使用 SimpleExoPlayer带有自定义MediaCodecVideoRenderer . 我没有将播放器绑定(bind)到 SimpleExoPlayerView,因为如果我想手动抓取图像数据,则不允许这样做。 在我的自定义 MediaCodecVideoRenderer我覆盖了 processOutputBuffer并使用 getOutputImage获得一个不错的标准 Android 图像。 然后我通过我的 OpenCV 代码发送它,然后使用 OpenCV Utils 将它转换回 Android 位图。每当我需要。小心此代码需要 API >= 21。此外,此代码不输出任何音频。

private void startPlayer(Context context, Uri uri) {

BandwidthMeter bandwidthMeter = new DefaultBandwidthMeter();
TrackSelection.Factory videoTrackSelectionFactory = new AdaptiveTrackSelection.Factory(bandwidthMeter);
TrackSelector trackSelector = new DefaultTrackSelector(videoTrackSelectionFactory);

SimpleExoPlayer player = ExoPlayerFactory.newSimpleInstance((eventHandler, videoRendererEventListener, audioRendererEventListener, textRendererOutput, metadataRendererOutput) -> {
return new Renderer[]{new CustomMediaCodecVideoRenderer(context, MediaCodecSelector.DEFAULT, 1000, eventHandler, videoRendererEventListener, 100)};
}, trackSelector);

DataSource.Factory dataSourceFactory = new DefaultDataSourceFactory(context, Util.getUserAgent(context, context.getPackageName()));
ExtractorMediaSource videoSource = new ExtractorMediaSource.Factory(dataSourceFactory).createMediaSource(uri);

LoopingMediaSource loopingMediaSource = new LoopingMediaSource(videoSource);

player.setPlayWhenReady(true);
player.prepare(loopingMediaSource);

}

class CustomMediaCodecVideoRenderer extends MediaCodecVideoRenderer {
CustomMediaCodecVideoRenderer(Context context, MediaCodecSelector mediaCodecSelector, long allowedJoiningTimeMs, @Nullable Handler eventHandler, @Nullable VideoRendererEventListener eventListener, int maxDroppedFrameCountToNotify) {
super(context, mediaCodecSelector, allowedJoiningTimeMs, eventHandler, eventListener, maxDroppedFrameCountToNotify);
}

@Override
protected boolean processOutputBuffer(long positionUs, long elapsedRealtimeUs, MediaCodec codec, ByteBuffer buffer, int bufferIndex, int bufferFlags, long bufferPresentationTimeUs, boolean shouldSkip) throws ExoPlaybackException {


Image image = codec.getOutputImage(bufferIndex);

if (image != null) {
Log.d(MainActivity.class.getName(), "test");

Mat mat = convertYuv420888ToMat(image, false);
// TODO do your thing with the OpenCV Mat
}

return super.processOutputBuffer(positionUs, elapsedRealtimeUs, codec, buffer, bufferIndex, bufferFlags, bufferPresentationTimeUs, shouldSkip);
}
}

我找到了 Image to Mat 转换 here .
private Mat convertYuv420888ToMat(Image image, boolean isGreyOnly) {
int width = image.getWidth();
int height = image.getHeight();

Image.Plane yPlane = image.getPlanes()[0];
int ySize = yPlane.getBuffer().remaining();

if (isGreyOnly) {
byte[] data = new byte[ySize];
yPlane.getBuffer().get(data, 0, ySize);

Mat greyMat = new Mat(height, width, CvType.CV_8UC1);
greyMat.put(0, 0, data);

return greyMat;
}

Image.Plane uPlane = image.getPlanes()[1];
Image.Plane vPlane = image.getPlanes()[2];

// be aware that this size does not include the padding at the end, if there is any
// (e.g. if pixel stride is 2 the size is ySize / 2 - 1)
int uSize = uPlane.getBuffer().remaining();
int vSize = vPlane.getBuffer().remaining();

byte[] data = new byte[ySize + (ySize/2)];

yPlane.getBuffer().get(data, 0, ySize);

ByteBuffer ub = uPlane.getBuffer();
ByteBuffer vb = vPlane.getBuffer();

int uvPixelStride = uPlane.getPixelStride(); //stride guaranteed to be the same for u and v planes
if (uvPixelStride == 1) {
uPlane.getBuffer().get(data, ySize, uSize);
vPlane.getBuffer().get(data, ySize + uSize, vSize);

Mat yuvMat = new Mat(height + (height / 2), width, CvType.CV_8UC1);
yuvMat.put(0, 0, data);
Mat rgbMat = new Mat(height, width, CvType.CV_8UC3);
Imgproc.cvtColor(yuvMat, rgbMat, Imgproc.COLOR_YUV2RGB_I420, 3);
yuvMat.release();
return rgbMat;
}

// if pixel stride is 2 there is padding between each pixel
// converting it to NV21 by filling the gaps of the v plane with the u values
vb.get(data, ySize, vSize);
for (int i = 0; i < uSize; i += 2) {
data[ySize + i + 1] = ub.get(i);
}

Mat yuvMat = new Mat(height + (height / 2), width, CvType.CV_8UC1);
yuvMat.put(0, 0, data);
Mat rgbMat = new Mat(height, width, CvType.CV_8UC3);
Imgproc.cvtColor(yuvMat, rgbMat, Imgproc.COLOR_YUV2RGB_NV21, 3);
yuvMat.release();
return rgbMat;
}

关于android - 从 Android 中的 Exoplayer 获取视频像素数据,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/48923111/

29 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com