opencv - FFMPEG 的视频帧差异

Question

我需要计算源视频和压缩视频之间的帧差异。现在我使用 OpenCV 和 Java，通过提取每一帧并做一个简单的差异，但它很慢(工作 0.5 fps，这意味着 500 帧视频将花费超过 15 分钟)，所以我想转移到 FFMPEG。

FFMPEG 感觉快很多(一切都在 1 分钟内完成)但它有一个大问题导致结果无用:压缩源文件时，使用 FFMPEG同样，在开头添加了一个额外的灰色帧，这会伪造结果，因为比较的是不同的帧。

这就是我现在正在做的(知道额外的框架会搞砸这一切):

ffmpeg -y -i src.avi -i compressed.avi -filter_complex "blend=all_mode=difference,hue=s=0" -c:v libx264 -crf 18 -c:a copy difference.avi

为了解决帧问题，我试图通过使用此命令重新编码压缩视频来删除第一帧

ffmpeg -y -ss 0.02 -i compressed.mpg -an -f mpeg2video compressed-cut.mpg"

(注意 -ss 是 0.02 因为它是一个 50 fps 视频，所以我做了 1/FPS 按照建议 here )

但是我得到了这个回复

Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

所以，最后，问题是:由于提取所有帧然后用 OpenCV 计算差异真的很慢，我如何使用 FFMPEG 来制作包含两个来源之间的差异，同时牢记其中一个来源在开头有一个额外的框架？

编辑:我想避免发布无休止的控制台输出，但既然你要求了，我们就开始吧。

1)编码

输入

ffmpeg -i "720p50_mobcal_ter.avi" -an -f mpeg2video -y "720p50_mobcal_ter.mpg"

输出

ffmpeg version N-76684-g1fe82ab Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55.  6.100 / 55.  6.100
  libavcodec     57. 15.100 / 57. 15.100
  libavformat    57. 14.100 / 57. 14.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 15.100 /  6. 15.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, avi, from '720p50_mobcal_ter.avi':
  Metadata:
    encoder         : Lavf57.14.100
  Duration: 00:00:10.08, start: 0.000000, bitrate: 552974 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 554059 kb/s, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 50 tbn, 50 tbc
Output #0, mpeg2video, to '720p50_mobcal_ter.mpg':
  Metadata:
    encoder         : Lavf57.14.100
    Stream #0:0: Video: mpeg2video, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc57.15.100 mpeg2video
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg2video (native))
Press [q] to stop, [?] for help
frame=   41 fps=0.0 q=31.0 size=     984kB time=00:00:00.78 bitrate=10330.5kbits/frame=   80 fps= 78 q=31.0 size=    1323kB time=00:00:01.56 bitrate=6948.1kbits/frame=  124 fps= 80 q=31.0 size=    1725kB time=00:00:02.44 bitrate=5790.0kbits/frame=  168 fps= 81 q=31.0 size=    2084kB time=00:00:03.32 bitrate=5142.8kbits/frame=  212 fps= 81 q=31.0 size=    2482kB time=00:00:04.20 bitrate=4841.4kbits/frame=  255 fps= 82 q=31.0 size=    2840kB time=00:00:05.06 bitrate=4597.2kbits/frame=  296 fps= 82 q=31.0 size=    3133kB time=00:00:05.88 bitrate=4364.5kbits/frame=  338 fps= 82 q=24.8 size=    3453kB time=00:00:06.72 bitrate=4209.2kbits/frame=  382 fps= 82 q=31.0 size=    3723kB time=00:00:07.60 bitrate=4013.4kbits/frame=  426 fps= 83 q=31.0 size=    4005kB time=00:00:08.48 bitrate=3869.1kbits/frame=  470 fps= 83 q=24.8 size=    4276kB time=00:00:09.36 bitrate=3742.5kbits/frame=  504 fps= 83 q=31.0 Lsize=    4469kB time=00:00:10.06 bitrate=3639.3kbits/s
video:4469kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

这会在开头添加额外的灰色框架，它只是复制第一个

2) 删除第一帧

输入

ffmpeg -y -i "720p50_mobcal_ter.mpg" -an -f mpeg2video -vf select=gte(n\,1) "CUT-720p50_mobcal_ter.mpg"

输出

ffmpeg version N-76684-g1fe82ab Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55.  6.100 / 55.  6.100
  libavcodec     57. 15.100 / 57. 15.100
  libavformat    57. 14.100 / 57. 14.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 15.100 /  6. 15.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mpegvideo, from '720p50_mobcal_ter.mpg':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: mpeg2video (Main), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 50 fps, 50 tbr, 1200k tbn, 100 tbc
Output #0, mpeg2video, to 'CUT-720p50_mobcal_ter.mpg':
  Metadata:
    encoder         : Lavf57.14.100
    Stream #0:0: Video: mpeg2video, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc57.15.100 mpeg2video
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> mpeg2video (native))
Press [q] to stop, [?] for help
frame=  255 fps=0.0 q=31.0 size=    2781kB time=00:00:05.10 bitrate=4467.3kbits/frame=  503 fps=0.0 q=31.0 Lsize=    4415kB time=00:00:10.08 bitrate=3588.5kbits/s
video:4415kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

3)帧差异

输入

ffmpeg -y -i "720p50_mobcal_ter.avi" -i "CUT-720p50_mobcal_ter.mpg" -filter_complex "blend=all_mode=difference,hue=s=0" -c:v libx264 -crf 18 -c:a copy "DIFF-720p50_mobcal_ter.mpg"

输出

ffmpeg version N-76684-g1fe82ab Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55.  6.100 / 55.  6.100
  libavcodec     57. 15.100 / 57. 15.100
  libavformat    57. 14.100 / 57. 14.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 15.100 /  6. 15.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, avi, from '720p50_mobcal_ter.avi':
  Metadata:
    encoder         : Lavf57.14.100
  Duration: 00:00:10.08, start: 0.000000, bitrate: 552974 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 554059 kb/s, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 50 tbn, 50 tbc
Input #1, mpegvideo, from 'CUT-720p50_mobcal_ter.mpg':
  Duration: N/A, bitrate: N/A
    Stream #1:0: Video: mpeg2video (Main), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 50 fps, 50 tbr, 1200k tbn, 100 tbc
[libx264 @ 000002784dbeb980] using SAR=1/1
[libx264 @ 000002784dbeb980] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 000002784dbeb980] profile High, level 3.2
[mpeg @ 000002784dbeaf20] VBV buffer size not set, using default size of 130KB
If you want the mpeg file to be compliant to some specification
Like DVD, VCD or others, make sure you set the correct buffer size
Output #0, mpeg, to 'D:\DOWNLOADS\TMP\Video TDI\AVI\DIFF-720p50_mobcal_ter.mpg':
  Metadata:
    encoder         : Lavf57.14.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 50 fps, 90k tbn, 50 tbc (default)
    Metadata:
      encoder         : Lavc57.15.100 libx264
Stream mapping:
  Stream #0:0 (rawvideo) -> blend:top
  Stream #1:0 (mpeg2video) -> blend:bottom
  hue -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
frame=  504 fps= 39 q=-1.0 Lsize=   32182kB time=00:00:10.04 bitrate=26258.5kbits/s
video:32061kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.377054%
[libx264 @ 000002784dbeb980] frame I:30    Avg QP:19.69  size:149974
[libx264 @ 000002784dbeb980] frame P:299   Avg QP:23.28  size: 69423
[libx264 @ 000002784dbeb980] frame B:175   Avg QP:24.48  size: 43280
[libx264 @ 000002784dbeb980] consecutive B-frames: 30.6% 69.4%  0.0%  0.0%
[libx264 @ 000002784dbeb980] mb I  I16..4: 18.3% 51.4% 30.4%
[libx264 @ 000002784dbeb980] mb P  I16..4:  0.6%  5.6%  2.4%  P16..4: 35.9% 22.9% 15.6%  0.0%  0.0%    skip:17.0%
[libx264 @ 000002784dbeb980] mb B  I16..4:  0.2%  0.5%  0.3%  B16..8: 49.5% 12.4%  5.6%  direct:15.5%  skip:16.1%  L0:47.8% L1:42.1% BI:10.1%
[libx264 @ 000002784dbeb980] 8x8 transform intra:57.5% inter:38.5%
[libx264 @ 000002784dbeb980] coded y,uvDC,uvAC intra: 90.7% 0.0% 0.0% inter: 50.3% 0.0% 0.0%
[libx264 @ 000002784dbeb980] i16 v,h,dc,p: 32% 23% 35% 10%
[libx264 @ 000002784dbeb980] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 11% 41%  7%  5%  6%  5%  6%  8%
[libx264 @ 000002784dbeb980] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 14% 23%  8%  7%  7%  7%  7%  8%
[libx264 @ 000002784dbeb980] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 000002784dbeb980] Weighted P-Frames: Y:33.8% UV:0.0%
[libx264 @ 000002784dbeb980] ref P L0: 58.1% 16.3% 14.2%  9.4%  2.0%
[libx264 @ 000002784dbeb980] ref B L0: 79.3% 20.7%
[libx264 @ 000002784dbeb980] kb/s:26056.02

第二个命令使一切正常，而第一部分中的第二个命令却没有(带有 -ss 选项的命令)，所以我对此感到非常高兴，但我不是确保 FFMPEG 是否复制每个视频的第一帧，或者它是否与我现在使用的视频有关，所以最好从具有相同帧数的压缩视频开始原始的。

那么让我们来看看最后一个问题:为什么 FFMPEG 在压缩视频的开头添加重复的第一帧，我该如何避免这种情况？

Answer 1

编码方式

ffmpeg -i "720p50_mobcal_ter.avi" -c:v mpeg2video "720p50_mobcal_ter.mpg"

代替

ffmpeg -i "720p50_mobcal_ter.avi" -an -f mpeg2video -y "720p50_mobcal_ter.mpg"

防止创建重复框架并使一切正常工作。