ffmpeg - 将 RTSP 信号与 FFMPEG 同步

Question

我已经实现将两个 RTSP 信号与 FFMPEG 合并为一个，但在结果流中，第一个输入相对于第二个输入有 2 到 5 秒的延迟。与第二个输入相比，延迟总是第一个输入。

两个 RTSP 信号来自相同的相机型号、相同的配置、相同的房间......

但是，如果我将相同的 RTSP 信号(两者之一)作为输入 1 和输入 2，则会发生同样的事情。尽管是相同的信号，但与第二个输入相比，第一个输入延迟了。

我怎样才能让它们同步？

这是我执行的命令:

ffmpeg -rtsp_transport tcp -thread_queue_size 512 -rtbufsize 50M -r 15 
-i rtsp://XXXX -rtsp_transport tcp -thread_queue_size 512 -rtbufsize 50M -r 15 
-c:a aac -i rtsp://YYYY -filter_complex "[0:v]pad=iw*2:ih,setpts=PTS-STARTPTS[bg]; 
[1:v]setpts=PTS-STARTPTS[fg]; [bg][fg]overlay=w[out]" -map "[out]" -f hls 
-hls_time 2 -hls_list_size 5 -use_localtime 1 -use_localtime_mkdir 1 
-hls_segment_filename 'LIVE/file-%s.ts' -map a -ar 16000 -ac 1 -ab 64000 -c:a aac 
-y output.m3u8

这里有流程信息:

ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers 
built with Apple LLVM version 9.0.0 (clang-900.0.39.2) 
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[rtsp @ 0x7fe284000e00] Missing PPS in sprop-parameter-sets, ignoring
Input #0, rtsp, from 'rtsp://XXXX':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1280x720, 15 fps, 15 tbr, 90k tbn, 30 tbc
    Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
[rtsp @ 0x7fe28484de00] Missing PPS in sprop-parameter-sets, ignoring
Input #1, rtsp, from 'rtsp://XXXX':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #1:0: Video: h264 (Main), yuv420p(progressive), 1280x720, 15 fps, 15 tbr, 90k tbn, 30 tbc
    Stream #1:1: Audio: aac (LC), 16000 Hz, mono, fltp
Stream mapping:
  Stream #0:0 (h264) -> pad (graph 0)
  Stream #1:0 (h264) -> setpts (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x7fe286802000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fe286802000] profile High, level 4.0
[libx264 @ 0x7fe286802000] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=1 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x7fe286800000] Opening 'LIVE/file-1524728763.ts' for writing
Output #0, hls, to 'output.m3u8':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 2560x720, q=-1--1, 15 fps, 90k tbn, 15 tbc (default)
    Metadata:
      encoder         : Lavc57.107.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc57.107.100 aac
[hls @ 0x7fe286800000] Opening 'LIVE/file-1524728782.ts' for writinged=1.09x    
[hls @ 0x7fe286800000] Opening 'output.m3u8.tmp' for writing
[hls @ 0x7fe286800000] Opening 'output.m3u8.tmp' for writing=N/A speed=1.07x    
frame=  396 fps= 15 q=-1.0 Lsize=N/A time=00:00:27.00 bitrate=N/A speed=1.05x    
video:2946kB audio:147kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Answer 1

来自 ffmpeg 文档
https://ffmpeg.org/ffmpeg-filters.html#Examples-139 :
尝试设置视频点
setpts='(RTCTIME - RTCSTART) / (TB * 1000000)'
(一些文档说这已被弃用，但我使用的是 11 月 27 日的版本，它可以工作。)

将视频流同步到第一个流，即
-map "[out]",0:v
并在音频上设置 pts
-af 'asetpts=N/SR/TB'
YMMV，但它适用于我的廉价中国 IP 摄像头