FFMpeg - 拆分窗口 RTMP - 第二个流的延迟

Question

我正在尝试将两个直播 RTMP源到一个分屏输出与组合音频。然后将输出发送到接收 RTMP服务器。

使用以下命令，它使用相同的 RTMP两个提要的输入，我已经设法使上述工作正常进行，但是 左侧输入延迟约 2 秒 从右边的那个。

ffmpeg -re -i rtmp://myserver.tld/live/stream_key -re -i rtmp://myserver.tld/live/stream_key \
-filter_complex "\
nullsrc=size=1152x720 [base];\
[0:v] crop=576:720 [upperleft];\
[1:v] crop=576:720 [upperright];\
[base][upperleft] overlay=shortest=1 [tmp1];\
[tmp1][upperright] overlay=shortest=1:x=576;\
[0:a][1:a]amix \
" -c:a libfdk_aac -ar 44100 -threads 32 -c:v libx264 -g 50 -preset ultrafast -tune zerolatency -f flv rtmp://myserver.tld/live/new_stream_key

输出:

ffmpeg version N-76137-gb0bb1dc Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04)
configuration: --prefix=/home/ubuntu/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/ubuntu/ffmpeg_build/include --extra-ldflags=-L/home/ubuntu/ffmpeg_build/lib --bindir=/home/ubuntu/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil      55.  4.100 / 55.  4.100
libavcodec     57.  7.100 / 57.  7.100
libavformat    57.  8.102 / 57.  8.102
libavdevice    57.  0.100 / 57.  0.100
libavfilter     6. 12.100 /  6. 12.100
libswscale      4.  0.100 /  4.  0.100
libswresample   2.  0.100 /  2.  0.100
libpostproc    54.  0.100 / 54.  0.100
[flv @ 0x3a0e940] video stream discovered after head already parsed
Input #0, flv, from 'rtmp://myserver.tld/live/stream_key':
Metadata:
Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth    : 1152
displayHeight   : 720
fps             : 29
profile         :
level           :
Duration: 00:00:00.00, start: 5.717000, bitrate: N/A
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 163 kb/s
Stream #0:1: Video: h264 (High), yuv420p, 1152x720, 30.30 fps, 29.97 tbr, 1k tbn, 59.94 tbc
[flv @ 0x3a49e00] video stream discovered after head already parsed
Input #1, flv, from 'rtmp://myserver.tld/live/stream_key':
Metadata:
Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth    : 1152
displayHeight   : 720
fps             : 29
profile         :
level           :
Duration: 00:00:00.00, start: 9.685000, bitrate: N/A
Stream #1:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 163 kb/s
Stream #1:1: Video: h264 (High), yuv420p, 1152x720, 30.30 fps, 29.97 tbr, 1k tbn, 59.94 tbc
[libx264 @ 0x3a9cd60] Application has requested 32 threads. Using a thread count greater than 16 is not recommended.
[libx264 @ 0x3a9cd60] using SAR=1/1
[libx264 @ 0x3a9cd60] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x3a9cd60] profile Constrained Baseline, level 3.1
[libx264 @ 0x3a9cd60] 264 - core 142 r2389 956c8d8 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=11 lookahead_threads=11 sliced_threads=1 slices=11 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=50 keyint_min=5 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, flv, to 'rtmp://myserver.tld/live/new_stream_key':
Metadata:
Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
displayWidth    : 1152
displayHeight   : 720
fps             : 29
profile         :
level           :
encoder         : Lavf57.8.102
Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 1152x720 [SAR 1:1 DAR 8:5], q=-1--1, 25 fps, 1k tbn, 25 tbc (default)
Metadata:
encoder         : Lavc57.7.100 libx264
Stream #0:1: Audio: aac (libfdk_aac) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, s16, 128 kb/s (default)
Metadata:
encoder         : Lavc57.7.100 libfdk_aac
Stream mapping:
Stream #0:0 (aac) -> amix:input0
Stream #0:1 (h264) -> crop
Stream #1:0 (aac) -> amix:input1
Stream #1:1 (h264) -> crop
overlay -> Stream #0:0 (libx264)
amix -> Stream #0:1 (libfdk_aac)
Press [q] to stop, [?] for help
[flv @ 0x3a0e940] Thread message queue blocking; consider raising the thread_queue_size option (current value: 512)
frame=   81 fps= 20 q=15.0 size=     674kB time=00:00:03.24 bitrate=1703.3kbits/frame=  102 fps= 22 q=22.0 size=     945kB time=00:00:04.08 bitrate=1896.4kbits/

有没有办法强制 FFMpeg阅读这两个 RTMP同时输入？

Answer 1

尝试

ffmpeg -re -i rtmp://1 -re -i rtmp://2 \
-filter_complex \
"[0:v] crop=576:720,setpts=PTS-STARTPTS [upperleft];\
 [1:v] crop=576:720,setpts=PTS-STARTPTS [upperright];\
 [upperleft][upperright] hstack=shortest=1[v];\
 [0:a]asetpts=PTS-STARTPTS[ul];[1:a]asetpts=PTS-STARTPTS[ur]; \
 [ul][ur]amix[a]" \
-map [v] -map [a] -c:a libfdk_aac -ar 44100 -threads 32 \
-c:v libx264 -g 50 -preset ultrafast -tune zerolatency \
-f flv rtmp://myserver.tld/live/new_stream_key

对于相同的输入，

ffmpeg -re -i rtmp://1 \
-filter_complex \
"[0:v]crop=576:720,split[ulv][urv]; \
 [ulv][urv] hstack=shortest=1[v]" \
-map [v] -map 0:a -c:a libfdk_aac -ar 44100 -threads 32 \
-c:v libx264 -g 50 -preset ultrafast -tune zerolatency \
-f flv rtmp://myserver.tld/live/new_stream_key