windows - 如何在 RTMP 流中嵌入 pic_timing SEI 挂钟时间码？

Question

我需要将我的桌面流式传输到 AWS MediaLive 服务，并且作为要求，我必须在流中包含挂钟时间码。 AWS 支持人员亲切地通知我，对于 h.264 编码流，我需要提供时间码作为“pic_timing SEI 消息”。

我正在通过 Windows 10 上的 RTMP 协议(protocol)使用 FFmpeg 进行流式传输，所以我尝试添加 use_wallclock_as_timestamps和 copyts标志我的命令。

ffmpeg -f gdigrab -framerate 30 -offset_x 0 -offset_y 0 -video_size 1920x1080 -show_region 1 -use_wallclock_as_timestamps 1 -i desktop -vf scale=320:240 -c:v libx264 -c:a aac -profile:v main -level 3.1 -pix_fmt yuv420p -copyts -f flv rtmp://<ip>:1935/<app>/<stream>

但是，AWS MediaLive 不会获取时间码。

我的问题是:

use_wallclock_as_timestamps flag 实际上根据需要将时间码创建为“pic_timing SEI 消息”？

如果没有，如何将挂钟时间码添加为“pic_timing SEI 消息”？它不必是每一帧。每 2 或 3 秒就足够了。

我难住了。我在 FFmpeg 文档中找不到答案。

谢谢你的帮助。

Answer 1

对于“pic_timing SEI 消息”支持，唯一直接公开的编码器包装是 h264_qsv和 h264_vaapi .您可以通过以下方式确认对相同的支持:
(一个)。对于h264_vaapi编码器包装器(仅适用于 Linux):

ffmpeg -h encoder=h264_vaapi

Encoder h264_vaapi [H.264/AVC (VAAPI)]:
    General capabilities: delay hardware 
    Threading capabilities: none
    Supported hardware devices: vaapi 
    Supported pixel formats: vaapi_vld
h264_vaapi AVOptions:
  -low_power         <boolean>    E..V....... Use low-power encoding mode (only available on some platforms; may not support all encoding features) (default false)
  -idr_interval      <int>        E..V....... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
  -b_depth           <int>        E..V....... Maximum B-frame reference depth (from 1 to INT_MAX) (default 1)
  -rc_mode           <int>        E..V....... Set rate control mode (from 0 to 6) (default auto)
     auto            0            E..V....... Choose mode automatically based on other parameters
     CQP             1            E..V....... Constant-quality
     CBR             2            E..V....... Constant-bitrate
     VBR             3            E..V....... Variable-bitrate
     ICQ             4            E..V....... Intelligent constant-quality
     QVBR            5            E..V....... Quality-defined variable-bitrate
     AVBR            6            E..V....... Average variable-bitrate
  -qp                <int>        E..V....... Constant QP (for P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default 0)
  -quality           <int>        E..V....... Set encode quality (trades off against speed, higher is faster) (from -1 to INT_MAX) (default -1)
  -coder             <int>        E..V....... Entropy coder type (from 0 to 1) (default cabac)
     cavlc           0            E..V.......
     cabac           1            E..V.......
     vlc             0            E..V.......
     ac              1            E..V.......
  -aud               <boolean>    E..V....... Include AUD (default false)
  -sei               <flags>      E..V....... Set SEI to include (default identifier+timing+recovery_point)
     identifier                   E..V....... Include encoder version identifier
     timing                       E..V....... Include timing parameters (buffering_period and pic_timing)
     recovery_point               E..V....... Include recovery points where appropriate
  -profile           <int>        E..V....... Set profile (profile_idc and constraint_set*_flag) (from -99 to 65535) (default -99)
     constrained_baseline 578          E..V.......
     main            77           E..V.......
     high            100          E..V.......
  -level             <int>        E..V....... Set level (level_idc) (from -99 to 255) (default -99)
     1               10           E..V.......
     1.1             11           E..V.......
     1.2             12           E..V.......
     1.3             13           E..V.......
     2               20           E..V.......
     2.1             21           E..V.......
     2.2             22           E..V.......
     3               30           E..V.......
     3.1             31           E..V.......
     3.2             32           E..V.......
     4               40           E..V.......
     4.1             41           E..V.......
     4.2             42           E..V.......
     5               50           E..V.......
     5.1             51           E..V.......
     5.2             52           E..V.......
     6               60           E..V.......
     6.1             61           E..V.......
     6.2             62           E..V.......

对于 VAAPI，私有(private)编解码器选项 -sei:v +timing必须传递给 h264_vaapi .
(b)。对于 h264_qsv编码器包装:

ffmpeg -h encoder=h264_qsv

Encoder h264_qsv [H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration)]:
    General capabilities: delay hybrid 
    Threading capabilities: none
    Supported hardware devices: qsv qsv qsv 
    Supported pixel formats: nv12 p010le qsv
h264_qsv encoder AVOptions:
  -async_depth       <int>        E..V....... Maximum processing parallelism (from 1 to INT_MAX) (default 4)
  -avbr_accuracy     <int>        E..V....... Accuracy of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -avbr_convergence  <int>        E..V....... Convergence of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -preset            <int>        E..V....... (from 1 to 7) (default medium)
     veryfast        7            E..V.......
     faster          6            E..V.......
     fast            5            E..V.......
     medium          4            E..V.......
     slow            3            E..V.......
     slower          2            E..V.......
     veryslow        1            E..V.......
  -rdo               <int>        E..V....... Enable rate distortion optimization (from -1 to 1) (default -1)
  -max_frame_size    <int>        E..V....... Maximum encoded frame size in bytes (from -1 to 65535) (default -1)
  -max_slice_size    <int>        E..V....... Maximum encoded slice size in bytes (from -1 to 65535) (default -1)
  -bitrate_limit     <int>        E..V....... Toggle bitrate limitations (from -1 to 1) (default -1)
  -mbbrc             <int>        E..V....... MB level bitrate control (from -1 to 1) (default -1)
  -extbrc            <int>        E..V....... Extended bitrate control (from -1 to 1) (default -1)
  -adaptive_i        <int>        E..V....... Adaptive I-frame placement (from -1 to 1) (default -1)
  -adaptive_b        <int>        E..V....... Adaptive B-frame placement (from -1 to 1) (default -1)
  -b_strategy        <int>        E..V....... Strategy to choose between I/P/B-frames (from -1 to 1) (default -1)
  -forced_idr        <boolean>    E..V....... Forcing I frames as IDR frames (default false)
  -low_power         <boolean>    E..V....... enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.) (default false)
  -cavlc             <int>        E..V....... Enable CAVLC (from 0 to 1) (default 0)
  -idr_interval      <int>        E..V....... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
  -pic_timing_sei    <int>        E..V....... Insert picture timing SEI with pic_struct_syntax element (from 0 to 1) (default 1)
  -single_sei_nal_unit <int>        E..V....... Put all the SEI messages into one NALU (from -1 to 1) (default -1)
  -max_dec_frame_buffering <int>        E..V....... Maximum number of frames buffered in the DPB (from 0 to 65535) (default 0)
  -look_ahead        <int>        E..V....... Use VBR algorithm with look ahead (from 0 to 1) (default 0)
  -look_ahead_depth  <int>        E..V....... Depth of look ahead in number frames (from 0 to 100) (default 0)
  -look_ahead_downsampling <int>        E..V....... Downscaling factor for the frames saved for the lookahead analysis (from 0 to 3) (default unknown)
     unknown         0            E..V.......
     auto            0            E..V.......
     off             1            E..V.......
     2x              2            E..V.......
     4x              3            E..V.......
  -int_ref_type      <int>        E..V....... Intra refresh type (from -1 to 65535) (default -1)
     none            0            E..V.......
     vertical        1            E..V.......
  -int_ref_cycle_size <int>        E..V....... Number of frames in the intra refresh cycle (from -1 to 65535) (default -1)
  -int_ref_qp_delta  <int>        E..V....... QP difference for the refresh MBs (from -32768 to 32767) (default -32768)
  -recovery_point_sei <int>        E..V....... Insert recovery point SEI messages (from -1 to 1) (default -1)
  -profile           <int>        E..V....... (from 0 to INT_MAX) (default unknown)
     unknown         0            E..V.......
     baseline        66           E..V.......
     main            77           E..V.......
     high            100          E..V.......
  -a53cc             <int>        E..V....... Use A53 Closed Captions (if available) (from 0 to 1) (default 1)
  -aud               <int>        E..V....... Insert the Access Unit Delimiter NAL (from 0 to 1) (default 0)
  -mfmode            <int>        E..V....... Multi-Frame Mode (from 0 to 2) (default auto)
     off             1            E..V.......
     auto            2            E..V.......
  -repeat_pps        <boolean>    E..V....... repeat pps for every frame (default false)

QSV 编码器包装器需要私有(private)编解码器选项 -pic_timing_sei:v 1要设置，这是默认设置。这可以通过传递 0 被覆盖以禁用它。到上述参数。
更新:
将来，这也可能在 h264_nvenc 上实现。和 libx264编码器包装器，在邮件列表中有待完成的补丁工作:

对于 nvenc:https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210525111635.29482-2-bradh@frogmouth.net/

对于 libx264:https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210517081944.35238-2-bradh@frogmouth.net/

我没有在答案中包含任何 HEVC 编码器包装器，因为 RTMP 不支持 HEVC。