c++ - 使用 FFMPEG 将 RGB 图像序列保存到 .mp4 时遇到问题

Question

我用 OpenGL 渲染了一些图像，需要将它们组合成一个视频文件。每个图像都是代表 sRGB 颜色分量的 uint8_t 值序列(图像数组看起来像 ...rgbrgbrgb ...)

我对视频处理知之甚少，根本没有使用 ffmpeg 库的经验。我使用这些资源作为引用做了一个小测试程序:

https://ffmpeg.org/doxygen/trunk/encode_video_8c-example.html

How to convert RGB from YUV420p for ffmpeg encoder?

该测试程序应该制作一个关于生长绿色垂直条纹的视频。我只是想弄清楚如何使用一些原始 RGB 数据源制作视频。

这是我的代码:

#include <iostream>
#include <vector>
#include <algorithm>

extern "C" {
    #include <libavcodec/avcodec.h>
    #include <libavutil/opt.h>
    #include <libavutil/imgutils.h>
    #include <libswscale/swscale.h>
}

static void encode( AVCodecContext* enc_ctx,
                    AVFrame* frame, AVPacket* pkt,
                    FILE* outfile                  )
{
    int ret;
    ret = avcodec_send_frame(enc_ctx, frame);
    if (ret < 0) {
        std::cerr << "Error sending a frame for encoding\n";
        return;
    }
    while (ret >= 0) {
        ret = avcodec_receive_packet(enc_ctx, pkt);
        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
            return;
        else if (ret < 0) {
            fprintf(stderr, "Error during encoding\n");
            exit(1);
        }
        fwrite(pkt->data, 1, pkt->size, outfile);
        av_packet_unref(pkt);
    }
}

static constexpr int w = 1920, h = 1080;
static constexpr float fps = 20.f, time = 5.f;
static constexpr int nFrames = static_cast<int>(fps * time);
static std::vector<uint8_t> imageRGB(w * h * 3, 0);

static void UpdateImageRGB()
{
    static int d = 50;
    imageRGB.assign(w * h * 3, 0);
    for (int i = 0; i < h; ++i)
        for ( int j = std::max(0, w / 2 - d);
              j < std::min(w, w / 2 + d);
              ++j                             )
        {
            imageRGB[(w * i + j) * 3 + 0] = 50;
            imageRGB[(w * i + j) * 3 + 1] = 200;
            imageRGB[(w * i + j) * 3 + 2] = 50;
        }
    d += 5;
}

int main()
{
    int ret = 0;
    auto filename = "test.mp4";

    auto codec = avcodec_find_encoder(AV_CODEC_ID_H264);
    if (!codec) {
        std::cerr << "Codec \"x.264\" not found\n";
        return 1;
    }
    auto c = avcodec_alloc_context3(codec);
    if (!c) {
        std::cerr << "Could not allocate video codec context\n";
        return 1;
    }
    auto pkt = av_packet_alloc();
    if (!pkt) return 1;

    // 1.8 bits / (pixel * frame)
    c->bit_rate = static_cast<int64_t>(1.8f * w * h * fps);
    /* resolution must be a multiple of two */
    c->width = w;
    c->height = h;
    /* frames per second */
    c->time_base = AVRational{ 1, static_cast<int>(fps) };
    c->framerate = AVRational{ static_cast<int>(fps), 1 };

    c->gop_size = 10;
    c->max_b_frames = 1;
    c->pix_fmt = AV_PIX_FMT_YUV420P;
    av_opt_set(c->priv_data, "preset", "slow", 0);
    av_opt_set(c->priv_data, "preset", "slow", 0);

    ret = avcodec_open2(c, codec, NULL);
    if (ret < 0) {
        char str[AV_ERROR_MAX_STRING_SIZE];
        std::cerr << "Could not open codec: "
                  << av_make_error_string(str, AV_ERROR_MAX_STRING_SIZE, ret)
                  << "\n";
        return 1;
    }

    FILE * f;
    fopen_s(&f, filename, "wb");
    if (!f) {
        std::cerr << "Could not open " << filename << '\n';
        return 1;
    }

    auto frame = av_frame_alloc();
    if (!frame) {
        std::cerr << "Could not allocate video frame\n";
        return 1;
    }
    frame->format = c->pix_fmt;
    frame->width = c->width;
    frame->height = c->height;
    ret = av_frame_get_buffer(frame, 0);
    if (ret < 0) {
        std::cerr << stderr, "Could not allocate the video frame data\n";
        return 1;
    }

    SwsContext* ctx = sws_getContext( w, h, AV_PIX_FMT_RGB24,
                                      w, h, AV_PIX_FMT_YUV420P,
                                      0, 0, 0, 0                );

    for (int i = 0; i < nFrames; i++)
    {
        ret = av_frame_make_writable(frame);
        UpdateImageRGB();
        static const uint8_t* rgbData[1] = { &imageRGB[0] };
        static constexpr int rgbLinesize[1] = { 3 * w };
        sws_scale( ctx, rgbData, rgbLinesize, 0, h,
                   frame->data, frame->linesize     );
        frame->pts = i;
        /* encode the image */
        encode(c, frame, pkt, f);
    }
    encode(c, NULL, pkt, f);

    fclose(f);
    avcodec_free_context(&c);
    av_frame_free(&frame);
    av_packet_free(&pkt);
    return 0;
}

该程序生成 33.9k 视频文件，并带有进一步的控制台输出:

[libx264 @ 0000020c18681800] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000020c18681800] profile High, level 5.0, 4:2:0, 8-bit
[libx264 @ 0000020c18681800] frame I:11    Avg QP: 0.00  size:   639
[libx264 @ 0000020c18681800] frame P:74    Avg QP: 0.32  size:   174
[libx264 @ 0000020c18681800] frame B:15    Avg QP: 2.26  size:   990
[libx264 @ 0000020c18681800] consecutive B-frames: 70.0% 30.0%
[libx264 @ 0000020c18681800] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0000020c18681800] mb P  I16..4:  0.6%  0.0%  0.0%  P16..4:  2.1%  0.0%  0.0%  0.0%  0.0%    skip:97.3%
[libx264 @ 0000020c18681800] mb B  I16..4:  0.1%  0.0%  0.0%  B16..8:  0.6%  0.0%  0.0%  direct: 0.6%  skip:98.7%  L0:39.8% L1:60.2% BI: 0.0%
[libx264 @ 0000020c18681800] final ratefactor: -46.47
[libx264 @ 0000020c18681800] 8x8 transform intra:0.0%
[libx264 @ 0000020c18681800] direct mvs  spatial:0.0% temporal:100.0%
[libx264 @ 0000020c18681800] coded y,uvDC,uvAC intra: 0.0% 0.1% 0.1% inter: 0.0% 0.1% 0.1%
[libx264 @ 0000020c18681800] i16 v,h,dc,p: 99%  0%  1%  0%
[libx264 @ 0000020c18681800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  0%  0% 100%  0%  0%  0%  0%  0%  0%
[libx264 @ 0000020c18681800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 46%  0% 54%  0%  0%  0%  0%  0%  0%
[libx264 @ 0000020c18681800] i8c dc,h,v,p: 96%  1%  3%  0%
[libx264 @ 0000020c18681800] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000020c18681800] ref P L0: 70.2%  0.0% 29.8%  0.0%  0.0%
[libx264 @ 0000020c18681800] kb/s:55.61

Windows 上的“Media Player Classic”播放此视频，但时间 slider 不动，视频无法快进到某些帧

VLC 根本无法播放视频。它启动，显示 VLC Logo ，时间 slider (异常大)从左到右跳跃，没有响应我的点击

如果我设置时间 = 0.05 来制作只有 1 帧的视频，即使使用“Media Player Classic”也无法播放。我想制定一种算法，将任意数量的原始 RGB 图像转换为视频文件，即使只有一个图像，并且具有任意图像大小(即宽度和高度可能是奇数)。

正如我所说，我真的不明白我在做什么。第 83-84 行有低级编解码器设置。他们都好吗？

我是否必须手动设置比特率(第 75 行)？它不应该由编解码器自动计算吗？

Answer 1

嘿，我通过引用此处使用的方法解决了这个问题。我不知道您是否可以在代码中应用它，但我建议您检查一下:

https://superuser.com/questions/469273/ffmpeg-convert-rgb-images-to-video