个人中心
gpt4 book ai didi

c++ - 使用 libavformat 不播放的 H.264 混合到 MP4

转载 作者：IT老高 更新时间：2023-10-28 22:22:24 28 4
gpt4 key购买 nike

我正在尝试将 H.264 数据复用到 MP4 文件中。将此 H.264 Annex B 数据保存为 MP4 文件时似乎没有错误，但该文件无法播放。

我对文件进行了二进制比较，问题似乎出在写入 MP4 文件页脚(预告片)的内容中。

我怀疑它必须与创建流的方式有关。

在里面:

AVOutputFormat* fmt = av_guess_format( 0, "out.mp4", 0 );
oc = avformat_alloc_context();
oc->oformat = fmt;
strcpy(oc->filename, filename);

我拥有的这个原型(prototype)应用程序的一部分是为每个 IFrame 创建一个 png 文件。因此，当遇到第一个 IFrame 时，我创建视频流并写入 av header 等: 
void addVideoStream(AVCodecContext* decoder)
{
    videoStream = av_new_stream(oc, 0);
    if (!videoStream)
    {
         cout << "ERROR creating video stream" << endl;
         return;        
    }
    vi = videoStream->index;    
    videoContext = videoStream->codec;      
    videoContext->codec_type = AVMEDIA_TYPE_VIDEO;
    videoContext->codec_id = decoder->codec_id;
    videoContext->bit_rate = 512000;
    videoContext->width = decoder->width;
    videoContext->height = decoder->height;
    videoContext->time_base.den = 25;
    videoContext->time_base.num = 1;    
    videoContext->gop_size = decoder->gop_size;
    videoContext->pix_fmt = decoder->pix_fmt;       

    if (oc->oformat->flags & AVFMT_GLOBALHEADER)
        videoContext->flags |= CODEC_FLAG_GLOBAL_HEADER;

    av_dump_format(oc, 0, filename, 1);

    if (!(oc->oformat->flags & AVFMT_NOFILE))
    {
        if (avio_open(&oc->pb, filename, AVIO_FLAG_WRITE) < 0) {
        cout << "Error opening file" << endl;
    }
    avformat_write_header(oc, NULL);
}

我写出数据包: 
unsigned char* data = block->getData();
unsigned char videoFrameType = data[4];
int dataLen = block->getDataLen();

// store pps
if (videoFrameType == 0x68)
{
    if (ppsFrame != NULL)
    {
        delete ppsFrame; ppsFrameLength = 0; ppsFrame = NULL;
    }
    ppsFrameLength = block->getDataLen();
    ppsFrame = new unsigned char[ppsFrameLength];
    memcpy(ppsFrame, block->getData(), ppsFrameLength);
}
else if (videoFrameType == 0x67)
{
    // sps
    if (spsFrame != NULL)
    {
        delete spsFrame; spsFrameLength = 0; spsFrame = NULL;
}
    spsFrameLength = block->getDataLen();
    spsFrame = new unsigned char[spsFrameLength];
    memcpy(spsFrame, block->getData(), spsFrameLength);                 
}                                           

if (videoFrameType == 0x65 || videoFrameType == 0x41)
{
    videoFrameNumber++;
}
if (videoFrameType == 0x65)
{
    decodeIFrame(videoFrameNumber, spsFrame, spsFrameLength, ppsFrame, ppsFrameLength, data, dataLen);
}

if (videoStream != NULL)
{
    AVPacket pkt = { 0 };
    av_init_packet(&pkt);
    pkt.stream_index = vi;
    pkt.flags = 0;                      
    pkt.pts = pkt.dts = 0;                                  

    if (videoFrameType == 0x65)
    {
        // combine the SPS PPS & I frames together
        pkt.flags |= AV_PKT_FLAG_KEY;                                                   
        unsigned char* videoFrame = new unsigned char[spsFrameLength+ppsFrameLength+dataLen];
        memcpy(videoFrame, spsFrame, spsFrameLength);
        memcpy(&videoFrame[spsFrameLength], ppsFrame, ppsFrameLength);
        memcpy(&videoFrame[spsFrameLength+ppsFrameLength], data, dataLen);

        // overwrite the start code (00 00 00 01 with a 32-bit length)
        setLength(videoFrame, spsFrameLength-4);
        setLength(&videoFrame[spsFrameLength], ppsFrameLength-4);
        setLength(&videoFrame[spsFrameLength+ppsFrameLength], dataLen-4);
        pkt.size = dataLen + spsFrameLength + ppsFrameLength;
        pkt.data = videoFrame;
        av_interleaved_write_frame(oc, &pkt);
        delete videoFrame; videoFrame = NULL;
    }
    else if (videoFrameType != 0x67 && videoFrameType != 0x68)
    {   
        // Send other frames except pps & sps which are caught and stored                   
        pkt.size = dataLen;
        pkt.data = data;
        setLength(data, dataLen-4);                     
        av_interleaved_write_frame(oc, &pkt);
    }

最后关闭文件: 
av_write_trailer(oc);
int i = 0;
for (i = 0; i < oc->nb_streams; i++)
{
    av_freep(&oc->streams[i]->codec);
    av_freep(&oc->streams[i]);      
}

if (!(oc->oformat->flags & AVFMT_NOFILE))
{
    avio_close(oc->pb);
}
av_free(oc);

如果我单独使用 H.264 数据并对其进行转换: 
ffmpeg -i recording.h264 -vcodec copy recording.mp4

除了文件的“页脚”之外，所有文件都是相同的。

我的程序的输出:
readrec 录音.tcp out.mp4
**** 开始 **** 01-03-2013 14:26:01 180000
输出 #0，mp4，到 'out.mp4':
流 #0:0:视频:h264、yuv420p、352x288、q=2-31、512 kb/s、90k tbn、25 tbc
**** 完 **** 01-03-2013 14:27:01 102000
写了 1499 个视频帧。

如果我尝试使用 ffmpeg 转换使用 CODE 创建的 MP4 文件: 
ffmpeg -i out.mp4 -vcodec copy out2.mp4
ffmpeg version 0.11.1 Copyright (c) 2000-2012 the FFmpeg developers
      built on Mar  7 2013 12:49:22 with suncc 0x5110
      configuration: --extra-cflags=-KPIC -g --disable-mmx
      --disable-protocol=udp --disable-encoder=nellymoser --cc=cc --cxx=CC
libavutil      51. 54.100 / 51. 54.100
libavcodec     54. 23.100 / 54. 23.100
libavformat    54.  6.100 / 54.  6.100
libavdevice    54.  0.100 / 54.  0.100
libavfilter     2. 77.100 /  2. 77.100
libswscale      2.  1.100 /  2.  1.100
libswresample   0. 15.100 /  0. 15.100
h264 @ 12eaac0] no frame!
    Last message repeated 1 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 23 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 74 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 64 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 34 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 49 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 24 times
[h264 @ 12eaac0] Partitioned H.264 support is incomplete
[h264 @ 12eaac0] no frame!
    Last message repeated 23 times
[h264 @ 12eaac0] sps_id out of range
[h264 @ 12eaac0] no frame!
    Last message repeated 148 times
[h264 @ 12eaac0] sps_id (32) out of range
    Last message repeated 1 times
[h264 @ 12eaac0] no frame!
    Last message repeated 33 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 128 times
[h264 @ 12eaac0] sps_id (32) out of range
    Last message repeated 1 times
[h264 @ 12eaac0] no frame!
    Last message repeated 3 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 3 times
[h264 @ 12eaac0] slice type too large (0) at 0 0
[h264 @ 12eaac0] decode_slice_header error
[h264 @ 12eaac0] no frame!
    Last message repeated 309 times
[h264 @ 12eaac0] sps_id (32) out of range
    Last message repeated 1 times
[h264 @ 12eaac0] no frame!
    Last message repeated 192 times
[h264 @ 12eaac0] Partitioned H.264 support is incomplete
[h264 @ 12eaac0] no frame!
    Last message repeated 73 times
[h264 @ 12eaac0] sps_id (32) out of range
    Last message repeated 1 times
[h264 @ 12eaac0] no frame!
    Last message repeated 99 times
[h264 @ 12eaac0] sps_id (32) out of range
    Last message repeated 1 times
[h264 @ 12eaac0] no frame!
    Last message repeated 197 times
[mov,mp4,m4a,3gp,3g2,mj2 @ 12e3100] decoding for stream 0 failed
[mov,mp4,m4a,3gp,3g2,mj2 @ 12e3100] Could not find codec parameters
(Video: h264 (avc1 / 0x31637661), 393539 kb/s)
out.mp4: could not find codec parameters

我真的不知道问题出在哪里，除了它必须与流的设置方式有关。我查看了其他人正在做类似事情的代码，并尝试使用此建议来设置流，但无济于事!

给我一个 H.264/AAC 混合(同步)文件的最终代码如下。首先是一些背景信息。数据来自网络摄像机。数据通过第 3 方 API 以视频/音频数据包的形式呈现。视频数据包作为 RTP 有效载荷数据(无 header )呈现，并由重建并转换为附件 B 格式的 H.264 视频的 NALU 组成。 AAC 音频显示为原始 AAC，并转换为 adts 格式以启用播放。这些数据包已被放入比特流格式，允许传输时间戳(自 1970 年 1 月 1 日以来的 64 位毫秒)以及其他一些内容。

这或多或少是一个原型(prototype)，在任何方面都不干净。可能漏水不好。但是，我愿意，希望这可以帮助其他人尝试实现与我相似的目标。

全局: 
AVFormatContext* oc = NULL;
AVCodecContext* videoContext = NULL;
AVStream* videoStream = NULL;
AVCodecContext* audioContext = NULL;
AVStream* audioStream = NULL;
AVCodec* videoCodec = NULL;
AVCodec* audioCodec = NULL;
int vi = 0;  // Video stream
int ai = 1;  // Audio stream

uint64_t firstVideoTimeStamp = 0;
uint64_t firstAudioTimeStamp = 0;
int audioStartOffset = 0;

char* filename = NULL;

Boolean first = TRUE;

int videoFrameNumber = 0;
int audioFrameNumber = 0;

主要的: 
int main(int argc, char* argv[])
{
    if (argc != 3)
    {   
        cout << argv[0] << " <stream playback file> <output mp4 file>" << endl;
        return 0;
    }
    char* input_stream_file = argv[1];
    filename = argv[2];

    av_register_all();    

    fstream inFile;
    inFile.open(input_stream_file, ios::in);

    // Used to store the latest pps & sps frames
    unsigned char* ppsFrame = NULL;
    int ppsFrameLength = 0;
    unsigned char* spsFrame = NULL;
    int spsFrameLength = 0;

    // Setup MP4 output file
    AVOutputFormat* fmt = av_guess_format( 0, filename, 0 );
    oc = avformat_alloc_context();
    oc->oformat = fmt;
    strcpy(oc->filename, filename);

    // Setup the bitstream filter for AAC in adts format.  Could probably also achieve
    // this by stripping the first 7 bytes!
    AVBitStreamFilterContext* bsfc = av_bitstream_filter_init("aac_adtstoasc");
    if (!bsfc)
    {       
        cout << "Error creating adtstoasc filter" << endl;
        return -1;
    }

    while (inFile.good())
    {
        TcpAVDataBlock* block = new TcpAVDataBlock();
        block->readStruct(inFile);
        DateTime dt = block->getTimestampAsDateTime();
        switch (block->getPacketType())
        {
            case TCP_PACKET_H264:
            {       
                if (firstVideoTimeStamp == 0)
                    firstVideoTimeStamp = block->getTimeStamp();
                unsigned char* data = block->getData();
                unsigned char videoFrameType = data[4];
                int dataLen = block->getDataLen();

                // pps
                if (videoFrameType == 0x68)
                {
                    if (ppsFrame != NULL)
                    {
                        delete ppsFrame; ppsFrameLength = 0;
                        ppsFrame = NULL;
                    }
                    ppsFrameLength = block->getDataLen();
                    ppsFrame = new unsigned char[ppsFrameLength];
                    memcpy(ppsFrame, block->getData(), ppsFrameLength);
                }
                else if (videoFrameType == 0x67)
                {
                    // sps
                    if (spsFrame != NULL)
                    {
                        delete spsFrame; spsFrameLength = 0;
                        spsFrame = NULL;
                    }
                    spsFrameLength = block->getDataLen();
                    spsFrame = new unsigned char[spsFrameLength];
                    memcpy(spsFrame, block->getData(), spsFrameLength);                   
                }                                           

                if (videoFrameType == 0x65 || videoFrameType == 0x41)
                {
                    videoFrameNumber++;
                }
                // Extract a thumbnail for each I-Frame
                if (videoFrameType == 0x65)
                {
                    decodeIFrame(h264, spsFrame, spsFrameLength, ppsFrame, ppsFrameLength, data, dataLen);
                }
                if (videoStream != NULL)
                {
                    AVPacket pkt = { 0 };
                    av_init_packet(&pkt);
                    pkt.stream_index = vi;
                    pkt.flags = 0;           
                    pkt.pts = videoFrameNumber;
                    pkt.dts = videoFrameNumber;           
                    if (videoFrameType == 0x65)
                    {
                        pkt.flags = 1;                           

                        unsigned char* videoFrame = new unsigned char[spsFrameLength+ppsFrameLength+dataLen];
                        memcpy(videoFrame, spsFrame, spsFrameLength);
                        memcpy(&videoFrame[spsFrameLength], ppsFrame, ppsFrameLength);

                        memcpy(&videoFrame[spsFrameLength+ppsFrameLength], data, dataLen);
                        pkt.data = videoFrame;
                        av_interleaved_write_frame(oc, &pkt);
                        delete videoFrame; videoFrame = NULL;
                    }
                    else if (videoFrameType != 0x67 && videoFrameType != 0x68)
                    {                       
                        pkt.size = dataLen;
                        pkt.data = data;
                        av_interleaved_write_frame(oc, &pkt);
                    }                       
                }
                break;
            }

        case TCP_PACKET_AAC:

            if (firstAudioTimeStamp == 0)
            {
                firstAudioTimeStamp = block->getTimeStamp();
                uint64_t millseconds_difference = firstAudioTimeStamp - firstVideoTimeStamp;
                audioStartOffset = millseconds_difference * 16000 / 1000;
                cout << "audio offset: " << audioStartOffset << endl;
            }

            if (audioStream != NULL)
            {
                AVPacket pkt = { 0 };
                av_init_packet(&pkt);
                pkt.stream_index = ai;
                pkt.flags = 1;           
                pkt.pts = audioFrameNumber*1024;
                pkt.dts = audioFrameNumber*1024;
                pkt.data = block->getData();
                pkt.size = block->getDataLen();
                pkt.duration = 1024;

                AVPacket newpacket = pkt;                       
                int rc = av_bitstream_filter_filter(bsfc, audioContext,
                    NULL,
                    &newpacket.data, &newpacket.size,
                    pkt.data, pkt.size,
                    pkt.flags & AV_PKT_FLAG_KEY);

                if (rc >= 0)
                {
                    //cout << "Write audio frame" << endl;
                    newpacket.pts = audioFrameNumber*1024;
                    newpacket.dts = audioFrameNumber*1024;
                    audioFrameNumber++;
                    newpacket.duration = 1024;                   

                    av_interleaved_write_frame(oc, &newpacket);
                    av_free_packet(&newpacket);
                }   
                else
                {
                    cout << "Error filtering aac packet" << endl;

                }
            }
            break;

        case TCP_PACKET_START:
            break;

        case TCP_PACKET_END:
            break;
        }
        delete block;
    }
    inFile.close();

    av_write_trailer(oc);
    int i = 0;
    for (i = 0; i < oc->nb_streams; i++)
    {
        av_freep(&oc->streams[i]->codec);
        av_freep(&oc->streams[i]);       
    }

    if (!(oc->oformat->flags & AVFMT_NOFILE))
    {
        avio_close(oc->pb);
    }

    av_free(oc);

    delete spsFrame; spsFrame = NULL;
    delete ppsFrame; ppsFrame = NULL;

    cout << "Wrote " << videoFrameNumber << " video frames." << endl;

    return 0;
}

添加流流/编解码器，并在名为 addVideoAndAudioStream() 的函数中创建 header 。这个函数是从 decodeIFrame() 调用的，所以有一些假设(不一定很好)
1.先有视频包
2. AAC 存在

decodeIFrame 是一种单独的原型(prototype)，我在其中为每个 I Frame 创建了一个缩略图。生成缩略图的代码来自: https://gnunet.org/svn/Extractor/src/plugins/thumbnailffmpeg_extractor.c

decodeIFrame 函数将 AVCodecContext 传递给 addVideoAudioStream: 
void addVideoAndAudioStream(AVCodecContext* decoder = NULL)
{
    videoStream = av_new_stream(oc, 0);
    if (!videoStream)
    {
        cout << "ERROR creating video stream" << endl;
        return;       
    }
    vi = videoStream->index;   
    videoContext = videoStream->codec;       
    videoContext->codec_type = AVMEDIA_TYPE_VIDEO;
    videoContext->codec_id = decoder->codec_id;
    videoContext->bit_rate = 512000;
    videoContext->width = decoder->width;
    videoContext->height = decoder->height;
    videoContext->time_base.den = 25;
    videoContext->time_base.num = 1;
    videoContext->gop_size = decoder->gop_size;
    videoContext->pix_fmt = decoder->pix_fmt;       

    audioStream = av_new_stream(oc, 1);
    if (!audioStream)
    {
        cout << "ERROR creating audio stream" << endl;
        return;
    }
    ai = audioStream->index;
    audioContext = audioStream->codec;
    audioContext->codec_type = AVMEDIA_TYPE_AUDIO;
    audioContext->codec_id = CODEC_ID_AAC;
    audioContext->bit_rate = 64000;
    audioContext->sample_rate = 16000;
    audioContext->channels = 1;

    if (oc->oformat->flags & AVFMT_GLOBALHEADER)
    {
        videoContext->flags |= CODEC_FLAG_GLOBAL_HEADER;
        audioContext->flags |= CODEC_FLAG_GLOBAL_HEADER;
    }

    av_dump_format(oc, 0, filename, 1);

    if (!(oc->oformat->flags & AVFMT_NOFILE))
    {
        if (avio_open(&oc->pb, filename, AVIO_FLAG_WRITE) < 0) {
            cout << "Error opening file" << endl;
        }
    }

    avformat_write_header(oc, NULL);
}

据我所知，一些假设似乎并不重要，例如:
1. 比特率。实际视频比特率约为 262k，而我指定的是 512kbit
2. AAC channel 。我指定了单声道，虽然实际输出是内存中的立体声

您仍然需要知道视频和音频的帧速率(时基)是多少。

与许多其他示例相反，在视频数据包上设置 pts & dts 时，它无法播放。我需要知道时基 (25fps)，然后根据该时基设置 pts 和 dts，即第一帧 = 0(PPS、SPS、I)，第二帧 = 1(中间帧，不管它叫什么；)) .

AAC 我还必须假设它是 16000 hz。每个 AAC 数据包 1024 个样本(我认为您也可以使用 AAC @ 960 个样本)来确定音频“偏移”。我将此添加到 pts & dts。因此 pts/dts 是要播放的样本编号。您还需要确保在写入之前在数据包中设置了 1024 的持续时间。

——

我今天另外发现附件 B 与任何其他播放器并不真正兼容，因此应该真正使用 AVCC 格式。

这些 URL 有助于:
Problem to Decode H264 video over RTP with ffmpeg (libavcodec)
http://aviadr1.blogspot.com.au/2010/05/h264-extradata-partially-explained-for.html

在构建视频流时，我填写了 extradata 和 extradata_size: 
// Extradata contains PPS & SPS for AVCC format
int extradata_len = 8 + spsFrameLen-4 + 1 + 2 + ppsFrameLen-4;
videoContext->extradata = (uint8_t*)av_mallocz(extradata_len);
videoContext->extradata_size = extradata_len;
videoContext->extradata[0] = 0x01;
videoContext->extradata[1] = spsFrame[4+1];
videoContext->extradata[2] = spsFrame[4+2];
videoContext->extradata[3] = spsFrame[4+3];
videoContext->extradata[4] = 0xFC | 3;
videoContext->extradata[5] = 0xE0 | 1;
int tmp = spsFrameLen - 4;
videoContext->extradata[6] = (tmp >> 8) & 0x00ff;
videoContext->extradata[7] = tmp & 0x00ff;
int i = 0;
for (i=0;i<tmp;i++)
    videoContext->extradata[8+i] = spsFrame[4+i];
videoContext->extradata[8+tmp] = 0x01;
int tmp2 = ppsFrameLen-4;   
videoContext->extradata[8+tmp+1] = (tmp2 >> 8) & 0x00ff;
videoContext->extradata[8+tmp+2] = tmp2 & 0x00ff;
for (i=0;i<tmp2;i++)
    videoContext->extradata[8+tmp+3+i] = ppsFrame[4+i];

写出帧时，不要添加 SPS 和 PPS 帧，只需写出 I 帧和 P 帧。另外，将前4个字节(0x00 0x00 0x00 0x01)中包含的Annex B起始码替换为I/P帧的大小。

最佳答案

请让我总结一下:您(原始)代码的问题在于 av_interleaved_write_frame() 的输入不应以数据包长度开头。如果您不删除 00 00 00 01，该文件可能仍可播放开始代码，但恕我直言是玩家的弹性行为，我不会指望这一点。

关于c++ - 使用 libavformat 不播放的 H.264 混合到 MP4，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/15263458/

28 4 0
文章推荐： c++ - 在 M X N 矩阵中查找 m x n 子矩阵的最快方法
文章推荐： android - 多个 dex 文件定义 Lcom/google/firebase/FirebaseException
文章推荐： python - pyserial:没有名为工具的模块
文章推荐： android - Oreo - 前台服务不显示前台通知
IT老高
个人简介

我是一名优秀的程序员,十分优秀！

滴滴打车优惠券免费领取
滴滴打车优惠券
全站热门文章
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com