个人中心
gpt4 book ai didi

ios - 使用AVAssetReader读取mp4文件时，第一个音频CMSampleBuffer丢失

转载 作者：行者123 更新时间：2023-12-01 21:59:47 26 4
gpt4 key购买 nike

我正在使用AVAssetWriter将音频CMSampleBuffer写入mp4文件，但是当我稍后使用AVAssetReader读取该文件时，似乎丢失了最初的数据块。

这是传递给编写器输入append方法的第一个CMSampleBuffer的调试说明(请注意，启动时长附加为1024 / 44_100):

CMSampleBuffer 0x102ea5b60 retainCount: 7 allocator: 0x1c061f840
            invalid = NO
            dataReady = YES
            makeDataReadyCallback = 0x0
            makeDataReadyRefcon = 0x0
            buffer-level attachments:
                TrimDurationAtStart    = {
            epoch = 0;
            flags = 1;
            timescale = 44100;
            value = 1024;
        }
            formatDescription = <CMAudioFormatDescription 0x281fd9720 [0x1c061f840]> {
            mediaType:'soun' 
            mediaSubType:'aac ' 
            mediaSpecific: {
                ASBD: {
                    mSampleRate: 44100.000000 
                    mFormatID: 'aac ' 
                    mFormatFlags: 0x2 
                    mBytesPerPacket: 0 
                    mFramesPerPacket: 1024 
                    mBytesPerFrame: 0 
                    mChannelsPerFrame: 2 
                    mBitsPerChannel: 0  } 
                cookie: {<CFData 0x2805f50a0 [0x1c061f840]>{length = 39, capacity = 39, bytes = 0x03808080220000000480808014401400 ... 1210068080800102}} 
                ACL: {(null)}
                FormatList Array: {
                    Index: 0 
                    ChannelLayoutTag: 0x650002 
                    ASBD: {
                    mSampleRate: 44100.000000 
                    mFormatID: 'aac ' 
                    mFormatFlags: 0x0 
                    mBytesPerPacket: 0 
                    mFramesPerPacket: 1024 
                    mBytesPerFrame: 0 
                    mChannelsPerFrame: 2 
                    mBitsPerChannel: 0  }} 
            } 
            extensions: {(null)}
        }
            sbufToTrackReadiness = 0x0
            numSamples = 1
            outputPTS = {6683542167/44100 = 151554.244, rounded}(based on cachedOutputPresentationTimeStamp)
            sampleTimingArray[1] = {
                {PTS = {6683541143/44100 = 151554.221, rounded}, DTS = {6683541143/44100 = 151554.221, rounded}, duration = {1024/44100 = 0.023}},
            }
            sampleSizeArray[1] = {
                sampleSize = 163,
            }
            dataBuffer = 0x281cc7a80

这是第二个CMSampleBuffer的调试说明(请注意启动持续时间附件为1088 / 44_100，与先前的修整持续时间相结合，得出的标准值为2112): 
CMSampleBuffer 0x102e584f0 retainCount: 7 allocator: 0x1c061f840
    invalid = NO
    dataReady = YES
    makeDataReadyCallback = 0x0
    makeDataReadyRefcon = 0x0
    buffer-level attachments:
        TrimDurationAtStart    = {
    epoch = 0;
    flags = 1;
    timescale = 44100;
    value = 1088;
}
    formatDescription = <CMAudioFormatDescription 0x281fd9720 [0x1c061f840]> {
    mediaType:'soun' 
    mediaSubType:'aac ' 
    mediaSpecific: {
        ASBD: {
            mSampleRate: 44100.000000 
            mFormatID: 'aac ' 
            mFormatFlags: 0x2 
            mBytesPerPacket: 0 
            mFramesPerPacket: 1024 
            mBytesPerFrame: 0 
            mChannelsPerFrame: 2 
            mBitsPerChannel: 0  } 
        cookie: {<CFData 0x2805f50a0 [0x1c061f840]>{length = 39, capacity = 39, bytes = 0x03808080220000000480808014401400 ... 1210068080800102}} 
        ACL: {(null)}
        FormatList Array: {
            Index: 0 
            ChannelLayoutTag: 0x650002 
            ASBD: {
            mSampleRate: 44100.000000 
            mFormatID: 'aac ' 
            mFormatFlags: 0x0 
            mBytesPerPacket: 0 
            mFramesPerPacket: 1024 
            mBytesPerFrame: 0 
            mChannelsPerFrame: 2 
            mBitsPerChannel: 0  }} 
    } 
    extensions: {(null)}
}
    sbufToTrackReadiness = 0x0
    numSamples = 1
    outputPTS = {6683543255/44100 = 151554.269, rounded}(based on cachedOutputPresentationTimeStamp)
    sampleTimingArray[1] = {
        {PTS = {6683542167/44100 = 151554.244, rounded}, DTS = {6683542167/44100 = 151554.244, rounded}, duration = {1024/44100 = 0.023}},
    }
    sampleSizeArray[1] = {
        sampleSize = 179,
    }
    dataBuffer = 0x281cc4750

现在，当我使用AVAssetReader读取音轨时，我得到的第一个CMSampleBuffer是: 
CMSampleBuffer 0x102ed7b20 retainCount: 7 allocator: 0x1c061f840
    invalid = NO
    dataReady = YES
    makeDataReadyCallback = 0x0
    makeDataReadyRefcon = 0x0
    buffer-level attachments:
        EmptyMedia(P) = true
    formatDescription = (null)
    sbufToTrackReadiness = 0x0
    numSamples = 0
    outputPTS = {0/1 = 0.000}(based on outputPresentationTimeStamp)
    sampleTimingArray[1] = {
        {PTS = {0/1 = 0.000}, DTS = {INVALID}, duration = {0/1 = 0.000}},
    }
    dataBuffer = 0x0

下一个包含启动信息1088 / 44_100: 
CMSampleBuffer 0x10318bc00 retainCount: 7 allocator: 0x1c061f840
    invalid = NO
    dataReady = YES
    makeDataReadyCallback = 0x0
    makeDataReadyRefcon = 0x0
    buffer-level attachments:
        FillDiscontinuitiesWithSilence(P) = true
        GradualDecoderRefresh(P) = 1
        TrimDurationAtStart(P) = {
    epoch = 0;
    flags = 1;
    timescale = 44100;
    value = 1088;
}
        IsGradualDecoderRefreshAuthoritative(P) = false
    formatDescription = <CMAudioFormatDescription 0x281fdcaa0 [0x1c061f840]> {
    mediaType:'soun' 
    mediaSubType:'aac ' 
    mediaSpecific: {
        ASBD: {
            mSampleRate: 44100.000000 
            mFormatID: 'aac ' 
            mFormatFlags: 0x0 
            mBytesPerPacket: 0 
            mFramesPerPacket: 1024 
            mBytesPerFrame: 0 
            mChannelsPerFrame: 2 
            mBitsPerChannel: 0  } 
        cookie: {<CFData 0x2805f3800 [0x1c061f840]>{length = 39, capacity = 39, bytes = 0x03808080220000000480808014401400 ... 1210068080800102}} 
        ACL: {Stereo (L R)}
        FormatList Array: {
            Index: 0 
            ChannelLayoutTag: 0x650002 
            ASBD: {
            mSampleRate: 44100.000000 
            mFormatID: 'aac ' 
            mFormatFlags: 0x0 
            mBytesPerPacket: 0 
            mFramesPerPacket: 1024 
            mBytesPerFrame: 0 
            mChannelsPerFrame: 2 
            mBitsPerChannel: 0  }} 
    } 
    extensions: {{
    VerbatimISOSampleEntry = {length = 87, bytes = 0x00000057 6d703461 00000000 00000001 ... 12100680 80800102 };
}}
}
    sbufToTrackReadiness = 0x0
    numSamples = 43
    outputPTS = {83/600 = 0.138}(based on outputPresentationTimeStamp)
    sampleTimingArray[1] = {
        {PTS = {1024/44100 = 0.023}, DTS = {1024/44100 = 0.023}, duration = {1024/44100 = 0.023}},
    }
    sampleSizeArray[43] = {
        sampleSize = 179,
        sampleSize = 173,
        sampleSize = 178,
        sampleSize = 172,
        sampleSize = 172,
        sampleSize = 159,
        sampleSize = 180,
        sampleSize = 200,
        sampleSize = 187,
        sampleSize = 189,
        sampleSize = 206,
        sampleSize = 192,
        sampleSize = 195,
        sampleSize = 186,
        sampleSize = 183,
        sampleSize = 189,
        sampleSize = 211,
        sampleSize = 198,
        sampleSize = 204,
        sampleSize = 211,
        sampleSize = 204,
        sampleSize = 202,
        sampleSize = 218,
        sampleSize = 210,
        sampleSize = 206,
        sampleSize = 207,
        sampleSize = 221,
        sampleSize = 219,
        sampleSize = 236,
        sampleSize = 219,
        sampleSize = 227,
        sampleSize = 225,
        sampleSize = 225,
        sampleSize = 229,
        sampleSize = 225,
        sampleSize = 236,
        sampleSize = 233,
        sampleSize = 231,
        sampleSize = 249,
        sampleSize = 234,
        sampleSize = 250,
        sampleSize = 249,
        sampleSize = 259,
    }
    dataBuffer = 0x281cde370

输入的append方法将继续返回 true，这原则上意味着所有样本缓冲区都已附加，但是由于某种原因，阅读器会跳过第一部分数据。我在这里做错什么了吗？

我正在使用以下代码来读取文件: 
let asset = AVAsset(url: fileURL)
guard let assetReader = try? AVAssetReader(asset: asset) else {
    return
}

asset.loadValuesAsynchronously(forKeys: ["tracks"]) { in
    guard let audioTrack = asset.tracks(withMediaType: .audio).first else { return }
    let audioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: nil)
    assetReader.startReading()

    while assetReader.status == .reading {
        if let sampleBuffer = audioOutput.copyNextSampleBuffer() {
            // do something
        }
    }
}

最佳答案

首先要讲究一些技巧:您并没有丢失第一个采样缓冲区，而是丢失了第一个采样缓冲区中的第一个数据包。

在iOS 13和macOS 10.15(Catalina)上，读取AAC数据包数据时AVAssetReader和nil outputSettings的行为已更改。

以前，您将获得第一个AAC数据包，该数据包的显示时间戳(零)和修饰附件，该附件指示您丢弃通常的解码音频的前2112帧。

现在[iOS 13，macOS 10.15] AVAssetReader似乎会丢弃第一个数据包，而给您第二个数据包，其显示时间戳为1024，并且只需要丢弃已解码帧的2112 - 1024 = 1088。

在上述情况下，可能不是立即显而易见的事情是AVAssetReader在谈论两个时间轴，而不是一个。数据包时间戳记是未修剪的时间轴之一，而修整指令则暗示着另一个:未修剪的时间轴。

从未修剪的时间戳到修剪的时间戳的转换非常简单，通常是trimmed = untrimmed - 2112。

那么新行为是一个错误吗？如果您解码为LPCM并正确遵循修整说明，那么您仍然应该获得相同的音频，这使我相信更改是有意的(注意:我尚未亲自确认LPCM样本相同)。

但是，文档说:

输出设置的nil值将输出配置为以指定轨道出售的原始格式出售样本。

我认为您不能同时丢弃数据包(甚至是第一个数据包，这基本上是一个常量)，并声称要以其“原始格式”出售样本，因此从这个角度来看，我认为更改具有类似bug的功能质量。

我还认为这是一个不幸的更改，因为我曾经认为nil outputSettings AVAssetReader是一种“原始”模式，但现在它假定您唯一的用例是解码为LPCM。

只有一件事可以将“不幸的”降级为“严重的错误”，那就是如果这种新的“让我们假装第一个AAC数据包不存在”的方法扩展到了使用AVAssetWriter创建的文件，因为这会破坏与非AVAssetReader代码的互操作性，其中要修剪的帧数已收敛为恒定的2112帧。我也没有亲自确认。您是否使用上述示例缓冲区创建了可以共享的文件？

ps。我认为您的输入样本缓冲区在这里不相关，我认为您会丢失从任何AAC文件中读取的第一个数据包。但是，您的输入采样缓冲区似乎有些不寻常，因为它们具有主机时间[capture session？]风格的时间戳，但仍是AAC，每个采样缓冲区只有一个数据包，这不是很多，而且对于23ms的采样来说似乎开销很大音频。您是在AVCaptureSession-> AVAudioConverter链中自己创建它们吗？

关于ios - 使用AVAssetReader读取mp4文件时，第一个音频CMSampleBuffer丢失，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/60347324/

26 4 0
文章推荐： ios - 如何精简MKMapKit中的多个坐标折线条目
文章推荐： ios - 如果快速触发按钮上的长按手势，如何获取按钮文本？
文章推荐： ios - iOS图表更改网格线颜色
文章推荐： ios - 在 SwiftUI 中，当一个 actionsheet 出现时会发生什么？
行者123
个人简介

我是一名优秀的程序员,十分优秀！

滴滴打车优惠券免费领取
滴滴打车优惠券
全站热门文章
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com