ios - H.264 解码器无法正常工作？

Question

我已经检查了这个用于基本 h.264 比特流的解码器的代码一百次，沿途进行了一些调整，但没有成功。当我将输出 CMSampleBuffers 发送到 AVSampleBufferDisplayLayer 时，它们没有出现，大概是因为我解码它们的方式有问题。

我在任何地方都没有收到错误消息； AVSampleBufferDisplayLayer 没有错误，“status”为“1”(又名 .rendering)，CMSampleBufferIsValid() 在输出的 CMSampleBuffers 上返回“true”，我的解码器也没有遇到任何错误。

我很困惑，我希望更有经验的开发人员能够捕获我所遗漏的东西。

我在这里输入原始字节 (typealias FrameData = [UInt8])

func interpretRawFrameData(_ frameData: inout FrameData) -> CMSampleBuffer? {
    let size = UInt32(frameData.count)

    var naluType = frameData[4] & 0x1F
    var frame: CMSampleBuffer?

    // The start indices for nested packets. Default to 0.
    var ppsStartIndex = 0
    var frameStartIndex = 0

    switch naluType {
    // SPS
    case 7:
        print("===== NALU type SPS")
        for i in 4..<40 {
            if frameData[i] == 0 && frameData[i+1] == 0 && frameData[i+2] == 0 && frameData[i+3] == 1 {
                ppsStartIndex = i
                spsSize = i - 4 // Does not include the size of the header
                sps = Array(frameData[4..<i])

                // Set naluType to the nested PPS packet's NALU type
                naluType = frameData[i + 4] & 0x1F
                break
            }
        }
        // If nested frame was found, fallthrough
        if ppsStartIndex != 0 { fallthrough }
    // PPS
    case 8:
        print("===== NALU type PPS")
        for i in ppsStartIndex+4..<ppsStartIndex+34 {
            if frameData[i] == 0 && frameData[i+1] == 0 && frameData[i+2] == 0 && frameData[i+3] == 1 {
                frameStartIndex = i
                ppsSize = i - spsSize - 8 // Does not include the size of the header. Subtract 8 to account for both the SPS and PPS headers
                pps = Array(frameData[ppsStartIndex+4..<i])

                // Set naluType to the nested packet's NALU type
                naluType = frameData[i+4] & 0x1F
                break
            }
        }
        // If nested frame was found, fallthrough
        if frameStartIndex != 0 { fallthrough }
    // IDR frame
    case 5:
        print("===== NALU type IDR frame");

        // Replace start code with size
        let adjustedSize = size - UInt32(ppsSize) - UInt32(spsSize) - 8
        var blockSize = CFSwapInt32HostToBig(adjustedSize)
        memcpy(&frameData[frameStartIndex], &blockSize, 4)

        if createFormatDescription() {
            frame = decodeFrameData(Array(frameData[frameStartIndex...]))
        }
    // B/P frame
    default:
        print("===== NALU type B/P frame");

        // Replace start code with size
        var blockSize = CFSwapInt32HostToBig(size)
        memcpy(&frameData[frameStartIndex], &blockSize, 4)

        frame = decodeFrameData(Array(frameData[frameStartIndex...]))
        break;
    }

    return frame != nil ? frame : nil
}

这就是实际解码发生的地方:

private func decodeFrameData(_ frameData: FrameData) -> CMSampleBuffer? {
    let bufferPointer = UnsafeMutablePointer<UInt8>(mutating: frameData)
    let size = frameData.count

    var blockBuffer: CMBlockBuffer?
    var status = CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault,
                                                    bufferPointer,
                                                    size,
                                                    kCFAllocatorNull,
                                                    nil, 0, frameData.count,
                                                    0, &blockBuffer)

    if status != kCMBlockBufferNoErr { return nil }

    var sampleBuffer: CMSampleBuffer?
    let sampleSizeArray = [size]

    status = CMSampleBufferCreateReady(kCFAllocatorDefault,
                                       blockBuffer,
                                       formatDesc,
                                       1, 0, &sampleTimingInfo,
                                       1, sampleSizeArray,
                                       &sampleBuffer)

    if let buffer = sampleBuffer, status == kCMBlockBufferNoErr {
        let attachments: CFArray? = CMSampleBufferGetSampleAttachmentsArray(buffer, true)

        if let attachmentArray = attachments {
            let dic = unsafeBitCast(CFArrayGetValueAtIndex(attachmentArray, 0), to: CFMutableDictionary.self)

            let key = Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque()
            let val = Unmanaged.passUnretained(kCFBooleanTrue).toOpaque()
            CFDictionarySetValue(dic,
                                 key,
                                 val)
        }

        print("===== Successfully created sample buffer")
        return buffer
    }

    return nil
}

其他注意事项:

• 格式描述包含正确的信息(mediaType = "vide", mediaSubType = "avc1", dimensions = "640x480")
• 我正在解码的比特流总是将 SPS、PPS 和 IDR 帧组合在一起，每 20 帧左右将它们作为一个大数据包发送。每隔一段时间，发送一个单独的 B/P 帧。

谢谢!

Answer 1

那段代码非常难看，所以我决定稍微修改一下。结果证明成功了。那里一定有问题。

无论如何，这是一个工作版本。它将解码和解压缩的帧发送给它的委托(delegate)。理想情况下，调用 interpretRawFrameData 的任何人都将返回一个可显示的框架，我会努力解决这个问题，但与此同时这也有效。

https://github.com/philipshen/H264Decoder