ios - 旋转视频而不旋转 AVCaptureConnection 并且在 AVAssetWriter session 中间

Question

我正在使用 PBJVision实现点击录制视频功能。该库尚不支持方向，因此我正在尝试对其进行设计。据我所知，有三种旋转视频的方法-我需要帮助来确定最佳前进方式以及如何实现它.请注意，旋转可能发生在点击记录段之间。因此在录制 session 中，方向被锁定为用户点击按钮时的方向。下次用户点击按钮进行录制时，它应该将方向重新设置为设备的方向(因此生成的视频显示为右侧向上)。

issue page on GitHub as well 中概述了这些方法

方法一使用 setVideoOrientation: 旋转 AVCaptureConnection - 这会导致视频预览在每次切换时闪烁，因为这似乎切换了实际的硬件。不酷， Not Acceptable 。

方法二在用于编写视频的 AVAssetWriterInput 对象上设置 transform 属性。问题是，一旦 Assets 编写者开始编写，transform 属性就无法更改，因此这仅适用于视频的第一段。

方法三使用类似这样的方法旋转正在附加的图像缓冲区:How to directly rotate CVImageBuffer image in IOS 4 without converting to UIImage?但它一直在崩溃，我什至不确定我是否在咆哮正确的树。抛出一个异常，除了我错误地使用了 vImageRotate90_ARGB8888 函数之外，我无法真正追溯到它。

在我上面链接的 GitHub 问题页面上，解释更详细一些。欢迎任何建议 - 老实说，我在 AVFoundation 方面经验不足，所以我希望有一些我什至不知道的神奇方法来做到这一点!

Answer 1

根据 Apple's documentation，方法 1 不是首选方法(“物理旋转缓冲区确实会带来性能成本，因此只有在必要时才请求旋转”)。方法 2 对我有用，但如果我在不支持转换“元数据”的应用程序上播放我的视频，则视频不会正确旋转。方法三是我做的。

在您尝试将图像数据直接从 vImageRotate... 传递到 AVAssetWriterInputPixelBufferAdaptor 之前，我认为它会崩溃。您必须先创建一个 CVPixelBufferRef。这是我的代码:

在 captureOutput:didOutputSampleBuffer:fromConnection: 内部，我在将框架写入适配器之前旋转它:

if ([self.videoWriterInput isReadyForMoreMediaData])
{
    // Rotate buffer first and then write to adaptor
    CMTime sampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
    CVPixelBufferRef rotatedBuffer = [self correctBufferOrientation:sampleBuffer];
    [self.videoWriterInputAdaptor appendPixelBuffer:rotatedBuffer withPresentationTime:sampleTime];
    CVBufferRelease(rotatedBuffer);
}

执行vImage旋转的引用函数是:

/* rotationConstant:
 *  0 -- rotate 0 degrees (simply copy the data from src to dest)
 *  1 -- rotate 90 degrees counterclockwise
 *  2 -- rotate 180 degress
 *  3 -- rotate 270 degrees counterclockwise
 */

- (CVPixelBufferRef)rotateBuffer:(CMSampleBufferRef)sampleBuffer withConstant:(uint8_t)rotationConstant
{
    CVImageBufferRef imageBuffer        = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer, 0);        

    OSType pixelFormatType              = CVPixelBufferGetPixelFormatType(imageBuffer);
    NSAssert(pixelFormatType == kCVPixelFormatType_32ARGB, @"Code works only with 32ARGB format. Test/adapt for other formats!");

    const size_t kAlignment_32ARGB      = 32;
    const size_t kBytesPerPixel_32ARGB  = 4;

    size_t bytesPerRow                  = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width                        = CVPixelBufferGetWidth(imageBuffer);
    size_t height                       = CVPixelBufferGetHeight(imageBuffer);

    BOOL rotatePerpendicular            = (rotateDirection == 1) || (rotateDirection == 3); // Use enumeration values here
    const size_t outWidth               = rotatePerpendicular ? height : width;
    const size_t outHeight              = rotatePerpendicular ? width  : height;

    size_t bytesPerRowOut               = kBytesPerPixel_32ARGB * ceil(outWidth * 1.0 / kAlignment_32ARGB) * kAlignment_32ARGB;

    const size_t dstSize                = bytesPerRowOut * outHeight * sizeof(unsigned char);

    void *srcBuff                       = CVPixelBufferGetBaseAddress(imageBuffer);

    unsigned char *dstBuff              = (unsigned char *)malloc(dstSize);

    vImage_Buffer inbuff                = {srcBuff, height, width, bytesPerRow};
    vImage_Buffer outbuff               = {dstBuff, outHeight, outWidth, bytesPerRowOut};

    uint8_t bgColor[4]                  = {0, 0, 0, 0};

    vImage_Error err                    = vImageRotate90_ARGB8888(&inbuff, &outbuff, rotationConstant, bgColor, 0);
    if (err != kvImageNoError) 
    {
        NSLog(@"%ld", err);
    }

    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);

    CVPixelBufferRef rotatedBuffer      = NULL;
    CVPixelBufferCreateWithBytes(NULL,
                                 outWidth,
                                 outHeight,
                                 pixelFormatType,
                                 outbuff.data,
                                 bytesPerRowOut,
                                 freePixelBufferDataAfterRelease,
                                 NULL,
                                 NULL,
                                 &rotatedBuffer);

    return rotatedBuffer;
}

void freePixelBufferDataAfterRelease(void *releaseRefCon, const void *baseAddress)
{
    // Free the memory we malloced for the vImage rotation
    free((void *)baseAddress);
}

注意:您可能喜欢为 rotationConstant 使用枚举。类似这样的东西(不要用 MOVRotateDirectionUnknown 调用这个函数):

typedef NS_ENUM(uint8_t, MOVRotateDirection)
{
    MOVRotateDirectionNone = 0,
    MOVRotateDirectionCounterclockwise90,
    MOVRotateDirectionCounterclockwise180,
    MOVRotateDirectionCounterclockwise270,
    MOVRotateDirectionUnknown
};

注意:如果您需要IOSurface 支持，您应该使用CVPixelBufferCreate 而不是CVPixelBufferCreateWithBytes 并直接将字节数据传递给它:

NSDictionary *pixelBufferAttributes = @{ (NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{} };
CVPixelBufferCreate(kCFAllocatorDefault,
                    outWidth,
                    outHeight,
                    pixelFormatType,
                    (__bridge CFDictionaryRef)(pixelBufferAttributes),
                    &rotatedBuffer);

CVPixelBufferLockBaseAddress(rotatedBuffer, 0);
uint8_t *dest = CVPixelBufferGetBaseAddress(rotatedBuffer);
memcpy(dest, outbuff.data, bytesPerRowOut * outHeight);

CVPixelBufferUnlockBaseAddress(rotatedBuffer, 0);