iphone - AVCapture追加SampleBuffer

Question

我对这个简直要疯了 - 到处都看了，也尝试了我能想到的一切。

我正在制作一个使用 AVFoundation - 特别是 AVCapture 来使用 iPhone 相机捕获视频的 iPhone 应用程序。

我需要一个覆盖在录制内容中的视频源上的自定义图像。

到目前为止，我已经设置了 AVCapture session ，可以显示提要、访问框架、将其保存为 UIImage 并将覆盖图像添加到其上。然后将这个新的 UIImage 转换为 CVPixelBufferRef。为了仔细检查 bufferRef 是否正常工作，我将其转换回 UIImage，并且它仍然可以很好地显示图像。

当我尝试将 CVPixelBufferRef 转换为 CMSampleBufferRef 以附加到 AVCaptureSessions assetWriterInput 时，问题就开始了。当我尝试创建 CMSampleBufferRef 时，它总是返回 NULL。

这是 -(void)captureOutput 函数

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
    fromConnection:(AVCaptureConnection *)connection
{ 

 UIImage *botImage = [self imageFromSampleBuffer:sampleBuffer];
 UIImage *wheel = [self imageFromView:wheelView];

 UIImage *finalImage = [self overlaidImage:botImage :wheel];
 //[previewImage setImage:finalImage]; <- works -- the image is being merged into one UIImage

 CVPixelBufferRef pixelBuffer = NULL;
 CGImageRef cgImage = CGImageCreateCopy(finalImage.CGImage);
 CFDataRef image = CGDataProviderCopyData(CGImageGetDataProvider(cgImage));
 int status = CVPixelBufferCreateWithBytes(NULL,
             self.view.bounds.size.width,
             self.view.bounds.size.height,
             kCVPixelFormatType_32BGRA, 
             (void*)CFDataGetBytePtr(image), 
             CGImageGetBytesPerRow(cgImage), 
             NULL, 
             0,
             NULL, 
             &pixelBuffer);
 if(status == 0){
  OSStatus result = 0;

  CMVideoFormatDescriptionRef videoInfo = NULL;
  result = CMVideoFormatDescriptionCreateForImageBuffer(NULL, pixelBuffer, &videoInfo);
  NSParameterAssert(result == 0 && videoInfo != NULL);

  CMSampleBufferRef myBuffer = NULL;
  result = CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault,
            pixelBuffer, true, NULL, NULL, videoInfo, NULL, &myBuffer);
  NSParameterAssert(result == 0 && myBuffer != NULL);//always null :S

  NSLog(@"Trying to append");

  if (!CMSampleBufferDataIsReady(myBuffer)){
   NSLog(@"sampleBuffer data is not ready");
   return;
  }

  if (![assetWriterInput isReadyForMoreMediaData]){
   NSLog(@"Not ready for data :(");
   return;
  }

  if (![assetWriterInput appendSampleBuffer:myBuffer]){   
   NSLog(@"Failed to append pixel buffer");
  }



 }

}

我不断听到的另一个解决方案是使用 AVAssetWriterInputPixelBufferAdaptor，它消除了进行困惑的 CMSampleBufferRef 包装的需要。但是，我搜索了 stacked 和苹果开发者论坛和文档，但找不到有关如何设置或如何使用它的清晰描述或示例。如果有人有一个可行的示例，请您向我展示或帮助我解决上述问题 - 我已经为此不间断地工作了一个星期，但束手无策。

如果您需要任何其他信息，请告诉我

提前致谢，

迈克尔

Answer 1

您需要 AVAssetWriterInputPixelBufferAdaptor，这是创建它的代码:

    // Create dictionary for pixel buffer adaptor
NSDictionary *bufferAttributes = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA], kCVPixelBufferPixelFormatTypeKey, nil];

// Create pixel buffer adaptor
m_pixelsBufferAdaptor = [[AVAssetWriterInputPixelBufferAdaptor alloc] initWithAssetWriterInput:assetWriterInput sourcePixelBufferAttributes:bufferAttributes];

以及使用它的代码:

// If ready to have more media data
if (m_pixelsBufferAdaptor.assetWriterInput.readyForMoreMediaData) {
    // Create a pixel buffer
    CVPixelBufferRef pixelsBuffer = NULL;
    CVPixelBufferPoolCreatePixelBuffer(NULL, m_pixelsBufferAdaptor.pixelBufferPool, &pixelsBuffer);

    // Lock pixel buffer address
    CVPixelBufferLockBaseAddress(pixelsBuffer, 0);

    // Create your function to set your pixels data in the buffer (in your case, fill with your finalImage data)
    [self yourFunctionToPutDataInPixelBuffer:CVPixelBufferGetBaseAddress(pixelsBuffer)];

    // Unlock pixel buffer address
    CVPixelBufferUnlockBaseAddress(pixelsBuffer, 0);

    // Append pixel buffer (calculate currentFrameTime with your needing, the most simplest way is to have a frame time starting at 0 and increment each time you write a frame with the time of a frame (inverse of your framerate))
    [m_pixelsBufferAdaptor appendPixelBuffer:pixelsBuffer withPresentationTime:currentFrameTime];

    // Release pixel buffer
    CVPixelBufferRelease(pixelsBuffer);
}

并且不要忘记释放您的 PixelBufferAdaptor。