ios - 如何开始使用 Video-To-Texture 渲染？

Question

我有一个视频被下载并检索为 NSData*(稍后将成为 NSInputStream*)。

文件格式暂时无关紧要。我想通过 MTLTexture(或视频的等效对象)将此视频渲染到 3D 对象上。

我不知道从哪里开始，因为在 AVFoundation Framework 和 CoreGraphics Framework 中似乎有无穷无尽的类。能帮我看看我需要用到的相关类吗？

Answer 1

感谢@warrenm，我找到了正确的解决方案。
这是我的 VideoPlayer 类

的一部分

-(instancetype)initWithURL:(NSURL*)url {
        self = [super init];
        if (self) {
            CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, [[Renderer sharedInstance] renderDevice], nil, &self->_textureCache);
            NSError* error1;
            NSString* mimeType = @"video/mp4";
            NSArray* keys = @[@"tracks"];
            AVAsset* videoAsset = [[AVURLAsset alloc] initWithURL:url options:@{@"AVURLAssetOutOfBandMIMETypeKey" : mimeType}];
            [videoAsset loadValuesAsynchronouslyForKeys:keys completionHandler:^{
               [self onVideoLoaded:videoAsset];
            }
        ];
    }
    return self;
}


-(void)onVideoLoaded:(AVAsset*)videoAsset {
    NSError* error;
    AVKeyValueStatus status = [videoAsset statusOfValueForKey:@"tracks" error:&error];
    switch (status) {
        //error handlings
    }
    NSDictionary* settings = @{(id) kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA)};
    [self setVideoPlayerOutput:[[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:settings]];
    [self setVideoPlayerItem:[AVPlayerItem playerItemWithAsset:videoAsset]];
    [[self videoPlayerItem] addOutput:[self videoPlayerOutput]];
    [self setVideoPlayer:[[AVPlayer alloc] initWithPlayerItem:[self videoPlayerItem]]];
    [[self videoPlayer] setVolume:1.0f];
    [self setIsLoaded:YES];
}

要获取纹理，我有这个函数:

-(id <MTLTexture>)getCurrentFrameTexture {
    CVPixelBufferRef buffer = [[self videoPlayerOutput] copyPixelBufferForItemTime:[[self videoPlayerItem] currentTime] itemTimeForDisplay:nil];
    NSUInteger width = CVPixelBufferGetWidth(buffer);
    NSUInteger height = CVPixelBufferGetHeight(buffer);
    NSUInteger planeCount = CVPixelBufferGetPlaneCount(buffer);
    CVMetalTextureRef textureRef;
    CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, [self textureCache], buffer, nil, MTLPixelFormatBGRA8Unorm_sRGB, width, height, planeCount, &textureRef);
    id <MTLTexture> tex = CVMetalTextureGetTexture(textureRef);
    CVBufferRelease(textureRef);
    CVBufferRelease(buffer);
    return tex;
}

这也适用于流式传输，只需提供有效的 mp4 网址即可。文件未下载或任何内容