objective-c - 在 swift3.2/4 中正确解除分配 MTAudioProcessingTapRef

Question

我一直在我的项目中使用 MTAudioProcessingTapRef，以便在播放流式音频时实时分析缓冲区数据。问题是当我需要它时，我无法让 tap 处理器正确解除分配。

我有一个 AudioViewController swift 类，它引用了我的 AudioTapProcessor objective-C 类，swift 类负责告诉处理器开始和停止对 AVPlayerItem 的处理。处理器还有一个委托(delegate)(在本例中为 View Controller )来通知处理时缓冲区的变化。

我的问题是，如果我将处理器委托(delegate)声明为弱(应该如此)，处理器将随机崩溃以尝试通知已解除分配的委托(delegate)，因为 tap 处理器的处理方法在停止处理后执行了几次称呼。我发现解决此问题的唯一方法是将 tap 处理器委托(delegate)声明为强属性，这显然会导致保留周期，并且我的 AudioViewControllers 将永远不会被释放。

下面的一些代码可以帮助您了解相关情况:

AudioTapProcessor.h

@interface AudioTapProcessor : NSObject

@property (nonatomic, strong) AVPlayerItem *item;
@property (nonatomic, strong) id<AudioProcessorDelegate> delegate;

- (instancetype)initWithDelegate:(id<AudioProcessorDelegate>)delegate 
    item:(AVPlayerItem *)item;
- (void)startProcessing;
- (void)stopProcessing;

@end

AudioTapProcessor.m

void init(MTAudioProcessingTapRef tap, void *clientInfo, void 
**tapStorageOut) {
    *tapStorageOut = clientInfo;
}

void finalize(MTAudioProcessingTapRef tap) {}

void prepare(
         MTAudioProcessingTapRef tap,
         CMItemCount maxFrames,
         const AudioStreamBasicDescription *processingFormat
         ) {}

void unprepare(MTAudioProcessingTapRef tap) {}

void process(
         MTAudioProcessingTapRef tap,
         CMItemCount numberFrames,
         MTAudioProcessingTapFlags flags,
         AudioBufferList *bufferListInOut,
         CMItemCount *numberFramesOut,
         MTAudioProcessingTapFlags *flagsOut
         ) {
 //Random crashes here if I declare the delegate weak
 //Something like AUDeferredRenderer-0x7ff8f448ef (364): EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  AudioTapProcessor *processor = (__bridge AudioTapProcessor *)MTAudioProcessingTapGetStorage(tap);

  OSStatus err = MTAudioProcessingTapGetSourceAudio(tap, numberFrames, bufferListInOut, flagsOut, NULL, numberFramesOut);

  AudioBuffer *pBuffer = &bufferListInOut->mBuffers[0];
  UInt32 frameLength = pBuffer->mDataByteSize / sizeof(float);
  float *pData = (float *)pBuffer->mData;

  if (err == noErr && processor) {
    if ([processor.delegate 
      respondsToSelector:@selector(updateWith:withSize:)]) {
      [processor.delegate updateWith:pData withSize:frameLength];
    }
  }
 }

- (void)stopProcessing
{
  [self.item removeObserver:self forKeyPath:@"status"];
AVMutableAudioMixInputParameters *params =
(AVMutableAudioMixInputParameters *) _item.audioMix.inputParameters[0];
  MTAudioProcessingTapRef tap = params.audioTapProcessor;
  self.item.audioMix = nil;
  CFRelease(tap);
  //By doing this the tap processor does call its unprepare and finalize methods, so it is being deallocated fine.
}

然后在我的 AudioViewController.swift 中我有:

var processor: AudioTapProcessor!

override func prepareForPlayback() {
  super.prepareForPlayback()
  if processor == nil {
    processor = AudioTapProcessor(delegate: self, item: item)
    processor.startProcessing()
  }
}

override func viewWillDisappear(_ animated: Bool) {
   super.viewWillDisappear(animated)
   player.pause()
}

deinit {
  //I tried to do this early in the lifecycle(viewWillDissapear) and it is the same thing.
   processor.stopProcessing()
}

任何提示将不胜感激，我对此感到疯狂。谢谢

Answer 1

适用于所有 iOS 版本

⇨ 了解根本原因

1.AudioTapProcessor.m初始化

callbacks.clientInfo 持有指向 self 的指针。这不是弱引用或强引用，只是一个 C 指针。所以如果 self 被释放，我们 context->self 指向一个错误的内存地址

- (AVAudioMix *)audioMix {
    if (!_audioMix) {
        AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix];
        if (audioMix) {
            AVMutableAudioMixInputParameters *audioMixInputParameters = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:self.audioAssetTrack];
            if (audioMixInputParameters) {
                MTAudioProcessingTapCallbacks callbacks;

                ...
                callbacks.clientInfo = (__bridge void *)self;
                ...
            }
        }
     }
}

2.AudioTapProcessor.m processCallback

每次调用 processCallback 时都会进行安全检查以查看 self 是否正在被释放，但请记住在步骤 1 中。上面即使 self 被释放 context->self 也不是 nil 而是指向一个错误的内存地址导致EXC_BAD_ACCESS。

static void tap_ProcessCallback(MTAudioProcessingTapRef tap, CMItemCount numberFrames, MTAudioProcessingTapFlags flags, AudioBufferList *bufferListInOut, CMItemCount *numberFramesOut, MTAudioProcessingTapFlags *flagsOut) {
    ...
    MYAudioTapProcessor *self = ((__bridge MYAudioTapProcessor *)context->self);
    if (!self) {
      NSLog(@"AudioTapProcessor - processCallback CANCELLED (self is nil)");
      return;
    }
    NSLog(@"AudioTapProcessor - processCallback PROCESSING");
}

⇨ 现在，如何解决这个问题？

1.ViewController.swift 或 audioTapProcessor 的拥有者

deinit {
  print("ViewController - Dealloc")
  audioTapProcessor.stopProcessing()
}

2.AudioTapProcessor.m

我们需要一种方法来告诉我们的 audioTapProcessor 停止 processCallback。最简单自然的方法是使用上面 processCallback 中已有的检查 if (!self) return;

所以停止 audioTapProcessor 只是将 context->self 正确设置为 NULL。

- (void)stopProcessing {
    NSLog(@"AudioTapProcessor - stopProcessing");
    AVMutableAudioMixInputParameters *params = (AVMutableAudioMixInputParameters *)_audioMix.inputParameters[0];
    MTAudioProcessingTapRef audioProcessingTap = params.audioTapProcessor;
    AVAudioTapProcessorContext *context = (AVAudioTapProcessorContext *)MTAudioProcessingTapGetStorage(audioProcessingTap);

    // nils out the pointer so that we know in tapProcessorCallbacks that self will be dealloc'ed
    context->self = NULL;
}

⇨ 因此生命周期得到纠正

代替这个

1. ViewController - Dealloc
2. AudioTapProcessor - 停止处理
3. AudioTapProcessor - processCallback 处理
4. EXC_BAD_ADDRESS

我们明白了

1. ViewController - Dealloc
2. AudioTapProcessor - 停止处理
3. AudioTapProcessor - processCallback 已取消(自身为零)
4. AudioTapProcessor - unprepareCallback
5. AudioTapProcessor - finalizeCallback
6. AudioTapProcessor - Dealloc