Problemi di prestazioni durante l'utilizzo di AVCaptureVideoDataOutput e AVCaptureAudioDataOutput

Question

Si verificano problemi di lag quando sto registrando audio + video utilizzando AVCaptureVideoDataOutput e AVCaptureAudioDataOutput. A volte il video si blocca per alcuni millisecondi, a volte l'audio non è sincronizzato con il video.

Ho inserito alcuni registri e ho osservato che prima ricevo molti buffer video nel callback captureOutput e dopo un po 'di tempo ottengo i buffer audio (a volte non ricevo affatto i buffer audio e il video risultante è senza suono). Se commento il codice che gestisce i buffer video, ottengo i buffer audio senza problemi.

Questo è il codice che sto utilizzando:

-(void)initMovieOutput:(AVCaptureSession *)captureSessionLocal 
{ 
    AVCaptureVideoDataOutput *dataOutput = [[AVCaptureVideoDataOutput alloc] init]; 
    self._videoOutput = dataOutput; 
    [dataOutput release]; 

    self._videoOutput.alwaysDiscardsLateVideoFrames = NO; 
    self._videoOutput.videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange] 
                  forKey:(id)kCVPixelBufferPixelFormatTypeKey 
            ];  
    AVCaptureAudioDataOutput *audioOutput = [[AVCaptureAudioDataOutput alloc] init]; 
    self._audioOutput = audioOutput; 
    [audioOutput release]; 

    [captureSessionLocal addOutput:self._videoOutput]; 
    [captureSessionLocal addOutput:self._audioOutput]; 


    // Setup the queue 
    dispatch_queue_t queue = dispatch_queue_create("MyQueue", NULL); 
    [self._videoOutput setSampleBufferDelegate:self queue:queue]; 
    [self._audioOutput setSampleBufferDelegate:self queue:queue]; 
    dispatch_release(queue); 
} 

Qui ho creato l'autore:

-(BOOL) setupWriter:(NSURL *)videoURL session:(AVCaptureSession *)captureSessionLocal 
{ 
    NSError *error = nil; 
    self._videoWriter = [[AVAssetWriter alloc] initWithURL:videoURL fileType:AVFileTypeQuickTimeMovie 
                 error:&error]; 
    NSParameterAssert(self._videoWriter); 


    // Add video input 
    NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: 
            AVVideoCodecH264, AVVideoCodecKey, 
            [NSNumber numberWithInt:640], AVVideoWidthKey, 
            [NSNumber numberWithInt:480], AVVideoHeightKey, 
            nil]; 

    self._videoWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo 
                     outputSettings:videoSettings]; 


    NSParameterAssert(self._videoWriterInput); 
    self._videoWriterInput.expectsMediaDataInRealTime = YES; 
    self._videoWriterInput.transform = [self returnOrientation]; 

    // Add the audio input 
    AudioChannelLayout acl; 
    bzero(&acl, sizeof(acl)); 
    acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; 


    NSDictionary* audioOutputSettings = nil;   
    // Both type of audio inputs causes output video file to be corrupted. 

     // should work on any device requires more space 
     audioOutputSettings = [ NSDictionary dictionaryWithObjectsAndKeys:      
           [ NSNumber numberWithInt: kAudioFormatAppleLossless ], AVFormatIDKey, 
           [ NSNumber numberWithInt: 16 ], AVEncoderBitDepthHintKey, 
           [ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey, 
           [ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey,          
           [ NSData dataWithBytes: &acl length: sizeof(acl) ], AVChannelLayoutKey, 
           nil ]; 

    self._audioWriterInput = [AVAssetWriterInput 
             assetWriterInputWithMediaType: AVMediaTypeAudio 
             outputSettings: audioOutputSettings ]; 

    self._audioWriterInput.expectsMediaDataInRealTime = YES;  

    // add input 
    [self._videoWriter addInput:_videoWriterInput]; 
    [self._videoWriter addInput:_audioWriterInput]; 

    return YES; 
} 

Ed ecco il callback:

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
     fromConnection:(AVCaptureConnection *)connection 
{ 

    if(!CMSampleBufferDataIsReady(sampleBuffer)) 
    { 
     NSLog(@"sample buffer is not ready. Skipping sample"); 
     return; 
    } 
    if(_videoWriter.status != AVAssetWriterStatusCompleted) 
    { 
     if(_videoWriter.status != AVAssetWriterStatusWriting ) 
     {    
      CMTime lastSampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer); 
      [_videoWriter startWriting]; 
      [_videoWriter startSessionAtSourceTime:lastSampleTime]; 
     } 

     if(captureOutput == _videoOutput) 
     { 
      if([self._videoWriterInput isReadyForMoreMediaData]) 
      { 

      [self newVideoSample:sampleBuffer]; 

      } 
     } 
     else if(captureOutput == _audioOutput) 
     { 
      if([self._audioWriterInput isReadyForMoreMediaData]) 
      { 

       [self newAudioSample:sampleBuffer]; 


      } 
     } 
    } 

} 

-(void) newAudioSample:(CMSampleBufferRef)sampleBuffer 
{ 

     if(_videoWriter.status > AVAssetWriterStatusWriting) 
     { 

      [self NSLogPrint:[NSString stringWithFormat:@"Audio:Warning: writer status is %d", _videoWriter.status]]; 
      if(_videoWriter.status == AVAssetWriterStatusFailed) 
       [self NSLogPrint:[NSString stringWithFormat:@"Audio:Error: %@", _videoWriter.error]]; 
      return; 
     } 

     if(![_audioWriterInput appendSampleBuffer:sampleBuffer]) 
      [self NSLogPrint:[NSString stringWithFormat:@"Unable to write to audio input"]]; 

} 

-(void) newVideoSample:(CMSampleBufferRef)sampleBuffer 
{ 
    if(_videoWriter.status > AVAssetWriterStatusWriting) 
    { 
     [self NSLogPrint:[NSString stringWithFormat:@"Video:Warning: writer status is %d", _videoWriter.status]]; 
     if(_videoWriter.status == AVAssetWriterStatusFailed) 
      [self NSLogPrint:[NSString stringWithFormat:@"Video:Error: %@", _videoWriter.error]]; 
     return; 
    } 


    if(![_videoWriterInput appendSampleBuffer:sampleBuffer]) 
     [self NSLogPrint:[NSString stringWithFormat:@"Unable to write to video input"]]; 
} 

C'è qualcosa sbagliato nel mio codice, perché il video è in ritardo? (Lo sto testando su un Iphone 4 ios 4.2.1)

Answer 1

Sembra che si stiano usando le code seriali. La coda di uscita audio è subito dopo la coda di uscita video. Prendi in considerazione l'uso di code simultanee.