Databases
StackOverflow2013
Postsdbo
POiOS: Audio and Video tap from Airplay
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POiOS: Audio and Video tap from Airplay
    primarykey
    data
    text
    <p>I have made a video player that is analyzing the realtime audio and video tracks from the video that is currently playing. The videos are stored on the iOS device (in the Apps Documents directory).</p> <p>This all works fine. I use MTAudioProcessingTap in order to get all the audio samples and do some FFT, and I am analyzing the video by just copy'ing the pixel buffers from the currently played CMTime (the AVPlayer currentTime property). As I said, this works fine.</p> <p>But now I want to support Airplay. Just the airplay itself is not difficult, but my taps stop working as soon as Airplay is toggled and the video is playing on the ATV. Somehow, the MTAudioProcessingTap won't process and the pixelbuffers are all empty... I can't get to the data.</p> <p>Is there any way to get to this data ?</p> <p>In order to get the pixel buffers, I just fire an event every few milli-sec and retrieving the player's currentTime. Then:</p> <pre><code>CVPixelBufferRef imageBuffer = [videoOutput copyPixelBufferForItemTime:time itemTimeForDisplay:nil]; CVPixelBufferLockBaseAddress(imageBuffer,0); uint8_t *tempAddress = (uint8_t *) CVPixelBufferGetBaseAddress(imageBuffer); size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); size_t height = CVPixelBufferGetHeight(imageBuffer); CVPixelBufferUnlockBaseAddress(imageBuffer,0); </code></pre> <p>Where <code>tempAddress</code> is my pixelbuffer, and <code>videoOutput</code> is an instance of <code>AVPlayerItemVideoOutput</code>.</p> <p>For audio, I use:</p> <pre><code>AVMutableAudioMixInputParameters *inputParams = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:audioTrack]; // Create a processing tap for the input parameters MTAudioProcessingTapCallbacks callbacks; callbacks.version = kMTAudioProcessingTapCallbacksVersion_0; callbacks.clientInfo = (__bridge void *)(self); callbacks.init = init; callbacks.prepare = prepare; callbacks.process = process; callbacks.unprepare = unprepare; callbacks.finalize = finalize; MTAudioProcessingTapRef tap; OSStatus err = MTAudioProcessingTapCreate(kCFAllocatorDefault, &amp;callbacks, kMTAudioProcessingTapCreationFlag_PostEffects, &amp;tap); if (err || !tap) { NSLog(@"Unable to create the Audio Processing Tap"); return; } inputParams.audioTapProcessor = tap; // Create a new AVAudioMix and assign it to our AVPlayerItem AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix]; audioMix.inputParameters = @[inputParams]; playerItem.audioMix = audioMix; </code></pre> <p>Regards, Niek</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USNiek van der Steen
    1. USNiek van der Steen
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POiOS: Audio and Video tap from Airplay
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POiOS: Audio and Video tap from Airplay
      1. USNiek van der Steen
      1. VTBountyStart
    3. VO
      singulars
      1. POiOS: Audio and Video tap from Airplay
      1. USuser2590894
      1. VTFavorite
    1. COCould you please add the code where you're accessing the pixel buffers?
      singulars
      1. POiOS: Audio and Video tap from Airplay
      1. USDominik Hadl
    2. COAnd code for playing the video could help too :)
      singulars
      1. POiOS: Audio and Video tap from Airplay
      1. USDominik Hadl
    3. CODone. any thoughts ?
      singulars
      1. POiOS: Audio and Video tap from Airplay
      1. USNiek van der Steen
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload