Databases
StackOverflow2013
Postsdbo
POIOSurfaces - Artefacts in video and unable to grab video surfaces
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POIOSurfaces - Artefacts in video and unable to grab video surfaces
    primarykey
    data
    text
    <p>This is a 2-part Question. I have the following code working which grabs the current display surface and creates a video out of the surfaces (everything happens in the background).</p> <pre><code>for(int i=0;i&lt;100;i++){ IOMobileFramebufferConnection connect; kern_return_t result; IOSurfaceRef screenSurface = NULL; io_service_t framebufferService = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("AppleH1CLCD")); if(!framebufferService) framebufferService = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("AppleM2CLCD")); if(!framebufferService) framebufferService = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("AppleCLCD")); result = IOMobileFramebufferOpen(framebufferService, mach_task_self(), 0, &amp;connect); result = IOMobileFramebufferGetLayerDefaultSurface(connect, 0, &amp;screenSurface); uint32_t aseed; IOSurfaceLock(screenSurface, kIOSurfaceLockReadOnly, &amp;aseed); uint32_t width = IOSurfaceGetWidth(screenSurface); uint32_t height = IOSurfaceGetHeight(screenSurface); m_width = width; m_height = height; CFMutableDictionaryRef dict; int pitch = width*4, size = width*height*4; int bPE=4; char pixelFormat[4] = {'A','R','G','B'}; dict = CFDictionaryCreateMutable(kCFAllocatorDefault, 0, &amp;kCFTypeDictionaryKeyCallBacks, &amp;kCFTypeDictionaryValueCallBacks); CFDictionarySetValue(dict, kIOSurfaceIsGlobal, kCFBooleanTrue); CFDictionarySetValue(dict, kIOSurfaceBytesPerRow, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &amp;pitch)); CFDictionarySetValue(dict, kIOSurfaceBytesPerElement, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &amp;bPE)); CFDictionarySetValue(dict, kIOSurfaceWidth, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &amp;width)); CFDictionarySetValue(dict, kIOSurfaceHeight, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &amp;height)); CFDictionarySetValue(dict, kIOSurfacePixelFormat, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, pixelFormat)); CFDictionarySetValue(dict, kIOSurfaceAllocSize, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &amp;size)); IOSurfaceRef destSurf = IOSurfaceCreate(dict); IOSurfaceAcceleratorRef outAcc; IOSurfaceAcceleratorCreate(NULL, 0, &amp;outAcc); IOSurfaceAcceleratorTransferSurface(outAcc, screenSurface, destSurf, dict, NULL); IOSurfaceUnlock(screenSurface, kIOSurfaceLockReadOnly, &amp;aseed); CFRelease(outAcc); // MOST RELEVANT PART OF CODE CVPixelBufferCreateWithBytes(NULL, width, height, kCVPixelFormatType_32BGRA, IOSurfaceGetBaseAddress(destSurf), IOSurfaceGetBytesPerRow(destSurf), NULL, NULL, NULL, &amp;sampleBuffer); CMTime frameTime = CMTimeMake(frameCount, (int32_t)5); [adaptor appendPixelBuffer:sampleBuffer withPresentationTime:frameTime]; CFRelease(sampleBuffer); CFRelease(destSurf); frameCount++; } </code></pre> <p>P.S: The last 4-5 lines of code are the most relevant(if you need to filter).</p> <p>1) The video that is produced has artefacts. I have worked on videos previously and have encountered such an issue before as well. I suppose there can be 2 reasons for this:<br/> i. The PixelBuffer that is passed to the adaptor is getting modified or released before the processing (encoding + writing) is complete. This can be due to asynchronous calls. But I am not sure if this itself is the problem and how to resolve it.<br/> ii. The timestamps that are passed are inaccurate (e.g. 2 frames having the same timestamp or a frame having a lower timestamp than the previous frame). I logged out the timestamp values and this doesn't seem to be the problem.</p> <p>2) The code above is not able to grab surfaces when a video is played or when we play games. All I get is a blank screen in the output. This might be due to hardware accelerated decoding that happens in such cases. </p> <p>Any inputs on either of the 2 parts of the questions will be really helpful. Also, if you have any good links to read on IOSurfaces in general, please do post them here.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USHrishikesh_Pardeshi
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POIOSurfaces - Artefacts in video and unable to grab video surfaces
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POIOSurfaces - Artefacts in video and unable to grab video surfaces
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload