Databases
StackOverflow2013
Postsdbo
POImprove OpenGL rendering speed
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POImprove OpenGL rendering speed
    primarykey
    data
    text
    <p>I've built a small video player that grabs frames (as a string/byte array) from a movie by using GStreamer and then renders each frame to an OpenGL texture. This goes fine up to 30 fps 1080p movies, but when I try a 60 fps movie, it can't keep up and the video lags behind on the audio stream! When I play the video with "gst-launch playbin2" it works perfectly, so the video gets decoded with sufficent speed.</p> <p>I've done a bit of measuring and it appears if the problem lies with <em>either</em> the updating of the texture with a new frame <em>or</em> the actual drawing of the frame to the screen. I'm using the old fashioned GL_BEGIN/QUADS/END method (because I don't know any better) to do the drawing part, but could this be the bottleneck? I thought the alternative and newer methods (GL_TRIANGLE_STRIP, VBOs/FBOs or glDrawArrays/Elements) are only beneficial when working with a large amount of textures/polygons and not so much with what I am trying to do, or am I wrong with this?</p> <p>Does anyone have any tips on how to improve the rendering speed in this specific situation?</p> <p><hr/> <b>Update:</b> Thanks to some good advice given here (Use glTexSubImage2D instead of glTexImage2D; Use Display lists), my code now looks as below. There has been some performance improvement, but still the movie runs a little too slow to reach 60 fps (just by a margin; only a little more optimization is required). </p> <p>The output of the time measurement for the rendering of a couple of frames now is as follows:</p> <pre><code>Texture updated in 16.2160497479 ms Frame drawn in 0.540967225085 ms Texture updated in 14.7260598703 ms Frame drawn in 0.606612686107 ms Texture updated in 17.0613363633 ms Frame drawn in 0.743171453788 ms Texture updated in 12.6152746452 ms Frame drawn in 2.45603172378 ms Texture updated in 13.3847853272 ms Frame drawn in 3.0869575436 ms Texture updated in 17.7117126901 ms Frame drawn in 0.572979517806 ms Texture updated in 13.8203956395 ms Frame drawn in 1.15892604026 ms Texture updated in 16.0600404733 ms Frame drawn in 0.563659483216 ms Texture updated in 13.0213039782 ms Frame drawn in 3.70653723435 ms </code></pre> <p>Even though the largest amount of time appears to be taken by the updating of the texture, even with glTexSubImage2D (which seems plausible as this involves transferring data from system memory to gpu) I guess I will try to improve performance by using VBO/FBOs/vertex arrays instead of drawing in immediate mode with glBegin/End</p> <pre><code>... def __textureSetup(self): # Setup texture in OpenGL to render video to glEnable(GL_TEXTURE_2D) glMatrixMode(GL_MODELVIEW) self.textureNo = glGenTextures(1) glBindTexture(GL_TEXTURE_2D, self.textureNo) glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR) glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR) # Fill texture with black to begin with. img = np.zeros([self.vidsize[0],self.vidsize[1],3],dtype=np.uint8) img.fill(0) glTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, self.vidsize[0], self.vidsize[1], 0, GL_RGB, GL_UNSIGNED_BYTE, img) # Create display list which draws to the quad to which the texture is rendered (x,y) = self.vidPos (w,h) = self.destsize self.frameQuad = glGenLists(1); glNewList(self.frameQuad, GL_COMPILE) glBegin(GL_QUADS) glTexCoord2f(0.0, 0.0); glVertex3i(x, y, 0) glTexCoord2f(1.0, 0.0); glVertex3i(x+w, y, 0) glTexCoord2f(1.0, 1.0); glVertex3i(x+w, y+h, 0) glTexCoord2f(0.0, 1.0); glVertex3i(x, y+h, 0) glEnd() glEndList() # Clear The Screen And The Depth Buffer glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT) def __texUpdate(self, appsink): """ Callback for GStreamer """ # Retrieve buffer from videosink self.buffer = appsink.emit('pull-buffer') self.texUpdated = True def drawFrame(self): glCallList(self.frameQuad) # Flip the buffer to show frame to screen pygame.display.flip() def play(self): # Start gst loop (which listens for events from the player) thread.start_new_thread(self.gst_loop.run, ()) # Signal player to start video playback self.player.set_state(gst.STATE_PLAYING) self.paused = False # While video is playing, render frames while self.gst_loop.is_running(): # Only draw a frame when a new texture has been received if self.texUpdated: t1 = time.clock() # Update texture glTexSubImage2D( GL_TEXTURE_2D, 0, 0, 0, self.vidsize[0], self.vidsize[1], GL_RGB, GL_UNSIGNED_BYTE, self.buffer.data) t2 = time.clock() print "Texture updated in {0} ms".format((t2-t1)*1000) self.drawFrame() print "Frame drawn in {0} ms".format((time.clock()-t2)*1000) for e in pygame.event.get(): if e.type == pygame.QUIT: self.stop() if e.type == pygame.KEYDOWN: if e.key == pygame.K_ESCAPE: self.stop() if e.key == pygame.K_SPACE: self.pause() pygame.event.pump() # Prevent freezing of screen while dragging window def stop(self): self.gst_loop.quit() self.player.set_state(gst.STATE_NULL) ... </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USDaniel Schreij
    1. USDaniel Schreij
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POImprove OpenGL rendering speed
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POImprove OpenGL rendering speed
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhat is the target gpu? I can assure you that a modern GPU can handle a single 1080p60 stream without much effort. You should look for asynchronous texture update paths using PBOs.
      singulars
      1. POImprove OpenGL rendering speed
      1. USderhass
    2. COAre you waiting for vsync? OpenGL normally is. How many FPS does your display support?
      singulars
      1. POImprove OpenGL rendering speed
      1. USLumpN
    3. COI've tried vsync on and off, and it didn't really matter. My display (at the moment) is a laptop with a 60 Hz screen. The target GPU is NVidia GTX 460M, which should be more than enough I reckon.
      singulars
      1. POImprove OpenGL rendering speed
      1. USDaniel Schreij
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload