StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POVideo processing on Android 3.x/Google TV/SDK: Video frame to SurfaceTexture/OpenGL/GLSL
text
Body
copied!<p>After spending some time into it, it might be better to ask. Thanks for help, guys!</p> <p>Question</p> <ul> <li>How to render a video frame from MediaPlayer or VideoView to SurfaceTexture or a OpenGL texture, in order to change the texture/fragment color via GLSL? (We need it for fancy GLES/GLSL video processing routines.)</li> </ul> <p>Context</p> <p>a) Google TV (LG G2 2012 device) is an Android 3.2 device with SDK-only support (no NDK)</p> <p>b) It is easy to render from camera to SurfaceTexture, but how to render video to SurfaceTexture in Android 3.x? For camera solution, see below.</p> <p>c) I'm already rendering video frames to a GLView/GLRenderer, but I'm not grabbing a frame in order to change it via GLSL. It seems not to work. But I need accessable GLES/GLSL textures with video data for video processing:</p> <p>MainActivity class:</p> <pre class="lang-java prettyprint-override"><code>public void onCreate(Bundle state) { super.onCreate(state); m_View = new GLSimpleView(this); setContentView(m_View); m_Holder = m_View.getHolder(); m_Holder.addCallback(this); m_Holder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS); } </code></pre> <p>GLSimpleView class:</p> <pre><code>public GLSimpleView(Context context) { super(context); m_Renderer = new GLTextureRenderer(context); this.setRenderer(m_Renderer); } </code></pre> <p>GLTextureRender class:</p> <pre><code>int[] textures = new int[1]; GLES20.glGenTextures(1, textures, 0); m_SurfaceTexture = textures[0]; GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureID); GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST); GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE); GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE); m_Surface = new SurfaceTexture(m_SurfaceTexture); m_Surface.setOnFrameAvailableListener(this); // THIS ONE DOESN'T WORK WITH ANDROID 3.x! // HOW TO BIND m_Surface TO m_MediaPlayer? //Surface surface = new Surface(m_Surface); //m_MediaPlayer.setSurface(surface); //surface.release(); </code></pre> <p>Also compare this:</p> <ul> <li><p>SurfaceTexture for camera (I need this one for MediaPlayer or VideoView!): <a href="https://stackoverflow.com/questions/6414003/using-surfacetexture-in-android">Using SurfaceTexture in Android</a></p></li> <li><p>Video to GLView (no texture access via GLSL!): <a href="https://stackoverflow.com/questions/11431676/playing-video-in-a-glsurfaceview-instead-of-surfaceview">Playing video in a GLSurfaceView instead of SurfaceView</a></p></li> <li><p>Android 4.x SDK VideoSurfaceTexture sample (not 3.2 compatible!): <a href="http://source-android.frandroid.com/cts/tests/src/android/media/cts/VideoSurfaceView.java" rel="nofollow noreferrer">http://source-android.frandroid.com/cts/tests/src/android/media/cts/VideoSurfaceView.java</a></p></li> <li><p>Android MediaPlayer (in Android 3.x no support of setSurface()!): <a href="http://developer.android.com/reference/android/media/MediaPlayer.html" rel="nofollow noreferrer">http://developer.android.com/reference/android/media/MediaPlayer.html</a></p></li> </ul> <p>Therefore the main question is still: How to access and manipulate a video frame with Android 3.x? Perhaps a different solution? Am I missing something, after spending too much time? Consider that there is no NDK support on Google TV at all, that we're very limited if we try to manipulate video data.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload