Databases
StackOverflow2013
Postsdbo
POMediaCodec and Camera: colorspaces don't match
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMediaCodec and Camera: colorspaces don't match
    primarykey
    data
    text
    <p>I have been trying to get H264 encoding to work with input captured by the camera on an Android tablet using the new low-level <a href="http://developer.android.com/reference/android/media/MediaCodec.html" rel="noreferrer">MediaCodec</a>. I have gone through some difficulties with this, since the MediaCodecAPI is poorly documented, but I've gotten something to work at last.</p> <p>I'm setting up the camera as follows:</p> <pre><code> Camera.Parameters parameters = mCamera.getParameters(); parameters.setPreviewFormat(ImageFormat.YV12); // &lt;1&gt; parameters.setPreviewFpsRange(4000,60000); parameters.setPreviewSize(640, 480); mCamera.setParameters(parameters); </code></pre> <p>For the encoding part, I'm instantiating the MediaCodec object as follows:</p> <pre><code> mediaCodec = MediaCodec.createEncoderByType("video/avc"); MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 640, 480); mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 500000); mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 15); mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar); // &lt;2&gt; mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5); mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE); mediaCodec.start(); </code></pre> <p>The final goal is to create an RTP-stream (and correspond with Skype), but so far I am only streaming the raw H264 directly to my desktop. There I use the following GStreamer-pipeline to show the result:</p> <pre><code>gst-launch udpsrc port=5555 ! video/x-h264,width=640,height=480,framerate=15/1 ! ffdec_h264 ! autovideosink </code></pre> <p>All works well, except for the colors. I need to set 2 colorformats in the computer: one for the camera-preview (line tagged with <code>&lt;1&gt;</code>) and one for the MediaCodec-object (tagged with <code>&lt;2&gt;</code>)</p> <p>To determine the acceptable values for the lines <code>&lt;1&gt;</code> I used <code>parameters.getSupportedPreviewFormats()</code>. From this, I know that the only supported formats on the camera are <a href="http://developer.android.com/reference/android/graphics/ImageFormat.html#NV21" rel="noreferrer">ImageFormat.NV21</a> and <a href="http://developer.android.com/reference/android/graphics/ImageFormat.html#YV12" rel="noreferrer">ImageFormat.YV2</a>.</p> <p>For <code>&lt;2&gt;</code>, I retrieved the <a href="http://developer.android.com/reference/android/media/MediaCodecInfo.CodecCapabilities.html" rel="noreferrer">MediaCodecInfo.CodecCapabilities</a>-object for type <em>video/avc</em>, being the integer values 19 (corresponding with <a href="http://developer.android.com/reference/android/media/MediaCodecInfo.CodecCapabilities.html#COLOR_FormatYUV420Planar" rel="noreferrer">MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar</a> and 2130708361 (which doesn't correspond with any value of <a href="http://developer.android.com/reference/android/media/MediaCodecInfo.CodecCapabilities.html" rel="noreferrer">MediaCodecInfo.CodecCapabilities</a>). </p> <p>Any other value than the above results in a crash.</p> <p>Combining these settings gives different results, which I'll show below. Here's the screenshot on Android (i.e. the "real" colors): <img src="https://i.stack.imgur.com/fjFXp.png" alt="Input on Android-tablet"> Here are the results as shown by Gstreamer:</p> <p><code>&lt;1&gt;</code> = NV21, <code>&lt;2&gt;</code> = COLOR_FormatYUV420Planar <img src="https://i.stack.imgur.com/kyrMZ.png" alt="Gstreamer-output for NV21-COLOR_FormatYUV420Planar"></p> <p><code>&lt;1&gt;</code> = NV21, <code>&lt;2&gt;</code> = 2130708361 <img src="https://i.stack.imgur.com/C2MZN.png" alt="Gstreamer-output for NV21-2130708361"></p> <p><code>&lt;1&gt;</code> = YV2, <code>&lt;2&gt;</code> = COLOR_FormatYUV420Planar <img src="https://i.stack.imgur.com/FdJZN.png" alt="Gstreamer-output for YV2-COLOR_FormatYUV420Planar"></p> <p><code>&lt;1&gt;</code> = YV2, <code>&lt;2&gt;</code> = 2130708361 <img src="https://i.stack.imgur.com/rl8xb.png" alt="Gstreamer-output for YV2-2130708361"></p> <p>As can be seen, none of these are satisfying. The YV2-colorspace looks the most promising, but it looks like red (Cr) and blue (Cb) are inverted. The NV21 looks interlaced I guess (however, I'm no expert in this field).</p> <p>Since the purpose is to communicate with Skype, I assume I shouldn't change the decoder (i.e. the Gstreamer-command), right? Is this to be solved in Android and if so: how? Or can this be solved by adding certain RTP payload information? Any other suggestion?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USgleerman
    1. USgleerman
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMediaCodec and Camera: colorspaces don't match
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POMediaCodec and Camera: colorspaces don't match
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POMediaCodec and Camera: colorspaces don't match
      1. This table or related slice is empty.
      1. VTUpMod
    1. COLooks like 2130708361 corresponds to [COLOR_FormatSurface](http://developer.android.com/reference/android/media/MediaCodecInfo.CodecCapabilities.html#COLOR_FormatSurface). You must be using API version > 17.
      singulars
      1. POMediaCodec and Camera: colorspaces don't match
      1. USRyan
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload