StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I thought I'd let interested folks know the approach I actually took. I'm using CSpeex to encode the voice, but I wrote my own block-based video codec to encode the video. It divides each frame up into 16x16 blocks, determines which blocks have sufficiently changed to warrant transmitting, and then Jpeg-encodes the changed blocks using a heavily modified version of FJCore. (FJCore is generally well done, but it needed to be modified to not write the JFIF headers, and to speed up initialization of the various objects.) All of this is being passed up to a proprietary media server using a proprietary protocol roughly based on RTP. </p> <p>With one stream up and four streams down at 144x176, I'm currently getting 5 frames per second, using a total of 474 Kbps (~82 Kbps / video stream + 32 Kbps / audio), and chewing up about 30% CPU on my dev box. The quality's not great, but it's acceptable for most video chat applications.</p> <p>Since I posted my original question, there have been several attempts to implement a solution. Probably the best is at the SocketCoder website <a href="http://socketcoder.com/ArticleFile.aspx?index=2&ArticleID=68" rel="nofollow noreferrer">here</a> (and <a href="http://socketcoder.com/ArticleFile.aspx?index=2&ArticleID=70" rel="nofollow noreferrer">here</a>).</p> <p>However, because the SocketCoder motion JPEG-style video codec translates the entirety of every frame rather than just the blocks that have changed, my assumption is that CPU and bandwidth requirements are going to be prohibitive for most applications.</p> <p>Unfortunately, my own solution is going to have to remain proprietary for the foreseeable future :-(.</p> <p>Edit 7/3/10: I just got permissions to share my modifications to the FJCore library. I've posted the project (without any sample code, unfortunately) here:</p> <p><a href="http://www.alanta.com/Alanta.Client.Media.Jpeg.zip" rel="nofollow noreferrer">http://www.alanta.com/Alanta.Client.Media.Jpeg.zip</a></p> <p>A (very rough) example of how to use it:</p> <pre><code> public void EncodeAsJpeg() { byte[][,] raster = GetSubsampledRaster(); var image = new Alanta.Client.Media.Jpeg.Image(colorModel, raster); EncodedStream = new MemoryStream(); var encoder = new JpegFrameEncoder(image, MediaConstants.JpegQuality, EncodedStream); encoder.Encode(); } public void DecodeFromJpeg() { EncodedStream.Seek(0, SeekOrigin.Begin); var decoder = new JpegFrameDecoder(EncodedStream, height, width, MediaConstants.JpegQuality); var raster = decoder.Decode(); } </code></pre> <p>Most of my changes are around the two new classes JpegFrameEncoder (instead of JpegEncoder) and JpegFrameDecoder (instead of JpegDecoder). Basically, the JpegFrameEncoder writes the encoded frame without any JFIF headers, and the JpegFrameDecoder decodes the frame without expecting any JFIF headers to tell it what values to use (it assumes you'll share the values in some other, out-of-band manner). It also instantiates whatever objects it needs just once (as "static"), so that you can instantiate the JpegFrameEncoder and JpegFrameDecoder quickly, with minimal overhead. The pre-existing JpegEncoder and JpegDecoder classes should work pretty much the same as they always have, though I've only done a very little bit of testing to confirm that.</p> <p>There are lots of things I'd like to improve about it (I don't like the static objects -- they should be instantiated and passed in separately), but it works well enough for our purposes at the moment. Hopefully it's helpful for someone else. I'll see if I can improve the code/documentation/sample code/etc. if I have time.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload