StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I'm developing a system which is kind of a DVR that must periodically record a h264 file from a video device and at the same time provide a local preview to allow for adjusting the video parameters and camera view. Although I am no expert at all in this field, I had a relative success streaming h264 over UDP, so I'll try to share what didn't and what did work out for me, which was all based on <strong>ffmpeg</strong> as a server (no audio in my case).</p> <p>Initially I had set up my application to simultaneously record the video to a file and feed it to a .ffm file for ffserver to stream it up using RTP/UDP for the camera preview. The problem with that approach was exactly that when the feeding ffmpeg process stopped to change the video file, the preview would stop and never resume although the subsequent ffmpeg process would have already started feeding ffserver again. With RTP ffserver apparently complains the frames timestamp started back at 0 instead of continuing where they have stopped. Then I realized that if I were able to send the h264 packets over pure UDP I would get exactly the same effect you described, with preview resuming as soon as the next ffmpeg process would take over.</p> <p>While trying to understand ffmpeg documentation I also tried using the mpegts format, but doing that I was getting MPEG2 video at the player on the other end (this shows a multicast address, but it also worked for a specific target):</p> <pre><code>$ ffmpeg -y -f video4linux2 -i /dev/video2 -vcodec libx264 -preset ultrafast /mnt/hd/video.mp4 -an -f mpegts udp://224.124.0.1:5000 ffmpeg version N-35860-g62adc60, Copyright (c) 2000-2011 the FFmpeg developers built on Dec 16 2011 09:47:41 with gcc 4.5.3 configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enab le-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable -libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394' libavutil 51. 32. 0 / 51. 32. 0 libavcodec 53. 46. 0 / 53. 46. 0 libavformat 53. 26. 0 / 53. 26. 0 libavdevice 53. 4. 0 / 53. 4. 0 libavfilter 2. 53. 0 / 2. 53. 0 libswscale 2. 1. 0 / 2. 1. 0 libpostproc 51. 2. 0 / 51. 2. 0 [video4linux2,v4l2 @ 0x8a96b00] Estimating duration from bitrate, this may be inaccurate Input #0, video4linux2,v4l2, from '/dev/video2': Duration: N/A, start: 1325538250.366878, bitrate: 27620 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27620 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc [buffer @ 0x8a9d8c0] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param: [buffer @ 0x8a9c860] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param: [libx264 @ 0x8a97780] using cpu capabilities: MMX2 Cache64 [libx264 @ 0x8a97780] profile Constrained Baseline, level 1.3 [libx264 @ 0x8a97780] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=0 ref= 1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp= 0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0 [mpegts @ 0x8a98100] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts Output #0, mp4, to '/mnt/hd/video.mp4': Metadata: encoder : Lavf53.26.0 Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240, q=-1--1, 30k tbn, 29.97 tbc Output #1, mpegts, to 'udp://224.124.0.1:5000': Metadata: encoder : Lavf53.26.0 Stream #1:0: Video: mpeg2video, yuv420p, 320x240, q=2-31, 200 kb/s, 90k tbn, 29.97 tbc Stream mapping: Stream #0:0 -> #0:0 (rawvideo -> libx264) Stream #0:0 -> #1:0 (rawvideo -> mpeg2video) Press [q] to stop, [?] for help </code></pre> <p>On the client PC I was able to watch the video with <strong>ffplay</strong>, and it was indeed in MPEG2 format:</p> <pre><code>$ ffplay -f mpegts udp://224.124.0.1:5000 ffplay version N-35860-g62adc60, Copyright (c) 2003-2011 the FFmpeg developers built on Dec 16 2011 09:47:41 with gcc 4.5.3 configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable-libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394' libavutil 51. 32. 0 / 51. 32. 0 libavcodec 53. 46. 0 / 53. 46. 0 libavformat 53. 26. 0 / 53. 26. 0 libavdevice 53. 4. 0 / 53. 4. 0 libavfilter 2. 53. 0 / 2. 53. 0 libswscale 2. 1. 0 / 2. 1. 0 libpostproc 51. 2. 0 / 51. 2. 0 [mpegts @ 0x80f02e0] Unable to seek back to the start [mpeg2video @ 0x8111a00] mpeg_decode_postinit() failure Last message repeated 6 times [mpegts @ 0x80f02e0] max_analyze_duration 5000000 reached at 5005000 [mpegts @ 0x80f02e0] Estimating duration from bitrate, this may be inaccurate Input #0, mpegts, from 'udp://224.124.0.1:5000': Duration: N/A, start: 255.420433, bitrate: 104857 kb/s Program 1 Metadata: service_name : Service01 service_provider: FFmpeg Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 104857 kb/s, 30.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc </code></pre> <p><strong>VLC</strong> also worked to play the stream, but only after specifying the switch <em>--demux ffmpeg</em> (thanks to <a href="http://forum.videolan.org/viewtopic.php?f=14&t=78204#p256976" rel="nofollow">this</a>):</p> <pre><code>vlc -vv --demux ffmpeg udp://@224.124.0.1:5000 </code></pre> <p>Since I also wanted the preview to be in h264 but I already had it encoding the file, I tried to use the copy codec for the UDP streaming, but ffmpeg failed with a segfault (versions included for reference):</p> <pre><code>ffmpeg -y -f video4linux2 -i /dev/video2 -vcodec libx264 -preset ultrafast /mnt/hd/video.mp4 -an -vcodec cop y -f h264 udp://224.124.0.1:5000 ffmpeg version N-35860-g62adc60, Copyright (c) 2000-2011 the FFmpeg developers built on Dec 16 2011 09:47:41 with gcc 4.5.3 configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enab le-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable -libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394' libavutil 51. 32. 0 / 51. 32. 0 libavcodec 53. 46. 0 / 53. 46. 0 libavformat 53. 26. 0 / 53. 26. 0 libavdevice 53. 4. 0 / 53. 4. 0 libavfilter 2. 53. 0 / 2. 53. 0 libswscale 2. 1. 0 / 2. 1. 0 libpostproc 51. 2. 0 / 51. 2. 0 [video4linux2,v4l2 @ 0x92c7b00] Estimating duration from bitrate, this may be inaccurate Input #0, video4linux2,v4l2, from '/dev/video2': Duration: N/A, start: 1325539132.411691, bitrate: 27620 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27620 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc [buffer @ 0x92ce860] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param: [libx264 @ 0x92c8780] using cpu capabilities: MMX2 Cache64 [libx264 @ 0x92c8780] profile Constrained Baseline, level 1.3 [libx264 @ 0x92c8780] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=0 ref= 1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp= 0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0 Output #0, mp4, to '/mnt/hd/video.mp4': Metadata: encoder : Lavf53.26.0 Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240, q=-1--1, 30k tbn, 29.97 tbc Output #1, h264, to 'udp://224.124.0.1:5000': Metadata: encoder : Lavf53.26.0 Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, q=2-31, 27620 kb/s, 90k tbn, 29.97 tbc Stream mapping: Stream #0:0 -> #0:0 (rawvideo -> libx264) Stream #0:0 -> #1:0 (copy) Press [q] to stop, [?] for help Segmentation fault </code></pre> <p>Although less than ideal, specifying the format h264 for the UDP streaming part led to a second concurrent h264 conversion, but it worked:</p> <pre><code>$ ffmpeg -y -f video4linux2 -i /dev/video2 -vcodec libx264 -preset ultrafast /mnt/hd/video.mp4 -an -f h264 -preset ultrafast udp://224.124.0.1:5000 ffmpeg version N-35860-g62adc60, Copyright (c) 2000-2011 the FFmpeg developers built on Dec 16 2011 09:47:41 with gcc 4.5.3 configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enab le-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable -libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394' libavutil 51. 32. 0 / 51. 32. 0 libavcodec 53. 46. 0 / 53. 46. 0 libavformat 53. 26. 0 / 53. 26. 0 libavdevice 53. 4. 0 / 53. 4. 0 libavfilter 2. 53. 0 / 2. 53. 0 libswscale 2. 1. 0 / 2. 1. 0 libpostproc 51. 2. 0 / 51. 2. 0 [video4linux2,v4l2 @ 0x913ab00] Estimating duration from bitrate, this may be inaccurate Input #0, video4linux2,v4l2, from '/dev/video2': Duration: N/A, start: 1325539689.729735, bitrate: 27620 kb/s Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27620 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc [buffer @ 0x9141840] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param: [buffer @ 0x913e480] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param: [libx264 @ 0x913b780] using cpu capabilities: MMX2 Cache64 [libx264 @ 0x913b780] profile Constrained Baseline, level 1.3 [libx264 @ 0x913b780] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=0 ref= 1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp= 0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0 [libx264 @ 0x913c820] using cpu capabilities: MMX2 Cache64 [libx264 @ 0x913c820] profile Constrained Baseline, level 1.3 Output #0, mp4, to '/mnt/hd/video.mp4': Metadata: encoder : Lavf53.26.0 Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240, q=-1--1, 30k tbn, 29.97 tbc Output #1, h264, to 'udp://224.124.0.1:5000': Metadata: encoder : Lavf53.26.0 Stream #1:0: Video: h264, yuv420p, 320x240, q=-1--1, 90k tbn, 29.97 tbc Stream mapping: Stream #0:0 -> #0:0 (rawvideo -> libx264) Stream #0:0 -> #1:0 (rawvideo -> libx264) Press [q] to stop, [?] for help </code></pre> <p>On the client side <strong>ffplay</strong> repeatedly complained about some missing information, but after some seconds it finally showed up the video, which was ok but a bit choppy:</p> <pre><code>$ ffplay -f h264 udp://224.124.0.1:5000 [h264 @ 0xa0be740] non-existing PPS referenced [h264 @ 0xa0be740] non-existing PPS 0 referenced [h264 @ 0xa0be740] decode_slice_header error [h264 @ 0xa0be740] no frame! (...) [h264 @ 0xa0be740] non-existing PPS referenced [h264 @ 0xa0be740] non-existing PPS 0 referenced [h264 @ 0xa0be740] decode_slice_header error [h264 @ 0xa0be740] no frame! [h264 @ 0xa0e72e0] max_analyze_duration 5000000 reached at 5013967 [h264 @ 0xa0e72e0] Estimating duration from bitrate, this may be inaccurate Input #0, h264, from 'udp://224.124.0.1:5000': Duration: N/A, bitrate: N/A Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 320x240, 47.27 fps, 29.97 tbr, 1200k tbn, 59.94 tbc [h264 @ 0xa0be740] Missing reference picture [h264 @ 0xa0be740] decode_slice_header error [h264 @ 0xa0be740] concealing 300 DC, 300 AC, 300 MV errors [h264 @ 0xa0be740] Missing reference picture 0KB sq= 0B f=0/0 0/0 [h264 @ 0xa0be740] decode_slice_header error [h264 @ 0xa0be740] mmco: unref short failure [h264 @ 0xa0be740] concealing 300 DC, 300 AC, 300 MV errors 10.78 A-V: 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0 </code></pre> <p>I then tried <strong>VLC</strong> with the same ''--demux ffmpeg'' switch; it also complained about SPS/PPS (don't know what it is about yet) but in the end it played the video really smoothly:</p> <pre><code>$ vlc -v --demux ffmpeg udp/h264://@224.124.0.1:5000 VLC media player 1.1.12 The Luggage (revision exported) Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS") Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE") [0x943346c] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface. Blocked: call to setlocale(6, "") Blocked: call to setlocale(6, "") [0x94cde8c] qt4 interface error: Unable to load extensions module [0x96720e4] h264 demux error: this doesn't look like a H264 ES stream, continuing anyway [0x963989c] access_udp access warning: unimplemented query in control [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS [0x94be644] main input error: Invalid PCR value in ES_OUT_SET_(GROUP_)PCR ! [0xb2a325f4] avcodec decoder warning: disabling direct rendering [0x96dfac4] main video output warning: vlc_object_find_name(postproc) is not safe! [0x94c2474] signals interface warning: signal 17 overridden (0xb6f31030) [0x94c2474] signals interface warning: /usr/lib/qt/lib/libQtCore.so.4(?)[(nil)] [0x96dfac4] main video output warning: late picture skipped (32703 > -4) </code></pre> <p>I'm still trying to figure out what is missing, but the results are pleasing already. After VLC is playing I can stop the feeding ffmpeg that playback pauses and resumes right away when I restart ffmpeg on the other end.</p> <p>Hope this can be of any help, and please let me know if you have any additional info on using h264 as the chosen format in place of mpegts - I suspect this missing SPS/PPS info might have something to do with it.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload