Buffering video while recording using webcam

[Guest]

I'm using Actionscript 3 to record video using a webcam and a Red5 media server. When the Internet speed is low, how do I ensure that the recorded video frames are buffered, and not dropped? When the Internet speed is high, it can stream the buffered frames to the server.

None of the buffer functions in the NetStream class seem to address this, or am I missing something?

[sinious]

Streaming video via lossy compression is self describing, lossy. That's what a live video stream is. Accepting loss is part of it to keep up with sync. If you really want to record full speed and transmit you're going to need to do just that, record local, encode for stream delivery and send it out. This will definitely add a delay.

The situation you want to handle is just not possible. Consider if the users bandwidth became restricted for over a minute (someone starts a torrent in the house, etc). What you want is to assure (in a TCP way) that every single frame is delivered. The user on the other side would be playing catch-up for so long it would become far desynched. All you can do is continuously monitor the network and adjust camera quality accordingly to keep the frame rate high. That may mean re-negotiating a smaller resolution, lower quality and frame rate as needed to keep up with the bandwidth. Or (in a UDP way) accept loss. 

[Guest]

Thanks, so can you (or someone) explain exactly what the NetStream.setBufferTime() and NetStream.bufferTime methods do? I would like to know how exactly it works - how the buffering is done, and so on.

The Adobe AS3 documentation states :

The number of seconds assigned to the buffer by NetStream.setBufferTime(). The default value is 9. To determine the number of seconds currently in the buffer, use NetStream.bufferLength.

When I added setBufferTime(10) to my NetStream object, I saw no difference in behaviour of the recording.

Thanks.

[sinious]

That's ActionScript 2.0 documentation actually. The method doesn't exist in AS3.0.

Here is AS3.0 NetStream:

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStream.html

The property equivalent would be bufferTime:

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStream.html#bufferTi...

And bufferLength:

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStream.html#bufferLe...

These help a one way streaming channel over NetConnection. Typically they're used for progressive or streamed playback but not live video. During live video these amounts should be very low. Otherwise your video on the receiving side may pause to allow the buffer to fill before continuing to receive the video.

[Guest]

Thanks, so if I set BufferTime to 60, what is it that I'm actually setting? Is it the maximum time worth of data that can be stored in the buffer? If yes, then how is it different from maxBufferTime?

[sinious]

bufferTime is the amount of time a stream will stall to fill before streaming anything. If you set it to 60 seconds, it will wait until it has 60 seconds worth of video in the buffer before streaming (that's a lot).

bufferTimeMax (not maxBufferTime) deals with live streams and can have a (sometimes amusing) speed effect on the video or audio to help the user "catch up" to the live stream. It's allowed to speed up the video or audio. How it does this is and how much is explained in its property details:

When bufferTimeMax > 0 and bufferLength >= bufferTimeMax, audio plays faster until bufferLength reaches bufferTime. If a live stream is video-only, video plays faster until bufferLength reaches bufferTime.

Depending on how much playback is lagging (the difference between bufferLength and bufferTime), Flash Player controls the rate of catch-up between 1.5% and 6.25%. If the stream contains audio, faster playback is achieved by frequency domain downsampling which minimizes audible distortion.

Are you trying to capture a perfect video feed for storage or for live streaming relay? Each approach would require different tooling. If you're trying to capture I can understand why you'd be willing to hold a huge buffer. For live streaming, loss is inevitable.

[Guest]

Thanks, I was capturing video for storage.

One thing which I completely failed to understand was this :

when I kept the buffertime value high - like 1000. Then over a slow internet connection, or for high video quality, the buffer got filled very quickly, (say till 200) and then when I stopped recording, the buffer emptied itself. However, the final video that was saved was very jerky, and mostly stalled throughout playback.

On a related note, when the video quality was low and/or the internet connection was good enough, the playback quality was very smooth, and the buffer didn't fil beyond 1.

This somehow contradicts what you said "bufferTime is the amount of time a stream will stall to fill before streaming anything" - it seems to stream even when it is not full. (because the bufferlength never went above 1).

I can assure you that my observations are correct, because I've been testing this aspect of netstream for over a week now.

How would you explain this?

[sinious]

Let me be a bit more clear. The first sentence in bufferTime explains the difference.

Specifies how long to buffer messages before starting to display the stream.

bufferTime stall is client-side, that is where video is 'displayed'. That means a client that has bufferTime set to receive video will wait up to the buffer before displaying.

When you're sending video, buffers are only used when the bandwidth drops below the amount necessary to fully stream. At that point the buffer fills (usually set at 0.1 to just drop frames instead) and then bufferTimeMax will come into play. When your bandwidth opens up bufferTimeMax can send the video/audio out faster to 'catch up' and resync to live. If you should run out of buffer, this is where you just start to drop frames completely or else the video will become terribly desynched.

If you're capturing for anything that isn't live and video quality is important, if you can use something like AIR then I would fully capture it local first, then transfer it later. But I really don't know what you want to do.

[Guest]

I'm trying to record video for later review. Which means that I cannot allow too much loss, and the application should run on a browser.

This is why I thought I need to keep a high buffer time. But I don't get why the recorded video is extremely jerky and stalls whenever, during the recording, the bufferlength exceeds 1 or 2 (ie when it reaches something around 100 or so), given that I wait while all the data in the buffer empties to the server.