How to make video chat application smarter ?

Report · Jul 28, 2013

Hi all,

I am implementing a video chat application which is working fine in high bandwith, but it is not working well on lower bandwith (200Kbps - 500Kbps). Please help me to over come the above constraint. Please find the video resolution details as below:

1) Video resolution : 640 x 480 (24fps , KeyFrame interval 15) -- publishing

2) Video resolution : 320 x 240 (24fps , KeyFrame interval 15) -- Playback

The above two are freezed. The following are the required to achieve

1) If bandwidht is low show an alert to the user

2) Detect speakers are not connected to the machine

3) Detect download / Upload speed of network

Please help me to achieve the above.

Thanks in advance.

Report · Jul 29, 2013

A good start would be pasting in your encoder settings here for us to look at.. Any H264VideoStreamSettings, NetStream videoStreamSettings, metaData object settings you're setting in the stream, etc.

For one, I'd never keyframe "video chat" faster than once per second. It's largely a talking head with a 50/50 chance the background is completely static.

Are you using a media server? For example the old FMS has a bandwidth detection as long as the BandwidthDetection was enabled. You then just called "checkBandwidth" from a NC call and keep stats on the bandwidth returned.

Otherwise bandwidth detection is very faulty. You'd need to monitor the throughput of the video and when you detect the speed lower than desired just show the user a dialog. Nothing more than that is necessary.

You're also not allowed to detect output devices yourself. For example Microsoft doesn't even let apps manage output devices without the user doing it themselves. They consider it a security breach.

Report · Jul 30, 2013

Hi ,

Thanks for your reply, helps me alot.

Please find the encoding setting as below,

var h264Settings:H264VideoStreamSettings = new H264VideoStreamSettings();

h264Settings.setProfileLevel( H264Profile.MAIN, H264Level.LEVEL_3 );

nsPublish.videoStreamSettings = h264Settings;

// set the buffer time to zero since it is chat

nsPublish.bufferTime=0;

/** MetaData information **/

var metaData:Object = new Object();

metaData.codec = nsPublish.videoStreamSettings.codec;

metaData.profile = h264Settings.profile;

metaData.level = h264Settings.level;

metaData.fps = _cam.fps;

metaData.bandwith = _cam.bandwidth;

metaData.height = _cam.height;

metaData.width = _cam.width;

metaData.keyFrameInterval = _cam.keyFrameInterval;

metaData.language = "English";

Camera settings are as below :

FPS : 24

Width : 640

Height : 480

KeyFrameInterval : 15

Band Width : 0

Microphone setting are as below:

Gain : 60

Rate : 11

Thanks in advance.

Report · Jul 30, 2013

On your camera are you using setQuality()? That works hand in hand with setting the bandwidth. You may want to try getting consistent video by actually specifying a bytes per second value. Setting it to 0 means it can use all available bandwidth which can change drastically and make the video appear much less consistent.

I also, again, would never set a keyframe interval less than the number of frames for at least 1 second. I'd typically set it 3~5 seconds (so 72 or 96). That'll really take down bandwidth itself.

You can get away with using half the macroblocks of LEVEL_3 (40,500) at 640x480@24 by using H264Level.LEVEL_2_2 (20,250) which supports the same pixel count (414,720). You should try using H264Profile.BASELINE as it was designed for video conferencing. Main is designed for broadcast. You can read about profiles here.

What is your mic audio quality?

Report · Jul 30, 2013

Your repIy help me alot , I am not setting audio quality parameter, so it will takes the default value. Please find the Microphone settings as below

_micPhone.setUseEchoSuppression(true);

_micPhone.rate = 11;

_micPhone.gain = 60;

_micPhone.codec = SoundCodec.SPEEX;

_micPhone.setLoopBack(false);

What could be the minimum band with is required to publish the video with the following resolution

1) 640 X 480 (24fps, keyFrame interval 3) ?

2) 320 X 240 (24fps, keyFrame interval 3) ?

Thanks in advance.

Report · Jul 31, 2013

If you're sending keyframe intervals at every 3 frames (unless you meant seconds?) you're using way too much bandwidth. Nobody can really tell you the "perfect" bandwidth for those settings because lossy compression works on the hope that "some" part of the video will remain the same frame to frame.

With lossy spatinal compression it analyzes the previous frame and the current frame looking for differences. If you have a person chatting in a room where nothing behind them is moving (no objects, people, tvs, etc), the bandwidth will be very minimal. With a normal keyframe interval set at "96" I would wager you could get away with an acceptable picture at 450kbit video, 23kbit mono audio. 320x240@24 same 96kf interval, just half it to 225/23.

If the chat user is in a very busy environment behind themselves then these settings will be far too low. In a very busy background at 640x480@24f 96kf you could spike over 1mbit just in video and still see plenty of compression.

If you did either of those and set your keyframe interval at "3", double the bandwidth at minimum. KF interval at 3 is saying every 3rd frame, or 8 times per second, send a "full frame" worth of data. That's way too much. In the highest end video I even put online I never go lower than once per second in extremely busy videos, unless I need seriously granular seek on HTTP progressive video.

FYI the 23kbit audio comes from the SPEEX codecs quality setting on the Microphone:

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/media/Microphone.html#encod...

That'd be setting:

_micPhone.encodeQuality = 7;

You might want to do yourself a favor and give the user simple controls to adjust their own settings. A video bandwidth/quality slider, same for audio. Behind the scenes just ramp up the bandwidth. The user can choose what works for their specific situation without you needing to find a magic formula.