Origin - edge limitation on VOD but not on LIVE

Report · Feb 23, 2012

Hi ,

we have an origin server where video materials are stored and one edge where clients are conected to.

If i allow clients to connect directly to core server it can go easily to 1000 users without any problems.

If the clients connects to edge after 100 users the player will start buffering the video every few seconds.

On the other hand if i connect a FMLE to core ( same video bitrate as materials already encoded ) clients can connect to edge and there is no problem

( tested with 1000 users ) and no buffering issues appeared.

There is no IO wait problem on core, edge is configured not to use any cache.

There is no connectivity isssue between master and edge.

Both origin and edge are 4.0.3.

Anyone experienced anything similar or has any ideea why this is happening ?

Report · Feb 24, 2012

So all 1000 or 100 users play same asset or different assets. And when you try to simulate same case with live - again do all 1000 users play same stream or different streams.

Have you tried playing around with FLVCACHE_MAXSIZE and see if that value makes any difference.

Report · Feb 24, 2012

Users play different assets it's many to many situation on VOD.

I'll play around with FLVCACHE_MAXSIZE and see if there will be a change.

Report · Feb 25, 2012

Can you also write the contents of your core logs here ? I am afraid application log won’t give me any useful information about the error.

You see just a control file written if the recording does not happen successfully and stops after creating the first file (.control).

Report · Feb 25, 2012

On origin server:

#Date: 2012-02-24
#Fields: date time	x-pid x-status	x-ctx x-comment
2012-02-24	17:24:52	6090	(i)2581237	Starting admin app on core (6090).	-
2012-02-24	17:24:52	6090	(i)2631174	Listener started ( _defaultRoot_ ) : 127.0.0.1:19353 (rtmfp-core)/v4	-
2012-02-24	17:24:52	6090	(i)2631174	Listener started ( _defaultRoot_ ) : xxx.xxx.xxx.xxx:19353 (rtmfp-core)/v4	-
2012-02-24	17:24:52	6090	(i)2631174	Listener started ( _defaultRoot_ ) : xxx.xxx.xxx.xxx:19353 (rtmfp-core)/v4	-
2012-02-24	17:24:52	6090	(i)2631174	Listener started ( _defaultRoot_ ) : xxx.xxx.xxx.xxx:19353 (rtmfp-core)/v4	-
2012-02-24	17:24:52	6090	(i)2631509	Public rtmfp-core addresses for listener _defaultRoot_ are: 127.0.0.1:19353;xxx.xxx.xxx.xxx:19353;xxx.xxx.xxx.xxx:193
2012-02-24	17:24:52	6090	(i)2581246	Core (6090) sending register cmd to edge.	-
2012-02-24	17:24:56	6090	(i)2581238	Core (6090) connecting to admin.	-
2012-02-24	17:24:56	6090	(i)2581231	Core (6090) connected to admin. -
2012-02-24	17:24:56	6090	(i)2581234	Core (6090) connection to admin accepted.	-
2012-02-24	17:24:56	5543	(i)2581238	Core (5543) connecting to admin.	-
2012-02-24	17:24:56	5543	(i)2581231	Core (5543) connected to admin. -
2012-02-24	17:24:56	5543	(i)2581234	Core (5543) connection to admin accepted.	-
2012-02-24	17:24:56	5542	(i)2581238	Core (5542) connecting to admin.	-
2012-02-24	17:24:56	5542	(i)2581231	Core (5542) connected to admin. -
2012-02-24	17:24:56	5542	(i)2581234	Core (5542) connection to admin accepted.	-
2012-02-24	17:27:52	5543	(w)2581409	Warning: Process virtual memory size has reached warn config value of: 1800	-

on edge:

Version: 1.0
#Start-Date: 2012-02-24 00:20:07
#Software: Adobe Flash Media Server 4.0.3 r4011 x86
#Date: 2012-02-24
#Fields: date time	x-pid x-status	x-ctx x-comment
2012-02-24	00:20:05	28743 (i)2581250	Edge disconnected from core (31114).	-
2012-02-24	15:24:28	28743 (e)2581279	Assert failed in tincan/server/edge/TCEdgeControlTransport.cpp line 273 -
2012-02-24	15:24:28	28743 (i)2581252	Registering core (2916).	-
2012-02-24	15:49:30	28743 (i)2581250	Edge disconnected from core (2916).	-
2012-02-24	17:55:22	28743 (e)2581279	Assert failed in tincan/server/edge/TCEdgeControlTransport.cpp line 273 -
2012-02-24	17:55:22	28743 (i)2581252	Registering core (3727).	-
2012-02-24	22:50:28	28743 (i)2581250	Edge disconnected from core (3727).	-

Report · Feb 25, 2012

From your logs it looks like you origin server reaches close to 2 GB limit for process.

Can you tell timing of Assert seen in edge is same time where your clients starts buffering?

Report · Feb 27, 2012

The timing of assert it's not correlated with the buffering issue. The buffering apears when aprox when 100 clients connects to edge. I have played with FLVCACHE_MAXSIZE and increased it from 500 to 2000 and no effect.

Still same behaviour.

Report · Mar 07, 2012

We have almost the exact same issue with our Origin/Edge setup. Though we are only streaming Live TV to our customers. It seems that when more than 20 different Live streams (1200 Kbit per stream) are pulled by the Edge server, the clients connected to the Edge starts to buffer.