Application Unexpected Unloaded with Error
We are using a Origin/Edge configuration for massive video delivery, and we are experiencing problems with huge audiences. Basically, when the edge server reaches thousands of users watching (6000) a live video in a single application, this instance is unloaded by the server with the following error:
==> edge.00.log <==
2010-06-1115:51:1018614(i)2701054Virtual host Disconnecting unresponsive core live/stream (28860) after 30 seconds is not available.-
==> core.00.log <==
2010-06-1115:51:1028860(i)2581233Core (28860) disconnecting from admin: 200.-
==> master.00.log <==
2010-06-1115:51:1018593(w)2581257Core (28860) _defaultRoot_:_defaultVHost_:::_0 is not responding and is being restarted!-
2010-06-1115:51:1018593(i)2581221Core (29375) started, arguments : -adaptor "_defaultRoot_" -vhost "_defaultVHost_" -app -inst
-tag "_0" -conf "/opt/adobe/fms/conf/Server.xml" -name "_defaultRoot_:_defaultVHost_:::_0".-
==> edge.00.log <==
2010-06-1115:51:1018614(i)2581250Edge disconnected from core (28860).-
2010-06-1115:51:1118614(i)2581252Registe ring core (29375).-
==> core.00.log <==
#Date: 2010-06-11
#Fields: datetimex-pidx-statusx-ctxx-comment
2010-06-1115: 51:1129375(i)2581237Starting admin app on core (29375).-
2010-06-1115:51:1129375(i)2581238Core (29375) connecting to admin.-
2010-06-1115:51:1129375(i)2581231Core (29375) connected to admin.-
2010-06-1115:51:1129375(i)2581234Core (29375) connection to admin accepted.-
2010-06-1115:51:1129375(i)2581246Core (29375) sending register cmd to edge.-
2010-06-1115:51:1129375(i)2581246Core (29375) sending register cmd to edge.-
2010-06-1115:51:1129375(i)2581246Core (29375) sending register cmd to edge.-
2010-06-1115:51:1129375(i)2581246Core (29375) sending register cmd to edge.-
==> edge.00.log <==
2010-06-1115:51:1118616(i)2581252Registering core (29375).-
2010-06-1115:51:1118616(i)2581250Edge disconnected from core (28860).-
2010-06-1115:51:1118613(i)2581252Regist ering core (29375).-
2010-06-1115:51:1118613(i)2581250Edge disconnected from core (28860).-
2010-06-1115:51:1118615(i)2581252Regist ering core (29375).-
2010-06-1115:51:1118615(i)2581250Edge disconnected from core (28860).-
==> core.00.log <==
2010-06-1115:51:2328860(e)2581279Assert failed in /util/TCAsyncIO.cpp line 1932-
2010-06-1115:51:4228860(e)2581279Assert failed in /util/TCAsyncIO.cpp line 1932-
2010-06-1115:51:5728860(e)2581279Assert failed in /util/TCAsyncIO.cpp line 1932-
==> edge.00.log <==
2010-06-1115:52:2718614(e)2661034Connect failed ( , 8134 ) : Connect failed: Connection refused (111)-
==> master.00.log <==
2010-06-1115:53:1118593(i)2581223Core (28860) is no longer active.-
In this situation, the CPU usage is something around 30%, the memory is almost unused, and the load average is very low. We are using FMIS 3.5.3 854a running in the CentOS.
Anybody knows what is happening, and how to fix it?
Thanks
