Skip to main content
andytes
Participating Frequently
August 21, 2007
Question

FMS2 core dies after a number of connections

  • August 21, 2007
  • 7 replies
  • 554 views
Howdy folks

My company developed a flash video chat (adult related) application. We have intensively tested the application without problems on the developer edition of FMS and afterwards we decided to buy a license. We have configured the license on the 1000/40/unlimited and fired the site up. But here's the problem. After a certain number of connections (we counted at about 40 simultaneous streaming sessions plus let's say an average of 2 guests per each stream) the core goes off and dies, therefore not loading the video anymore. Each time this happends we need to restart the FMS and it's working fine for another period of time. The system FMS2 is running on is a Dual Intel Xeon 3.2 Ghz with 4 GB of RAM. The processors are running at about 30 - 40% of their capacity right before the core goes down.
Anyone else had any similar experience and if yes what was the solution? Any help is greatly apreciated. Looking forward to hear from the community.

Warm Regards
Andy

    This topic is closed to new replies. Start a new post to keep the conversation going.

    7 replies

    andytesAuthor
    Participating Frequently
    August 23, 2007
    As a quickly follow-up. T
    he core doesnt really dies. I've been testing it when the fmscore went into this "coma" state. The messages about the queue are still there in the log; however if you wait long enough (the longest took about 2 minutes) the video eventually appears. Then i tested as a broadcaster (chat room host) and i saw that i'm connecting pretty much easier but the clients are still experiencing this weird "lag" issue. Anyway, since i'm running fms on the same box with apache/mysql/dns+mysql/etc for the time being i have ordered today a dual quad-core that i'll be dedicating to FMS all alone and see how it performs. I assume the other processes are overlapping with the normal FMS activity at a point (mainly when getting under heavy load). I'm still waiting for an Adobe answer to my web case so i can submit my logs etc. I'll keep you up2date with the state of things on the new server.

    Regards
    Andy
    Anonymous
    August 23, 2007
    quote:
    i'm running fms on the same box with apache/mysql/dns+mysql


    We had the same problem: video freeze, etc... and it was related to the fact that we were running Web Server and FMS on a single server. It was not the only source of problem : we converted video/audio file with ffmepg on the fly and it was the thing that kill our FMS process. We now use several server and everything look fine....

    The best thing to do will be to split your server task(web, FMS and other) among several server and in this case they don't need to be incredible beast!
    andytesAuthor
    Participating Frequently
    August 23, 2007
    quote:
    Originally posted by: anonymous thing
    quote:
    i'm running fms on the same box with apache/mysql/dns+mysql


    We had the same problem: video freeze, etc... and it was related to the fact that we were running Web Server and FMS on a single server. It was not the only source of problem : we converted video/audio file with ffmepg on the fly and it was the thing that kill our FMS process. We now use several server and everything look fine....

    The best thing to do will be to split your server task(web, FMS and other) among several server and in this case they don't need to be incredible beast!



    Hey anonymous

    Thanks for the feedback. I've been adviced by so many persons in so many ways but based on flv streaming scenarios. I do not do that. I am in to a many-to-many scenario where the application instances (sources of stream) come and goes dinamicaly. I get their webcams and stream them to any potential viewers.
    In the flv streaming case the fix for this issue is simple. I set the <Scope> to inst and define RollOver and MaxCores values and i have no stress since the cores will be replacing each other at the defined period of time.
    But in my case (many-to-many) i cant run the fms with the Scope inst but only as app.
    Anyway .. i do hope(think) that separing FMS will solve this issue.

    Regards
    Andy
    andytesAuthor
    Participating Frequently
    August 23, 2007
    Thanks again for answering me Kevin.

    I sent a message to Adobe. regarding this issue by opening a web case. I've been told ill get a reply in 1 business day but still didnt got one. I'm still waiting. However meanwhile i'll have to live with restarts cause as soon as i hit a 40 peak connections the core freezes .. i get those messages in the log and afterwards nothing happends anymore. Hopefully i will manage to solve this as soon as possible. Thanks once more for your time,

    Regards
    Andy
    andytesAuthor
    Participating Frequently
    August 22, 2007
    Hello Kevin

    Thanks for yoru answers. However i do not think there is any of the reasons listed above.

    In my core.log.00 i get right before the core dies about 60 - 70 lines like:
    "2007-08-20 01:29:36 5458 (w)2611173 TCMsgQueue video message queue size info : Size : 100, type: 1, clientID: -1
    396867168, streamID: 1
    "

    Checking the things from admin interfae of the FMS i get in the admin.log.00 the message:
    "2007-08-20 19:19:39 5356 (e)2671287 Admin Command Timeout: (Command (getVHostStats) timed out. No response from
    core(s) : 5458) -"

    The core is not restarted by the master and the edge log doesnt get anything.

    I'm almost pulling my hair out of my head.

    Yours
    Andy
    KevinStreeter
    Participating Frequently
    August 22, 2007
    The queue size messages indicate that your clients are not pulling stream data quickly enough and the queue is growing in size. This may or may not be directly related to the core shutdown, but is definitely a clue. The master should be re-starting the core at least once, depending on you server configuration. The fact that it is not may indicate a problem with the software or the software installation. If you are certain that this is not an out-of-memory issue, and that the core is not being shutdown by the edge as being unresponsive, then it may be a bug in the server. Contact FMS support, and they will help you collect log files, crash dumps, etc, that can be used to diagnose the issue.

    KevinStreeter
    Participating Frequently
    August 22, 2007
    Andy,

    There are two main causes of unexpected core shutdown:

    1) Running out of virtual memory. On Windows, a process can only use 2GB of VM, on Linux, 3GB. If you exceed this, the core will go down. This will be reported in the master log as an unexpected shutdown, and the core will be restarted.

    2) The core becomes unresponsive. This occurs when the core is not responding to new connections in a timely manner. This will be reported in the edge log, the edge will shudown the core, and the master will restart it.

    If you are encountering the first case, you probably have your video cache set too large, or are using alot of memory from within your server-side action script. If you are encountering the second, you probably have something in your script that is blocking the connection process for an extended period of time. Alternatively, you may have encountered a bug in the server. If you feel this is the case, contact Adobe FMS support, and have your log files available for review.
    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices