Skip to main content
X
Xrtili7
Known Participant
January 30, 2013
Question

Live stream problem with FMS 4.5 + FMLE

  • January 30, 2013
  • 2 replies
  • 2675 views

Hi,

I've got a live stream (and DVR) setup using FMS 4.5 and FMLE.

The hardware for the FMS server we use is 2x Xeon E5645 2.4GHz, 32GB RAM, OCZ RevoDrive R3 X2 PCI-e SSD. Installed OS is CentOS 5.8.

We have 6 workstation PCs, each equiped with 4 BlackMagic Intensity PRO capture cards and FMLE 3.2. Using these PCs we stream Live video to FMS server.

Video stream is grabbed from HDMI cable, to BlackMagic Cards, which are encoding by FMLE using H.264 format, 700kbps bitrate per stream.

Video stream is published to server through RTMP stream, which is later saved for DVR in RAW format. We only keep 3 days of recording and delete the old ones.

Server and Encoder PCs are in the same network, connected by gigabit managed switch.

The problem I'm having is that after 10-15 hours FMLE starts to drop frames, because video buffer is increased. What I observed is that this happens immediately when Server CPU load increases above 60%.

Based on the above observation I decreased the number of channels streamed by the server to 10, which reduced CPU load. But the problem still persists.

Whenever I restart FMS and delete all DVR data, the CPU load (when streaming 10 live channels) is only 1%, but after 2 days CPU load increases to 50-60%.

Whenever I restart FMS and don't delete DVR data, the CPU load is 5-10%, and after 2 days it still increases to 60-70%.

Another thing I observed is that there is only single fmscore process running, but it has lots of threads which are switched on and off in split seconds. These threads are launched on different CPU cores, but at any given point in time the distribution of the load isn't equal among CPU cores. This leads certain CPU cores being loaded by more than 60% and frame drops start to occur.

For the moment there are just 10 users watching this service, so I don't think this load accounts for the problem.

Has anybody had similar problem, or know how can I optimize or finetune the system to run without problem? I would appreciate any suggestions.

Another thing I noted through last couple of days:

When I was restarting FMS the CPU load was reduced to 30%, but after 1 week past when I restart it CPU load only goes down to 75%. Everything is the same, nothing has been changed and there is no disk IO issues involved.

P.S. I've modified application.xml using these values:

<Scope>vhost</Scope>

<Distribute numprocs="5">app</Distribute>

<LifeTime>

<RollOver></RollOver>

<MaxCores></MaxCores>

</LifeTime>

This topic has been closed for replies.

2 replies

X
Xrtili7Author
Known Participant
February 15, 2013

One thing I discovered through further research is that when I changed video format from H264 to VP6 the problem resolved for 1 week. during this time with 4 days of DVR for 21 Live streams together CPU load only reaches steady 30%. After 1 week the CPU load gradually increases and with the same amount of DVR content CPU load is more than 90%!!!

I've found out guys who had similar problems a year ago, but they couldn't find any solution, even from Adobe.

Can anybody suggest any ideas?

X
Xrtili7Author
Known Participant
February 25, 2013

I finally nailed down the source of the problem. As it appears it is the dvr Index file that causes all this trouble.

The thing is that this index file is constantly expanding in size, no matter that we are deleting old video segments. Once this index files become more than 3mb in size, FMS starts to cause problems.

Has anyone got idea how to work with these index files manually? I mean to manually remove the unnecessary data from them?

K
KarishmaB
Participating Frequently
February 25, 2013

Hi

How many channels are you publishing?

If there are too many channels, it is recommended to have one  FMSCore process start for each of them. To do so, you will have to change the scope to app.

---snippet---

<Application>

<Process>

    <Scope>app</Scope>

   <Distribute numprocs="3">inst</Distribute>

.

.

</Process>

</Application>

Also, to delete older content, you will have to enable disk management. Refer http://help.adobe.com/en_US/flashmediaserver/devguide/WSeb6b7485f9649bf23d103e5512e08f3a338-8000.html#WSec225f632fa00875-23954b6f1300b641158-8000 for more info.



A
Anonymous
February 12, 2013

I've experienced similar issues on Red Hat 5.0 as described above. Initially FMS restart was helping to decude CPU load, but when the problem wasn't resolved I moved to Wowza.

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices