AMS log core.xx.log is filled with Assert failed errors

[Nicolas Dion-Bouchard]

I get this message in my core.xx.log files:

Assert failed in .\core\FcRecordedStream.cpp line 2420

I would like to know what this error is related to. I assume it is related to the recording or playback of my recorded streams but what exactly? We have problem on the server where stream recording stops after a while, randomly, and I would like to be sure that error has nothing to do with it.

I use AMS 5.0.6 on a Windows Server 2008 R2.

Any help would be great! Thanks

[hparmar]

This will happen only for non H264 codec streams and if you are using older flash player...

Make sure you have <EnhancedSeek zone="client">true</EnhancedSeek> in <StreamManager> tag in Application.xml...

Just check documentation for this flag in Application.xml for details.

[Nicolas Dion-Bouchard]

Ok, that's good to know. And what exactly is an "older flash player"? We are requiring Flash Player 9 but most of our customers have 10.3 and up. Also, we cannot use EnhancedSeek on the client side. We have recorded sessions that can last for hours and if one would like to seek to, say 2 hours in the stream, he would have to wait for the stream to download all data till that point!? That's not a nice scenario.

So, you are saying that this error occurs when an old Flash Player tries to stream a recorded file which is not using the H264 codec? Can you be more precise about the codecs? I mean, if there is only audio, say nellymoser or even speex, will I get this error? We also use telephony codecs, G.711, for some rooms. Can that result in the same error?

Thanks!

[hparmar]

I do not know which FP version added support for enhanced seek..Use latest flash player available whenever possible...You can check FP version of the player playing a stream from access logs too.

I think there are only two codes possible in FMLE H.264 and VP6...What publisher you are using? BTW when doing seek you need to seek to the keyframe nearest to the point...if the client/FP can generated a key frame then you set zone to client and in that case you should not get this assert...Please see documentation in Application.xml for details.

[Nicolas Dion-Bouchard]

If I refer to the official documentation (Adobe Flash Media Server 4.5 * Application.xml file), I should use Flash Player 9 update 3 at least to be sure to be compatible.

We use Flash Player to publish camera and microphone feeds from the user directly. The stream is published in "live" mode. On the server side, whenever we want to record, we create a new Stream and attach the client's stream to it, then record the new stream. That way we can control the recording without interrupting the live session.

So the error comes from the fact that we are playing a recorded stream, with enhanced seek enabled on server-side, and the stream is in H264. If that is the case, it could be problematic but at least knowing it we can figure a workaround. I suppose the faulting streams are recorded in FLV with Nellymoser. But what I don't understand is that, in the documentation, it says that the server generates new keyframes for Sorenson Spark-encoded FLV files, not H264 files?! So anyway the solution would be to de-activate server-side enhanced seek and leave it to the client? That's not encouraging...

A clear error message in the server log would also be appreciated, maybe in a future update...

[hparmar]

if you wish you can upload one such bug file(should be about 15 mins) long and i can investigate the issue.

[Nicolas Dion-Bouchard]

By "bug file" you mean the log file? I don't really have anything specific, all I have in my core.00.log is like this:

#Fields: date

time

x-pid

x-status

x-ctx

x-comment

2014-10-19

00:02:10

37964

(e)2581279

Assert failed in .\core\FcRecordedStream.cpp line 2420

-

2014-10-19

00:02:10

37964

(e)2581279

Assert failed in .\core\FcRecordedStream.cpp line 2420

-

2014-10-19

00:04:29

37964

(e)2581279

Assert failed in .\core\FcRecordedStream.cpp line 2420

-

2014-10-19

00:04:32

37964

(e)2581279

Assert failed in .\core\FcRecordedStream.cpp line 2420

-

2014-10-19

00:04:33

37964

(e)2581279

Assert failed in .\core\FcRecordedStream.cpp line 2420

-

2014-10-19

00:04:35

37964

(e)2581279

Assert failed in .\core\FcRecordedStream.cpp line 2420

-

We have hundreds of clients connected at the same time and I don't really know what's happening at that precise time. I can't event find out what session the core 37964 is running. I searched the logs and normally I have a message in core.log saying something like "Core (_defaultRoot_:_defaultVHost_:[myInstanceName]) is shutdown by the script." but I can't find the message associated with that core so I have no information on the session where it happend.

What information would be pertinent for you so I can help a little more? Thanks!

[Nicolas Dion-Bouchard]

I don't know if that can help but I made a quick test recording a session in FLV. I have a single stream with audio (Nellymoser). I had my camera feed on the stream while it is recording at some point. When I go in playback mode and play the stream, at the point where I added the camera feed to the stream, I get the error in the log file. Pretty straight forward!

Does that mean that I would not be able to seek (using enhanced seek, sever side) in this stream? Because it seems to be behaving normally... And the seek seems to work too.

[hparmar]

Well i was looking for recorded flv which produces the error.

[Nicolas Dion-Bouchard]

Yeah, actually, I tried playing one of the files in a simple swf player I made (in AS2) which only connects to the server and plays the NetStream and I don't get the error. Then it must be some manipulation or the use of seek on the stream. I suppose if you have access to the original code in FcRecordedStream.cpp you must be able to tell where the error could come from.

[hparmar]

Normally when seeking Flash player would collect all the data from the point of last keyframe to the point to which it it is actually seeking...so in a typical stream if the key frame occurs at roughly 10 second interval, then client will download only data for at max 10 seconds...It would not normally be required to download the entire file.So i am not sure what your pain point is.

[Nicolas Dion-Bouchard]

Well the "pain point" is that we have multiple streams to synchronize together while we are in playback mode. So if we have, say, 10-15 NetStreams playing at the same time and we wish to seek in the playback, we have to seek on each stream at the same time, thus slowing the seeking process. It's really more optimal from what I know to use server-side enhanced seek instead of client-side.

At least I know now where the error comes from. Is there a way to prevent this particular error from filling the log files?

[hparmar]

Is there a way to prevent this particular error from filling the log files?

No...not that i know of of...The functionality to generate on the spot keyframes(for sorenson/screenshare i.e Vp6 codec) has been removed from code all together and this assert is to let AMS end users know that this functionality is non existent in lieu of it being already available on client side(Flash Players). You must use client side seek to avoid this assert...The assert however is benign by itself.

Btw client side seeking for 10-15 netstreams is not that big an ask...if the server can serve 100 client connections simultaneously it can surely send 10 seconds of frame data to about 15 subscribers without any performance delay. however upon closer inspection i see that you have audio only stream and there is no  key frame, then it may go back all the file looking for a key frame and even then it may not find it and looking back into the file may be an issue with with you...

Thus if you are sure that your stream is audio only stream, one thing i can suggest you is to adjust StreamManager><KeyFrameInterval> ...it is set to 60000, but its default value is 1000 and it can be anything > 0.Note that it will affect how deep server will look into before rejecting its search for keyframe.

That said, the best bet would still be to remove any video tracks from your flv files, since if there are no video tracks it probably will not seek for video keyframes at all and for that you can even trying recording your content/files as mp3 instead of flv.