FMS NetConnection.call stops responding

I have a FMS 3.5.1 server running an application I developed.  After some time it seems FMS becomes unstable and the flex client can connect (I receive a NetConnection.Connect.Success) but nothing happens after that.  I did some digging, and it turns out that FMS is in a strange state where clients can connect, and FMS can successfully call functions on the flex client, however any NetConnection.call() from Flex to FMS does not work.  I tried changing the flex client to make a function to fms test() that simply traces a test message, but I do not get any errors in Flex making the call, and do not get any errors in any fms log or application log, or the expected trace on FMS.  It's as if the call was never made.  The server however can continue to make function calls to the flex client. Also, after some time, some application instances continue to work while others do not.  Restarting an application that is not responding does not fix the issue.

I made a wireshark capture on the FMS server during this:

frame #47 Server BW, Client BW, Ping, Invoke

frame #48 Server calls onBWCheck on flex client

frame #49 Flex sends Server BW message

frame #51 Flex calls Test() function on FMS

frame #52 Flex calls _result on FMS

frame #53 FMS sends ACK for frame #52

frame #89 Flex sends RST, ACK for frame 53 - RTT for ACK was 16.8 seconds

Earlier in the day I see a bunch of error messages in the logs:

This is from the master.00.log and there are 4971 of these in a row with the exact same timestamp

2010-06-21    04:50:26    10077    (e)2581395    Failed to signal process condition: errno(22).    -

From the edge.00.log, I see a bunch of these about 4-5 per second

2010-06-21    04:48:21    10097    (e)2581394    Failed to wait for process condition: errno(43).    -

core.00.log

2010-06-21    05:57:12    22803    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -
2010-06-21    05:57:12    22803    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -
2010-06-21    05:57:12    22803    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -
2010-06-21    05:57:12    22803    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -
2010-06-21    06:05:17    22803    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -
2010-06-21    06:11:56    22505    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -
2010-06-21    06:51:26    22803    (e)2581279    Assert failed in /FlashPlayer6_02_FlashCom/shared_tcserver/tcparser.cpp line 109    -