Repeated Active Request Spikes - CF 11 Update 7

[ColdFusionSmurf]

We've been seeing a pretty severe problem with CF 11 Updater 7 (and now 8) where the active requests just spike out to our max and the server starts throwing timeouts and 503 errors.

We run a ColdFusion 11 Enterprise server (currently patched with update 6), which handles about 30 requests per second on average.

Periodically, the metrics graphs will just show a spike in active requests that jumps up to our max (100 threads).  This last for about 15-20 seconds and then recovers, but accompanying it are usually 10-20 timeout errors and who knows how many 503's.  And this ends up happening every 15 minutes or so.  It's like something is slowing down the processing of pages, or holding them up for long enough to queue out of control.

We tried installing update 7 and it happened all day, so we rolled back out of it.  When update 8 was released, we installed that and encountered the same problem.  I'm guessing that it was actually in update 7 and the fact that updates are cumulative means I got it again.

Has anyone experienced this before?  Is there any way I can install update 8 without 7?

[EddieLotter]

Enable logging on your Web server, and review them after this issue occurs. The logs will allow you to see the IP address of every connection. This might lead you to where the spike in activity is coming from.

Cheers

Eddie

[senchaworld]

I'd put a trial of fusion reactor on the server which will help you gather further detailed info.

also hotfix9 is out for cf11... try that first.

[ColdFusionSmurf]

I think everyone is misunderstanding the problem here...

1. The active request spike is the symptom, not the issue.  It's pretty clear to see when watching the graphs.  It's like something else is happening to slow down requests for some reason, and everything slows down just enough for the constant requests we get to spike out of control.

2. We have Fusion Reactor already installed, that's how I'm watching the request graphs.

3. I may at some point try Update 9, but I'm pretty certain it will only cause the same problems.  I feel pretty confident in saying that Update 7 is the problem here.

[EddieLotter]

ColdFusionSmurf wrote:

Is there any way I can install update 8 without 7?

Yes, the updates are cumulative. If you apply the currently available update 9, it will include all previous updates.

Cheers

Eddie

[ColdFusionSmurf]

I don't feel like you really understand what I'm asking here, Eddie.  The fact that the updates are cumulative is my problem.

[EddieLotter]

Really? That's basically what I said, ColdFusionSmurf. The fact that the updates are cumulative means that, no, you cannot apply a later update without effectively applying all previous updates. I'm sorry that's not the answer you wanted to hear, but it remains the answer none-the-less.

Cheers

Eddie

[ColdFusionSmurf]

Okay, I see where my confusion lay.

You quoted my question, and the answer to that question was "No."  However, your answer, while technically accurate began with you saying "Yes," which made me believe you were answering my question with a "Yes."  Based on that, I assumed you did not understand what I was asking.

[EddieLotter]

No problem. I'm sorry I was not clear.

Given that you suspect some process is hogging server resources, are you using server level resource monitoring software to try to figure out which process appears to be the hog when this starts happening?

Cheers

Eddie

[paule12345]

Do you have sandbox security enabled?  If so, turn it off and see if the problem goes away.

We recently had severe problems with the DB connection pool hanging when a certain (hard to define) threshold of incoming requests was reached.  CF10 and CF11 both exhibited the behavior.  With sandbox security disabled, the hangs disappeared.  If you must have sandbox security enabled, in the CF admin, try limiting your DSN's connections to, say, 10 max connections.  It seemed to help.

I've been meaning to submit a bug report on this...