ColdFusion Webservice Crashing at 30 Active Transactions

[sai manindra0D44]

Hello, 

We are using ColdFusion 2016 Configured with IIS 10 With MicroSoft SQL server Being the Database, We use Fusion Reactor to Monitor Application Load,
Application is Crashing When the Running Transactions :Web Request Exceed 25 and Goes down at 30 until a Restart is done, Is there a setting where we can increase the threads?

Our Application is Internal and gets around 15-20 Requests every Minute.

I have Looked All over but Didn't get the Answer, 

I am new to ColdFusion Administration Please guide me in the Right Direction.

Thanks,
SAI

[Charlie Arehart]

Sai, there are a couple of ways to answer this.

First, the real solution is not so much in RAISING that limit as finding WHY you are hitting it. But I appreciate that in the pickle you're in, raising the limit seems the "simplest" solution.

1) Raising the limit (only a band-aid, usuallly)

So for raising the limit, that's controlled in the CF Admin, in the first "Settings" section, in its "Request Tuning" page, where the first setting there is "Maximum number of simultaneous Template requests".  Raising the value will raise that limit you hit.

1a) Well, it will if you are on CF Enterprise, for sure. If so, you're done. You can then move on to step 2 below (to really solve the problem.)

1b) But if you are on CF Standard, there's an odd bug where the number shown in that CF Admin page there is NOT correct, and if you change it it may NOT work. In that case (with CF Standard), the best thing is to edit the underlying XML file that the admin would change. It's in the neo-runtime.xml file, which is found in the cfusion/lib (or [instancename]/lib if you were on CF Enterprise and have multiple instances).

You would want to copy that file, then edit it, changing the element:

<var name='requestLimit'><number>25.0</number></var>

Then restart CF. Again, with CF Standard there will be no way to KNOW that you raised the limit, because of the bug where it pulls in a value from somewhere else (the maxthreads for the ajp connector in the runtime/conf/server.xml file).

And you must be very careful when editing this file. You can't take out a quote or bracket by mistake or CF may not start. That's why I had you make a backup first. But that WILL change it. Let us know how things go. But that's just a band-aid, not solving the REAL problem.

2) Solving the REAL problem: the hanging requests

Since you say you reached this limit and things hung and went down, then just raising that admin value is not the REAL solution to the problem. It will likely only delay an inevitable new hangup at the new level. The REAL solution is to find out WHY requests are running slow and piling up, and fix that. There can be MANY reasons for such problems.

But the good news is that you can find this info easily with the FusionReactor you say you are using, using any of its various features:

• you can see for each request if it has one or more long-running queries, in the "jdbc" tab in the request details, either while the request is running or afterward
• you can see for each request if it has one or more calls out of CF to something else (such as via a cfhttp or other service connection) using the "relations" tab, again for a running or completed request
• you can use the FR "profile" feature (if using FR Ultimate or trial), to see from a JVM level where the most time was spent in a request (which more challenging to interpret, for most folks)
• you can use the FR "stack trace" feature while a request is running, to see what it's doing at that moment, down to the line of CFML code...and you can refresh that while a request is running to see if it's "stuck" on some one line of code
• you can use the FR Crash Protection alert emails, to find out much of the info above, about ALL running requests AT THE TIME of an alert

So bottom line, you can find and resolve this problem. Besides using the FR docs and site to find more, I have done blog posts, webinars, and presentations including youtube videos on using all the above to solve such problems.

Or I can be hired directly to help do it even faster, more at carehart.org/consulting.

About 25 vs 30

One last thing: I know you refer to 25 and 30 here, in the title and the body. You say it's "crashing" at 25 and "goes down" at 30. I'm not quite sure what difference you mean there, between crashing and going down.

Either way, I suspect you will find your limit (in the admin in Enterprise or the XML file in either Std or Ent) is indeed 25. That's the default. As for the difference you see at 25 or 30, there will be more you can dig into in FR to understand that distinction but again that's not the real matter to focus on.

Solving the hanging requests is the real problem, and above are several ways to solve that problem. (And some can be done in the CF PMT, for those running that in CF2018 or 2021, where FR can also still run. One can run both at the same time, and each has things the other does not.)