Coldfusion sites experiencing significant delays

I’m curious if you have Sandbox security enabled?  A while back we had an issue with CF’s Tomcat intermittently hanging and turning Sandbox security off resolved the issue.  Seemed to be related to  surges in traffic the site was getting from bots, etc.  This was using IIS on  a Windows server.

I’ve just had another idea. Though you mentioned there were no changes, Antivirus or EDR (End-Point Detection and Response) updates often occur automatically.

If an Antivirus update started "inspecting" the AJP packets on the AJP port, it would cause exactly this kind of delay, without any CPU spikes. So check from the logs of any such security software (in any is installed) whether it was active at the time. If so, then temporarily disable the Antivirus or add an exclusion for the ColdFusion and IIS directories, as well as the AJP port.

The errors in isapi_redirect logs are a good place to start the diagnosis. Apparently, connections are involved. So, could you please share the workers.properties file and the full text of the connector element <Connector protocol="AJP/1.3"> (from \runtime\conf\server.xml).

So to be clear, on the page where debug output says it took only 100ms, the page rendering time in the browser was several seconds (like all those you are saying are a sudden common problem)? That’s important for us to know.
 

And indeed, if it’s true, the next question is whether you find your browser’s “developer tools” feature (in its Network tab) confirms that there was indeed all that slowness in the serving of the CF page itself. You may well find it’s from something that the CF page pulls in as HTML, which the browser is then hanging up in serving.

If indeed the problem is “on the CF page” (despite what your debug output is telling you), you would next benefit from any tool that monitors what’s going on within any or all CF requests. The most popular tool is FusionReactor (which has a free 14-day trial), and the next is the CF PMT (new since CF2018, but far more heavyweight if implemented on the same server as CF itself).

I’ll note also (as some others will want to) that you are playing a game of Russian Roulette in still running CF2021, which got its last update in early Dec (it came out in late 2020 so is now EOL and will get NO more updates, including security updates). There was another CF update in January, which was NOT made available for CF2021, so you are now vulnerable to what it fixed (and your perf problems COULD be related to that, though not necessarily).

Indeed, you may report by tomorrow that the problem is “gone”. Some problems do come and go, and while you don’t see a DDOS, it could just be excessive request traffic of a new nature from perhaps spiders or bots (or hackers or AI engines). Again, a monitoring tool can help spot that, in tracking all requests (or you could try to dig through your web server logs, which is tough given that they track ALL requests not just CF).

Finally, I’ll say that if the problem remains and neither this note (or my subsequent replies) nor any thoughts from others helps resolve things, I help people solve such problems about daily via remote screenshare consulting. I can sometimes solve even such knotty problems in far less than an hour (my minimum time interval is just 15 mins). More on my rates, approach, satisfaction guarantee, online calendar and more are at carehart.org/consulting. But clearly I’m trying to help WITHOUT requiring that from you.

These sort of problem hit a lot of people, and there is ALWAYS an explanation--often a very surprising one, and it’s generally easily solved. The challenge if finding the root cause.  That can be heard for us to guess at here, which is why direct access (looking at it together) is often far more effective. But we’re here to try, if this is the only way you may be allowed or prefer to seek help..