ColdFusion 2023 scheduled tasks all fail with 'Connection failure'

[DNomer]

These tasks have been running nightly without trouble. Then we installed a new SSL cert on IIS 10, and now they all fail. I can run each task from my browser, but if I try to run one from the task scheduler, now it fails. Restarting CF after the new cert does not fix it. ColdFusion serves the many pages on our site just fine (SSL is required), it just won't run any scheduled task now.

[Charlie Arehart · Accepted solution]

I think you'll find first that it's not about scheduled tasks, but that there will be a perhaps simple solution (different what some may propose). 

1) First, in terms of identifying perhaps the actual problem, create a simple test cfm page that does a cfhttp to the same https url as the task. It will likely fail also, which shows that it's not about scheduled tasks, per se. 

Then you can cfdump the cfhttp variable to see what error it reports. Tell us that. 

2) My suspicion is that the new cert is of a sort that Cf has trouble using for https. More specifically, it's likely not cf itself but the jvm cf runs on.

So tell us what cf version and update you're running, and what jvm version that cf is using. You can find both in the cf admin "settings summary" page.

Some will jump to suggesting you import the cert into the jvm that Cf uses.

3) My experience is that instead all that's needed may be to update that jvm that Cf is using.

And I don't mean go from one major jvm version to another, but one point release of that current jvm to a more recent one.

I discuss all this more in a blog post here:

https://coldfusion.adobe.com/2019/06/error-calling-cf-via-https-solved-updating-jvm/

Let us know if this helps, or if it does not, first for getting a cfhttp test to work, then for your scheduled tasks. 

[DNomer]

You are quite the resource for these thorny CF internal issues, Charlie. Some into:

CF version: 2023.0.05.330608

Tomcat version: 9.0.78.0

Windows 10.0 on server

JVM Java version: 17.0.6  

JVM version: 17.0.6+9-LTS-190  by Oracle

I have never done a jvm update before, but I can follow cookbook directions.

[DNomer]

Somehow this forum lost my last post. Here is the test results as you requested.

[Ravi Shankar266617537wjh]

Hello Dnomer,

Please let us know if you have imported the certificate of the Scheduled task URL into ColdFusion cacerts, if not please import the certificate and check if the issue still occurs, 

You need to import the certificate into the JDK to which ColdFusion is pointed to , you can find it in jvm.config file

Thanks & Regards

Ravi

[Charlie Arehart]

Well, I'll say again, there may be no "need" (as you put it) to import a cert but rather merely to update the JVM. See more on my first reply above. 

BTW, when @DNomer confirmed a couple of days ago that they were indeed on an old jvm version, I'd offered a bit more in reply, though somehow it's appearing BELOW Ravi's response here.

Please do keep us posted on what attempts you made and where things stand. We're all just trying to help. 

[DNomer]

I find that tough to comprehend. If the site was working before on the current version, there should be no problem installing a new cert and it should still work. The same crew who installed this webserver in Azure should be able to duplicate what they did before. The original installation last fall had all of the same characteristics as it does now, except the different rewrite engine.

[DNomer]

An experienced tech got the CACerts file for the JVM updated with the new certs, and it did not fix the problem. Exact same error as shown above. This server was set up last fall with the same Sectigo certs, and worked fine all of these months. Sectigo gives you a pile of certs, more than one intermediate cert. The guy who installed the certs in Windows and in IIS is a rookie. The server does SSL fine, though, and IIS indicates that the cert chain is good. The only thing different between now and last fall is that I had to install the Microsoft rewrite engine, because Helicon went out of business. The Helicon one was the Apache-type rules. I have seen other posts about this that indicate that you have to put the intermediate cert and the main cert in the cacerts file. I believe that was done.

[Charlie Arehart]

@DNomer, two things:

1) Have you yet attended to what we spoke of two days ago here, about updating the jvm that CF uses? That may be all that's needed. See also what I'd written that day, appearing somehow BELOW this back-and-forth with Ravi, here.

2) In any case, if you may still want to try the cert import approach (which you say "a experienced tech" tried), a key point is that the cacerts you import into MUST be the one (within the lib/security folder) for the jvm which cf is currently using. Some people (like that tech,. perhaps) follow cookie cutter steps that fail to clarify this.

For instance, if someone HAS changed cf to use a jvm located outside of the cf jre folder, it does no good then to import into the [cf]/jre/lib/security/cacerts; yet some docs (even from Adobe) specifically tell you to do that, without asking that you confirm what jvm cf is using.

You can find that specified in the cf admin, either on the first field of the Java and jvm page, or (as I'd shared originally) on the admin "settings summary" or "system into" pages. When you reported the version info above, you didn't happen to mention WHERE it said the cf jvm was located.

As always, just trying to help. And again you need not slog through back and forth here. We could fix this is as little as 15 mins in a remote screenshare consulting session (see below for more). But I appreciate some want to or have to "go it alone". 

[DNomer]

For our ColdFusion, as reported in CFAdmin from the 'i' icon, under JVM Details, the 'Java Home' is: C:\ColdFusion2023\jre  . We use a tech firm who has quite a bit of experience in ColdFusion to handle difficult things, like setting up this server last fall, and troubleshooting this problem. There was a problem with the password on the keystore in cacerts at first, but they did some stuff there yesterday, and today at about 10:41 AM server time one of the techs was able to list cacerts, then saw an error, then updated it and felt that it was good. He spoke with me about it. Even though some cfhttp calls that were broken now work, the ones for the scheduled tasks still do not work. This is true for every task, and these tasks have not been changed. I see that the cacerts file in C:\ColdFusion2023\jre\lib\security shows updated at that time. So I think he got the right file. Whether everything is just right, I don't know.

Re: updating, I tried that, using the updater in CFAdmin a few nights ago. I updated to update 15. It broke everything. It broke ColdFusion and even CFAdmin now gave a 500 error, even though while doing the update, it indicated that it successfully stopped CF and then restarted it, and reported success, as if everything is perfect. One of these techs had to manually roll back that failed update, which put us back to where we were, on update 5, the one originally installed. He then tried updating one version at a time, and only the first one worked. Beyond that, CF broke, even though CFAdmin did not break again. So he reverted back to update 5. The update process is not very good.

[carl type3 · Accepted solution]

Hello,

JDK 17.0.16 has the new Sectigo root CA names. Refer:
https://bugs.openjdk.org/browse/JDK-8359170

Follow posted suggestion to install JDK 17.0.16 and use that with CF instead of the one in CF\jre then your schedule task problem will solve.

Regards, Carl.

[DNomer]

You are correct, sir. Tim at IPM did just what you said, in a small amount of time, and it fixed the problem. Scheduled tasks now run. Some of the other techs at IPM are seasoned techs, but Tim goes way, way back in ColdFusion. Anyway, I want to thank everyone for their contributions. Tech people on forums are often quite helpful folks.

[Charlie Arehart]

Just to clarify, this is what I was suggesting from the first reply. Glad you feel things are resolved.

For readers who find this thread in the future, the next two posts of mine which follow are actually from earlier in time. The threaded nature of the forum sometimes makes replies like that seem confusing. 

[DNomer]

Unfortunately, even though our scheduled tasks now work fine, and nearly all of our cfhttp calls work, there is one cfhttp call which was working BEFORE updating the jvm, which has the new Sectigo root cert, and now fails. My suspicion is that this interaction, which is with an old node server, has somehow upgraded to a new version of tls or something, and the other server cannot handle it, and so that now broke. I wonder what some of you think, or if you want some details.

[Charlie Arehart]

Well, that was in fact a problem a few years ago, when updates to Java 8 and 11 stopped supporting an old tls version, so that calls form cf (from Java) to such servers now failed. 

But your on java 17, which came out after that. 

Even so, tell us first what you see as the error when you dump the cfhttp struct after the cfhttp. It would be preferable to offer the text of the error (not just a screenshot as you did before), in case we may need to look it up.

In the meantime, you may want to look at what I shared in my post back then, as you may see some things to consider related generally to your current issue. 

[DNomer]

We have been working to get off of that old node server, so I used this opportunity to code that function in CF. Those node calls were made by a group of coders some years ago. Good point about 'the text of the error'. I appreciate the large range of expertise that is available here.

[Charlie Arehart]

@DNomer,. I did not suggest that you update cf. I suggested that you update the JVM, which Carl in his reply has now confirmed should address issues with sectigo certs.

As for getting that and cf updated correctly (if you want it), again you can work with your folks and engage in back and forth here, but we could have all this solved in as little as 15 minutes, as I've offered a couple of times.

I get it that some feel they "can't hire help", but it's been a couple of days that you've been troubled and it need not be so. I'd think someone would see the economic value in that. But otherwise, hope we've steered you the right direction. 

[Charlie Arehart]

Yep, this stuff is what I work with helping people about daily..

And yep, both those update levels are quite dated, with Cf2023 u5 being from Oct 2023, and jvm 17. 0. 6 being even older from Jan 2023..

As for updating the jvm cf uses, there are cookbooks. Indeed, the last link in my post there points to a resource I created on the key points to beware, which includes both a link to a rather brief post on doing it from Pete Freitag. Then I point to a preso I did (pdf and recording) covering more details, which some will need but many may get by with Pete's post.

As I point out often, the task can be done in minutes for one who knows what they're doing. But for those unfamiliar, any of various mistakes or missteps could leave cf not starting. If this is prod, you'd want to be very careful.

Of course I can help directly, via remote screenshare, getting it done a (nd explaining the key points) in as little as 15 mins, which is also my smallest billable increment. You won't pay for time you don't find to be valuable. More on my rates, approach, satisfaction guarantee, and online calendar at carehart.org/consulting.

Whichever way you go, hope it works out. 

[BKBK]

To summarize (assuming I see it clearly):

• the solution was to update the Java Development Kit that ColdFusion 2023 runs on from version 17.0.6  to  version 17.0.16 .

[DNomer]

That was the final fix, but we also had to put the new certs in the cacerts file for the jvm prior to that, using that keytool.

[BKBK]

Hi @DNomer , thanks for clarifying.