• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Strange JRE Update problem

Community Beginner ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

Hello Coldfusion Community!  It's nice to meet you all.  I'm seeking out some help performing a JRE update on our CF Server.  I am troubleshooting an high-priority incident with our software and hope I can some guideance toward a solution.

 

Problem:

  1. Yesterday morning, we noticed that emails were not being sent from the site, getting stuck in the "Undelivered Mail" list.
  2. Upon reviewing mail.log, I found this message each time CF attempted to send an email.
    javax.mail.MessagingException: Could not connect to SMTP host: secure.emailsrvr.com, port: 465; nested exception is: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
  3. Researching this error led me to believe that it's possible our email provider or their CA had a security configuration change that doesn't work with this old server.   (Coldfusion 11, Update 19 -- JRE 1.7.0_55)  -- They are refusing to confirm or deny whether there's been a change.
  4. Since the JRE we're running on connects over deprecated SSL/TLS protocols, we decided to proceed with a JRE Upgrade.  I downloaded JRE 8U401 from Adobe's download section, installed this on our server, and then updated JVM.config to point to the new file.  The "Coldfusion 11 Application Server" service refused to restart so we rolled back to the older JRE until more troubleshooting could be done.

    Here's the lines I updated in JVM.config:
    #java.home=C:\\ColdFusion11\\jre
    java.home=C:\\Program Files\\Java\\jre-1.8
  5. I performed some troubleshooting and learned that the vcruntime140.dll file was missing from the CF directory, so I dropped that in and got Coldfusion to start, but it continued showing the 503 under maintenance page and CF Admin would not load..  We recieved the following error message.

    Feb 22, 2024 4:26:55 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
    INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\ColdFusion11\cfusion\lib;C:\ColdFusion11\cfusion\jintegra\bin;C:\ColdFusion11\cfusion\jintegra\bin\international;C:\ColdFusion11\cfusion\lib\oosdk\classes\win
    java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.adobe.coldfusion.bootstrap.Bootstrap.init(Bootstrap.java:90)
    at com.adobe.coldfusion.bootstrap.Bootstrap.main(Bootstrap.java:165)
    Caused by: org.apache.catalina.LifecycleException: Failed to initialize component [StandardServer[8009]]
    at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
    at com.adobe.coldfusion.launcher.Launcher.run(Launcher.java:987)
    ... 6 more
    Caused by: java.lang.UnsatisfiedLinkError: C:\ColdFusion11\jdk1.8.0_391\jre\bin\sunmscapi.dll: Can't find dependent libraries
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1937)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1841)
    at java.lang.Runtime.loadLibrary0(Runtime.java:843)
    at java.lang.System.loadLibrary(System.java:1134)
    at sun.security.mscapi.SunMSCAPI$1.run(SunMSCAPI.java:52)
    at sun.security.mscapi.SunMSCAPI$1.run(SunMSCAPI.java:50)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.mscapi.SunMSCAPI.<clinit>(SunMSCAPI.java:50)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
    at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
    at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
    at sun.security.jca.ProviderList.loadAll(ProviderList.java:282)
    at sun.security.jca.ProviderList.removeInvalid(ProviderList.java:299)
    at sun.security.jca.Providers.getFullProviderList(Providers.java:174)
    at java.security.Security.getProviders(Security.java:410)
    at org.apache.catalina.core.JreMemoryLeakPreventionListener.lifecycleEvent(JreMemoryLeakPreventionListener.java:413)
    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
    at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
    at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:388)
    at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:101)
    ... 7 more
  6. Then, I attempted using the JDK of the same version and also tried version 8U391 -- still the same error.  We rolled back as we couldn't keep the system down any longer.
  7. Finally, as a sanity check, I have a newer version of CF running in a dev environment - I copied the email generating code and it's settings / credentials from the old one to the newer one and they worked as expected. 

 

I know we're long overdue for a full CF Update / Server rebuild, but I'm looking for a way to get these emails sending again until we are to that point.  Any help would be appreciated.

Thanks!

PL

Views

476

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Correct answer

Community Expert , Feb 23, 2024 Feb 23, 2024

Phil, I may have a solution for you. Sorry that it won't be a short note, but the suggested solution should be quick for you to try. And yes, of course getting off cf11 would be best for so many reasons (including potentially devastating security vulns), to get you to either currently supported version, cf2023 or cf2021.

 

Until then, in your crippled state, let's see if we can get you going where you are.  And you're right that updating Java should be the solution to your original mail deliver

...

Votes

Translate

Translate
Community Expert ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

Suggestion:

  • Download and install the latest Java 7, which is Java SE Development Kit 7u80. Then use its path as the value of java.home in jvm.config. For reference, see ColdFusion 11 Support Matrix.

After you do, use the keytool tool to import the certificates.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

While updating from that java 7u55 to 7u80 might suffice to solve Phil's SMTP problem, I'll note that 7u55 was from 2014 while 7u80 is only from 2015. That may not be recent enough to overcome that smtp problem.

 

FWIW, the 241 that I suggest below is from 2020, so again while not as recent as the 401 he tried (from 2024), it is 6 years more recent than his failing 7u55.

 

Just clarifying to you both and other readers. (And as you can tell, I kind of love this stuff. It's like solving a challenging puzzle.) Hope one or the other solution gets Phil going.


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 24, 2024 Feb 24, 2024

Copy link to clipboard

Copied

LATEST

 

Deleted. Please ignore

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

Phil, I may have a solution for you. Sorry that it won't be a short note, but the suggested solution should be quick for you to try. And yes, of course getting off cf11 would be best for so many reasons (including potentially devastating security vulns), to get you to either currently supported version, cf2023 or cf2021.

 

Until then, in your crippled state, let's see if we can get you going where you are.  And you're right that updating Java should be the solution to your original mail deliver problem. Perhaps you've seen that I cover the topic in a post here.

 

1) But I suspect your problem is you went "too recent" in the java 8 updates, given your current OS. Let me explain: that subsequent error of "Can't find dependent libraries" for sunmscapi.dll seems to have started with 1.8.u251, especially if you are on an older Windows machine. (You didn't say what version you're on, but might it be as old as Windows 2003?) While updating your OS could be one solution, an easier solution may be just to not update the JVM "so far" as the 8u401 you went to.

 

As this problem started in 8u251, try 241 (one shy of that), and see if that gets you past the SMTP problem without the sunmscapi.dll problem. Granted, it's almost as bad for your JVM to be so old as that, but you're trying to get the ship back to land, not change out the engine while on a stormy sea. And no, you can't get that old 241 jvm version from Adobe, but you can from Oracle's jdk archive (the download itself will require a login, and an account is free).

 

And if anyone following might wonder if Java 8 is supported at all by cf11, it was, after its update 3 (and Phil said he is on update 19). For more, see my table mapping CF versions to supported Java versions.

 

Let's see if that gets you past whatever jvm limitation prevented your connecting to your smtp server. Before concluding, let me offer two more tips for you.

 

2) As for checking if the mail delivery "now works", as you may know you can try just copying a file from cf's mail/undelivr folder to the spool.  Watch for it to disappear (per your cf admin mail spool interval). To know if it worked (assuming you can't see the mail delivered), you can certainly look in the undelivr folder, but beware that if it fails again, the mail file will be put there with the same name and date it had before. Look instead to the cf mail.log to see if an error occurred, or better look to the cf mailsent log of enabled. 

 

3) Finally, as for your updating from Java 7 to 8, don't stop at "the mail delivery works", as there may be problems that would soon arise. Again, as you may know, in updating major Java versions underlying cf, it's also important to clear out the compiled Java files that CF created from your CFML over the years from your cfml. Those are in the cfusion/wwwroot/WEB-INF as cfclasses (NOT classes). I discuss this more in a post on solving common problems when updating the JVM. See specifically my point on clearing cfclasses there. (Then again, rather than DELETE those files of that cfclasses folder, I'd recommend you just rename it--while CF is down--so that it creates a new one. You may need to use that cfclasses folder to look for evidence of a vuln as I discuss in a very long post last March.)

 

Related to the last point, you have one more step when making a jump in major Java versions: see the next section of my post there, on removing also the related "stubs", "rest-skeletons", and "cfc-skeletons" files, related to other aspects of compilation of things in your past use of CF with Java 7.

 

Again, sorry for this long missive. It's clear you'd done your own homework (and perhaps you'd already seen some of these resources), but I hope I've connected the dots for you and given you enough to resolve things on your own. If not, while you could certainly ask follow-ups here, I'll just say that I can often help folks in your situation solve the problem quickly via direct remote assistance, discussed more on my consulting page.


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

Charlie, thank you for your thoughtful and detailed  reply.  I also want to say a big thank you for the info you have shared online as I've been reviewing your site and watching a few of your webinars and this has all been extremely helpful while I was coming on board in my current role learning feeding/caring for, and eventually modernizing our CF environment.  I appreciate you detailing the extra steps I should take after doing the upgrade.

I'm pleased to hear there's still hope for JRE 8 so we can be certain the TLS functionality within would still be usable for at least some time. 

This one's running on Windows Server 2008 (Which I know will need an update soon as well)  I believe it's the standard edition of coldfusion as well, if that matters.  I'll attempt the version you suggested shortly, after which we'll see how it goes.

It was also recommended by our mail provider to extract their server's certificate and import it into java, similarly the previous reply  That's going to be the contingency plan at this point.  Though I'd prefer not to risk a requirement to do that on a regular basis.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

Charlie, THANK YOU!  Following up to let you know that going to this version of the JDK got us running again. 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 23, 2024 Feb 23, 2024

Copy link to clipboard

Copied

Very good to hear.

 

And thanks very much for the kind regards, Phil. Again, I love doing this stuff and helping folks. Such comments help keep me going (trust me, they are few and far between...in this "take, take, take" world.)

And I had started writing that paragraph above and the below, before your last reply here. I got held up submitting it because I got pulled into client calls. While some of it's moot (you didn't need to import certs), I'll go ahead and send it along here, for the sake of other readers.

 

First, ok being on 2008. While the one link I pointed to indicated the sunmscapi.dll happening as of 8u251 on Win2k3, that could be an incomplete discussion. I wouldn't be surprised it impacted one on 2k8 as well. 

 

And yes, while many will suggest "import a cert" as "the solution" for this sort of issue (and it wasn't the issue for you), you will indeed find in my post (shared above) on solving problems of CF calling out via https that I make the case for that often being no longer necessary, if instead one would just update their JVM--which which beings other benefits like security updates, bug fixes, etc. But as this post shows, even that's often a balancing act. My point on importing certs is as much that sometimes people are carrying them forward into each new JVM when in fact they may find they no longer need to. 🙂 

 

And if one DOES need to import certs, be careful not to blindly follow most steps offered online (even from Adobe), which would have you import them into the jre/lib/security/cacerts in the CF folder. That is NOT the place to put them, if you change CF to point to another JVM; instead, one should put the cert into the cacerts within THAT jvm. I discuss that more in some of those other posts I linked to.

 

While we're on the topic of importing certs, I'll note that there are other tools to help with that beyond java's commandline keytool. Look at the free, multiplatform tool https://keystore-explorer.org/, or the certman tool (a free CF Admin extension, of which there are various versions on github--and they do work with CF11 and above, despite what some say or ask, at least for seeing what certs are there and for adding them. But there's a bug in VIEWING, DELETING, or DOWNLOADING them--and I have a fix for that that I will be pushing soon.)


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Resources
Documentation