Coldfusion 2023 start-up on SuSE-Linux takes at least 9 minutes

[michaels14629018]

We have installed CF2023 (30-days-trial, Update 4, version 2023.0.04.330500) on two different VMs with openSUSE-Leap-15.5-1.

One Server is using the build-in Tomcat (port 8500), the second one Apache (port 80).

Both servers are getting stuck for at least 9 minutes at the same point on every restart:

coldfusion-out.log
Aug 28, 2023 19:31:26 PM Information [main] - Starting logging...
Aug 28, 2023 19:31:26 PM Information [main] - Starting license...
Aug 28, 2023 19:31:26 PM Information [main] - Evaluation Edition enabled
*** Minimum pause 9 minutes (on every restart) ***
Aug 28, 2023 19:40:04 PM Information [main] - Starting crypto...

We tried it with and without this JVM-paramters: 

-Djava.security.egd=file:/dev/urandom

-Djava.security.egd=file:/dev/random

(no difference)

Both servers use the build-in JRE (CF standard), version 17.0.6+9-LTS-190

CF2018 ist starting up immediately on those machines.

On Windows Server CF2023 also starts-up immediately.

Any help would be highly appreciated.

[Charlie Arehart]

My money would be on the activation process, Michael, and there are a couple of simple things to test/confirm and perhaps to resolve. (I do realize the log portion you shared doesn't indicate this. As you may know, Adobe implemented activation starting with CF2021, so your cf2018 would not perform it. That's my reasoning for suspecting this.)

So first, note that there's a doc from Adobe about the mechanism here: 

https://helpx.adobe.com/coldfusion/using/coldfusion-licensing-activation.html

Second, as it indicates, the urls cf calls out to are:

https://coldfusion.adobe.io

https://cfactivation.adobe.com

If you do a curl or wget of those at the command line of the servers, what happens? If either hangs or is blocked, that's the problem. Some servers are setup to block outbound calls, or to limit them via a whitelist. 

So one solution may be to open a firewall hole (but beware also that either of these may do a redirect to another domain). 

Another solution is that your firewall may offer a proxy option, allowing requests to any domain to work. There is a way to configure that in the cf admin for this activation: note that it's done using the new "package manager" page instead. (And since cf2021, one can manage admin settings from the CLI using a new cfsetup command.) 

Finally, note also that problems can also be caused by anti-virus solutions. If you have a way to disable that (and/or the firewall) during cf startup to find out, it would help confirm things.

Let us know what you may find. If this isn't the right track, there are still other diagnostics to help get to the bottom of the hangup. 

[michaels14629018]

Dear Charlie,

thank you very much for your reply!

I checked both URLs with curl: Both are reachable, although at https://coldfusion.adobe.io I get a 404 error.

The local firewalls are deactivated and there are no anti-virus-tools installed.

What I just realized is, that everytime I click the "Licensing and Activation"-Link in the CF-Admin it takes also at least 10 minutes before the page opens while the post request to http://10.0.0.105:8500/CFIDE/administrator/activation/activation.cfc?method=isPendingOfflineRequest is never getting a response. When clicking, the network-trace shows a connection-trial to the local address 169.254.169.254 only.

We have a windows machine in the same network. When I click the link there, it takes about 3 seconds for the page to open and for the post request to get a response. 

Michael

[Charlie Arehart]

Well there you go: the hangup in the "Licensing and Activation" page confirms the root cause. As for the 404, that's an ok thing. It wants a specific call, but at least that confirms cf could reach it. 

The call to that specific ip may be the real hangup for you. As you may know, it's a special address used for local routing. You can find more on it at this article. 

And perhaps someone else here may offer a ready solution for what's needed next for you. Until then, I have more diagnostics ton consider. 

First, what happens if you ping or curl that ip on the server's command line? Does that hang up? 

And note the article I shared concludes with mention of how one might want to limit calls to it via iptables. That raises the interesting question of whether perhaps it is for some reason blocked. Can you check for any iptables logging, or just check your iptables rules for any related to that ip address or pattern? 

Indeed, that article indicates how they'd recommend blocking calls to that ip for processes running as other than root. That raises an interesting thought: when you installed cf, you probably told it to run as other than root. You could try changing it to run as other than root. Does that then change this hangup? 

Finally, if somehow the above prove fruitless or challenging, we can consider more diagnostics from a cf perspective. Can can you look at cf's http.log to see if it's tracking anything related to this activation hangup? Note that it tracks the start and end of requests. I'm not saying all calls out of cf get logged there (some, like these, may be too low-level), but it's worth a shot. 

Otherwise both the cf pmt (new since cf2018, and optionally installed) and fusionreactor (commercial, with free trial) can also track calls out of cf, which might also help here. 

These are among the things I alluded to at the end of my first reply. I realize it may be a bit much for some to stomach, if they just want the problem solved (without having to figure out such things). To that, I'll note in conclusion that I can help directly, remotely via screenshare. More on that (rates, approach, satisfaction guarantee, online calendar, and more) at carehart.org/consulting. 

But I do hope you may resolve this with what I or others may share, or you may find on your own. Please do let us know how it goes. It seems an important challenge that others might experience. 

[michaels14629018 · Accepted solution]

The ping to 169.254.169.254 made the router at our provider to answer with "network not reachable".

The curl to 169.254.169.254 created exactly the same hang-up-behaviour.

Rejecting this IP-address at the local firewall did the trick 🙂

iptables -A OUTPUT --destination 169.254.169.254 --jump REJECT

Thanks alot for your time and support! Have a great day! 

[Charlie Arehart]

Great to hear, and glad to have helped.

Let's hope this helps others who may find this, and perhaps that Adobe may be able to take away something from this, if some change in config or docs (or functionality) might be warranted.