TomCat 503 Service Unavailable error

[#1]

[Mon Feb 08 12:51:18 2021] [5516:14492] [info] jk_open_socket::jk_connect.c (816): connect to ::1:8020 failed (errno=61)
[Mon Feb 08 12:51:18 2021] [5516:14492] [info] ajp_connect_to_endpoint::jk_ajp_common.c

I am getting repeatedly getting these errors when runing a cfm file from within coldfusion builder 2018.

the coldfusion service itself is the latest 2021.

I am able to view the coldfusion admin page via port 8500.

Apache server (24) is running ports 80, 443 nothing else

netstat shows 0.0.0.0:8500 as well as [::]:8500

port shows up attached to coldfusion

anyone have an idea of where the error is?

These are the errors from the mod_jk.log

(1158): (cfusion) Failed opening socket to (::1:8020) (errno=61)
[Mon Feb 08 12:51:18 2021] [5516:14492] [error] ajp_send_request::jk_ajp_common.c (1829): (cfusion) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=61)
[Mon Feb 08 12:51:18 2021] [5516:14492] [info] ajp_service::jk_ajp_common.c (3000): (cfusion) sending request to tomcat failed (recoverable), because of error during request sending (attempt=2)
[Mon Feb 08 12:51:18 2021] [5516:14492] [error] ajp_service::jk_ajp_common.c (3021): (cfusion) connecting to tomcat failed (rc=-3, errors=22, client_errors=0).

Mel

[Merit LLC 2]

I'm experiencing the exact same issue with a fresh ColdFusion 2021 install.

[Charlie Arehart · Accepted solution]

Put address="::1" on the connector line for the ajp connector in cfusion/runtime/conf/server.xml, then restart cf. The value is suggested by the error msg, and the details are discussed in the technotes of the updates you have applied, above update 8, which point to the technote for update 8.

And yes, the same issue applies to cf2021. See the technote for CF2018 update for the details, or a blog post I did on the matter in March 2020 when the update came out. 

https://coldfusion.adobe.com/2020/03/three-reasons-sites-may-break-fix-applying-mar-2020-update-cf20...

Let us know how it goes. 

[Merit LLC 2]

I'm still encountering the same error. 



There was no "address" attribute in the server.xml file, so I added one like this:
vi /opt/ColdFusion2021/cfusion/runtime/conf/server.xml

Find: 

<Connector protocol="HTTP/1.1" port="8500" redirectPort="8453" connectionTimeout="20000"/>
Replace: 

<Connector address="::1" protocol="HTTP/1.1" port="8500" redirectPort="8453" connectionTimeout="20000"/>


I then restarted ColdFusion and attempted to load a .cfm file.



[Mon Feb 15 09:23:26 2021] [278549:140692198229760] [info] jk_open_socket::jk_connect.c (816): connect to ::1:8020 failed (errno=111)
[Mon Feb 15 09:23:26 2021] [278549:140692198229760] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1158): (cfusion) Failed opening socket to (::1:8020) (errno=111)
[Mon Feb 15 09:23:26 2021] [278549:140692198229760] [error] ajp_send_request::jk_ajp_common.c (1829): (cfusion) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=111)
[Mon Feb 15 09:23:26 2021] [278549:140692198229760] [info] ajp_service::jk_ajp_common.c (3000): (cfusion) sending request to tomcat failed (recoverable), because of error during request sending (attempt=1)


 



ping localhost
PING localhost(localhost (::1)) 56 data bytes
64 bytes from localhost (::1): icmp_seq=1 ttl=64 time=0.049 ms


 

[BKBK]

What happens when you modify the AJP connector instead. When you do, ensure that the settings in worker.properties match those of the connector. In particular, use the secret property. This can be any string you choose, but use a UUID for security.

For example:

In worker.properties

worker.list=yourWorkerName

worker.yourWorkerName.type=ajp13
worker.yourWorkerName.host=localhost
worker.yourWorkerName.secret=7177aa93-abba-43f4-80dc-c974a01e76a3
...etc

In server.xml

<!-- AJP connector -->
<Connector port="8020" connectionTimeout="60000" redirectPort="8453" protocol="AJP/1.3" tomcatAuthentication="false" maxThreads="500" secret="7177aa93-abba-43f4-80dc-c974a01e76a3"/>

[Merit LLC 2]

There is no worker.properties files, but I found 

/opt/ColdFusion2021/config/wsconfig/backup/1.2/workers.properties

heartbeat_interval=30
heartbeat_limit=90

#Start of workers.properties associated with 'cfusion'
worker.list=cfusion

worker.cfusion.type=ajp13
worker.cfusion.host=localhost
worker.cfusion.port=8020
worker.cfusion.heartbeat_servlet_path=/__cf_connector_heartbeat__
worker.cfusion.connection_pool_timeout=60
worker.cfusion.monitoringsecret=7df2bbdf-2a71-4bbc-8584-960f82223b6d
worker.cfusion.secret=20310a91-a232-4606-a347-39b7bf8544df
#End of workers.properties associated with 'cfusion'

I then updated the server.xml from:

<Connector protocol="AJP/1.3" port="8020" redirectPort="8453" secret="20310a91-a232-4606-a347-39b7bf8544df" maxThreads="500" connectionTimeout="60000" tomcatAuthentication="false"/>

To:

<Connector port="8020" connectionTimeout="60000" redirectPort="8453" protocol="AJP/1.3" tomcatAuthentication="false" maxThreads="500" secret="20310a91-a232-4606-a347-39b7bf8544df"/>

After stopping and restarting ColdFusion, I'm still getting the same error. 

[Merit LLC 2]

These are the connectors in /opt/ColdFusion2021/cfusion/runtime/conf/server.xml
<Connector address="::1" protocol="HTTP/1.1" port="8500" redirectPort="8453" connectionTimeout="20000"/>
<Connector port="8020" connectionTimeout="60000" redirectPort="8453" protocol="AJP/1.3" tomcatAuthentication="false" maxThreads="500" secret="20310a91-a232-4606-a347-39b7bf8544df"/> <Connector address="::1" protocol="HTTP/1.1" port="8500" redirectPort="8453" connectionTimeout="20000"/> <Connector port="8020" connectionTimeout="60000" redirectPort="8453" protocol="AJP/1.3" tomcatAuthentication="false" maxThreads="500" secret="20310a91-a232-4606-a347-39b7bf8544df"/>
The secret property in /etc/httpd/conf/workers.properties matches the connector's above.

[Jeff Coughlin]

Charlie,

You just saved me from hours of aggravation. Thank you my friend.

Jeff

[Charlie Arehart]

Wonderful to hear--and all the more that I could repay you for much great knowledge you'd shared over the years! 🙂 

[Charlie Arehart]

Mel, I had said in my first reply that you should add the address="::1" to the ajp connector in the server.xml. You added it instead to the http connector.

BKBK tried to steer you there, but then the discussion veered off into secrets (when you were likely so close to resolution). Making manual changes about that should not have been necessary. Then you said you have no workers.properties, which would be its own issue and suggests you've really perhaps dug yourself into a hole with your spinning tires. 

And had you read the blog post I pointed to? Or the technote for the updates (in CF2018 or 2016,which introduced this matter, carried forward into cf2021)? It is lamentable that one would have to deal with these things, but we must. It's a tomcat security issue that Adobe has inherited. I didn't elaborate further as you were again really close that day, needing only that one change I suspect. 

And if you feel lost and "just need it fixed", I could have it all working in 15 mins or less, via shared desktop consulting session. More via the consulting link on my site shown in my signature. 

[BKBK]

Sorry about that, Charlie. My suggestion can indeed be misunderstood as an alternative to yours. It is not.

@Merit LLC 2 , I assumed you would also take on board Charlie's suggestion about address. 

[Merit LLC 2 · Accepted solution]

Thank you very much. This is working now. I thought you were talking about the first connector that already had the address set to ::1. I wonder if CF2021 will be updated to include this address on other connectors to avoid this error. 

[Merit LLC 2]

Also, I had searched the entire server for 'worker.properties' like it was named in older CF versions. I later found that it was 'workers.properties'. 

[Charlie Arehart]

Glad you are working now. I did say the ajp connector, not the "first" one. But again, the docs I pointed to made things still more clear.

And no, the other file has been called workers.properties since cf10 in 2012, and in tomcat (where it comes from) before that.

And yep, BKBK, I understood you were proposing both things to be considered.

Finally, Mel, would you want to mark my first response last week as the "answer", to help future readers? 

[BKBK]

@Merit LLC 2 : There is no worker.properties files...I had searched the entire server for 'worker.properties' 

Writing it as worker.properties was just a typo.

[Charlie Arehart]

FWIW, Mel had written that, "There is no worker.properties files, but I found

/opt/ColdFusion2021/config/wsconfig/backup/1.2/workers.properties".

That's why I assumed they were looking more widely than just for worker.properties, when I expressed surprise there was no other one.

But again, the problem is resolved by their having addef the address as I proposed in my first comment. Again, Mel, would you please consider marking that as the answer to help future readers? 

[Pierre Chaillet]

Thanks, I had also an error 503 after upgrading from CF2018 to CF2021, I added address="::1" in the second connector  <Connector packetSize="65535" protocol="AJP/1.3" address="::1" port="8020" redirectPort="8453"...

and my cfm files were back. A CF upgrade is always an adventure...