Highlighted

Connector errors after ColdFusion 2018 Update 5: Bad gateway, Tomcat is down or refused connection

Adobe Community Professional ,
Oct 23, 2019

Copy link to clipboard

Copied

We have been getting frequent errors after installing Update 5 of ColdFusion 2018. The site serves web pages, but pressing F5 (to refresh the page) produces an Apache error page: 

bad_gateway.png

The isapi_redirect.log file entry corresponding to the failed request is:

 

[Wed Oct 23 17:18:44.903 2019] [13452:8640] [info] ajp_connection_tcp_get_message::jk_ajp_common.c (1425): (t1stdrdrci) can't receive the response header message from tomcat, tomcat (127.0.0.1:8030) has forced a connection close for socket 1464
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] ajp_get_reply::jk_ajp_common.c (2402): (t1stdrdrci) Tomcat is down or refused connection. No response has been sent to the client (yet)
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] ajp_service::jk_ajp_common.c (2975): (t1stdrdrci) sending request to tomcat failed (unrecoverable), (attempt=1)
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] HttpExtensionProc::jk_isapi_plugin.c (2739): service() failed with http error 502

 

We applied the patched isapi_redirect.dll that Adobe recently released. In vain.

I then reported a bug: https://tracker.adobe.com/#/view/CF-4205448 (Please follow this link and vote to have this fixed - Thnx!)

 

Our setup:

ColdFusion 2018 Enterprise Update 5

64 Bit Windows Server 2016 + IIS 

 

If you found a solution to this problem, could you please share it with us?

 

TOPICS
Connector

Views

392

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Connector errors after ColdFusion 2018 Update 5: Bad gateway, Tomcat is down or refused connection

Adobe Community Professional ,
Oct 23, 2019

Copy link to clipboard

Copied

We have been getting frequent errors after installing Update 5 of ColdFusion 2018. The site serves web pages, but pressing F5 (to refresh the page) produces an Apache error page: 

bad_gateway.png

The isapi_redirect.log file entry corresponding to the failed request is:

 

[Wed Oct 23 17:18:44.903 2019] [13452:8640] [info] ajp_connection_tcp_get_message::jk_ajp_common.c (1425): (t1stdrdrci) can't receive the response header message from tomcat, tomcat (127.0.0.1:8030) has forced a connection close for socket 1464
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] ajp_get_reply::jk_ajp_common.c (2402): (t1stdrdrci) Tomcat is down or refused connection. No response has been sent to the client (yet)
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] ajp_service::jk_ajp_common.c (2975): (t1stdrdrci) sending request to tomcat failed (unrecoverable), (attempt=1)
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] HttpExtensionProc::jk_isapi_plugin.c (2739): service() failed with http error 502

 

We applied the patched isapi_redirect.dll that Adobe recently released. In vain.

I then reported a bug: https://tracker.adobe.com/#/view/CF-4205448 (Please follow this link and vote to have this fixed - Thnx!)

 

Our setup:

ColdFusion 2018 Enterprise Update 5

64 Bit Windows Server 2016 + IIS 

 

If you found a solution to this problem, could you please share it with us?

 

TOPICS
Connector

Views

393

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Oct 23, 2019 0
Adobe Community Professional ,
Oct 23, 2019

Copy link to clipboard

Copied

BKBK, you say this is since applying update 5. Did you first try to revert to whatever update you were on before, to prove is this is indeed "the cause"? And if you may not want to do that, have you tried using whatever isapi_redirect.dll you had from BEFORE the update 5? Sadly, the CF wsconfig "update" feature does not save a backup (I have a feature request for that).

 

And in doing the manual update to the dll, did you restart IIS after that? 

 

Was it failing the same way after update 5 but before the new dll? Had you tried to upate the connector AS PER THE UPDATED CONNECTOR THAT CAME with update 5? Or did you perhaps skip that step? What update were you on before?

 

Finally, it's possible that your problem is NOT about the update at all. I assume the site was working fine before the update, right? Did you do ANYTHING else after applying the CF update? Did you update the connector (with the wsconfig, or not)? Did you change anything in IIS? Did you perhaps "remove" and re-add a connector, instead of using the "update" feature of the wsconfig tool?

 

Answers to any of the above could provide a clue as to what is amiss. I appreciate folks often just wish to "get the solution", but as you are finding this is a more challenging problems for you, and it may take a bit more digging to resolve. (I will assume you are not interested in a consulting session to try to address this, if the above does not help. But I will say to other readers who may find this that I can often help solve such things, and perhaps in just minutes when I can actually see what's amiss on your server.)

 

Hope something there helps.

/Charlie (server troubleshooter, carehart.org)

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 23, 2019 0
Adobe Community Professional ,
Oct 23, 2019

Copy link to clipboard

Copied

Hi Charlie,

Thanks for your feedback.

 

The issue does not occur when we revert to our previous version, ColdFusion 2018 Update 4.

 

We changed nothing in IIS. In doing the manual dll update, we did restart IIS and recycle the application pool.

 

We did indeed update as per the updated connector that came with update 5. In fact, we later tried 2 more patched isapi_redirect.dll which the Adobe ColdFusion Team was kind enough to send us. In one of the tests we updated the connector using wsconfig. (We didn't remove and re-add the connector). Still, no dice. All our attempts failed in the same way, with the same error messages.

 

There was one thing we did after applying Update 5. As a workaround for the issue https://tracker.adobe.com/#/view/CF-4205271, we added the following flag in JVM config

 

-Dcoldfusion.http.usepooling=false

 

Do you suggest we try the isapi_redirect.dll of Update 4 on the current Update 5 installation?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 23, 2019 0
Adobe Community Professional ,
Oct 23, 2019

Copy link to clipboard

Copied

Oh, ok. Good that you can recover by reverting. That was not clear (so I wasn't sure how desparate the situation was). Of course, I realize you may have a pressing/urgent desire to get to update 5, for its bug fixes.

 

And to be clear, is the DLL you are trying with u5 the one Adobe offers here: https://coldfusion.adobe.com/2019/10/error-accessing-coldfusion-administrator-using-connector-port/? There is a link for it in that blog post. If you have not tried it, it's worth a shot.

 

And then sure, if it does NOT work, then you could try the u4 connector with the u5 update in place. It has worked for some, and yet not for others. Odd. I just mean that if you did go forward to update 5 (and the updated one from Adobe still did NOT work), then you COULD try the older connector (from u4 or before).  If neither works, you could revert back to u4 and its connector dll.

 

Either way, if you feel confident that the problem is SOLELY in the update 5 and even that updated DLL they offer, this is where Adobe is saying (in their discussions of the issue) that they want to hear from folks, at cfinstal@adobe.com (one "L"). So yes, I am proposing that rather than (or in addition to) opening forum threads and bug tracker tickets, take their offer to reach out about this specific issue, because of course they TOO want to see people get to update 5, and they want to understand who still has problems with its updated connector.

 

I will say it's still possible there's SOMETHING in your environment to explain things, but I will leave it for them to sort that out as I am trusting your assertion that there is nothing else "changing" between it working with u4 and failing with u5. 🙂

/Charlie (server troubleshooter, carehart.org)

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 23, 2019 0
Adobe Community Professional ,
Oct 24, 2019

Copy link to clipboard

Copied

Thanks, Charlie.

We shall extract the isapi_redirect.dll from Update 4 (which worked fine), and test it on Update 5.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 24, 2019 0
BKBK LATEST
Adobe Community Professional ,
May 31, 2020

Copy link to clipboard

Copied

An update.

One part of our application runs on CFCs and CFM pages. The other is a REST API.

With the assistance of the ColdFusion Team at Adobe, we have discovered the following:

 

The 502 Bad Gateway occurs only in the REST API. We narrowed the root-cause down to the onRESTRequest event-handler in Application.cfc. The fact that OnRESTRequest is not single-threaded is the root-cause.

 

It turns out that, in our application, multiple REST requests sharing the same URI do occasionally have access to onRESTRequest. Some of the code in onRESTRequest involves flushing a servlet response outputstream to the client, as follows:

 

var out = response.getOutputStream();     
out.write(toBinary(toBase64(content)));       
out.flush();       
out.close();

 

As onRESTRequest is not single-threaded, two such responses sometimes create a race-condition. One of them is flushed to the client, as expected. But parts of the second response (for example, headers) may not be flushed or may be flushed in the wrong order. The result is a 502 Bad Gateway error.

 

Workaround:

Use a ColdFusion flush after the servlet flush.

 

var out = response.getOutputStream();     
out.write(toBinary(toBase64(content)));       
out.flush();       
out.close();

// Ensure ColdFusion flushes every response
cfflush();

 

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 31, 2020 0