Skip to main content
BKBK
Community Expert
Community Expert
October 23, 2019
Answered

Connector errors after ColdFusion 2018 Update 5: Bad gateway, Tomcat is down or refused connection

  • October 23, 2019
  • 2 replies
  • 3110 views

We have been getting frequent errors after installing Update 5 of ColdFusion 2018. The site serves web pages, but pressing F5 (to refresh the page) produces an Apache error page: 

The isapi_redirect.log file entry corresponding to the failed request is:

 

[Wed Oct 23 17:18:44.903 2019] [13452:8640] [info] ajp_connection_tcp_get_message::jk_ajp_common.c (1425): (t1stdrdrci) can't receive the response header message from tomcat, tomcat (127.0.0.1:8030) has forced a connection close for socket 1464
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] ajp_get_reply::jk_ajp_common.c (2402): (t1stdrdrci) Tomcat is down or refused connection. No response has been sent to the client (yet)
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] ajp_service::jk_ajp_common.c (2975): (t1stdrdrci) sending request to tomcat failed (unrecoverable), (attempt=1)
[Wed Oct 23 17:18:44.903 2019] [13452:8640] [error] HttpExtensionProc::jk_isapi_plugin.c (2739): service() failed with http error 502

 

We applied the patched isapi_redirect.dll that Adobe recently released. In vain.

I then reported a bug: https://tracker.adobe.com/#/view/CF-4205448 (Please follow this link and vote to have this fixed - Thnx!)

 

Our setup:

ColdFusion 2018 Enterprise Update 5

64 Bit Windows Server 2016 + IIS 

 

If you found a solution to this problem, could you please share it with us?

 

This topic has been closed for replies.
Correct answer BKBK

In the end, we found an even neater alternative to the above solution, namely:

if (isResponseContentBinary) {
    // Stream binary content
    writeoutput(toString(binaryResponse));
    cfflush();
} else {
    // Return REST response
    restSetResponse(response);
    return response;
}

2 replies

BKBK
Community Expert
BKBKCommunity ExpertAuthor
Community Expert
May 31, 2020

An update.

One part of our application runs on CFCs and CFM pages. The other is a REST API.

With the assistance of the ColdFusion Team at Adobe, we have discovered the following:

 

The 502 Bad Gateway occurs only in the REST API. We narrowed the root-cause down to the onRESTRequest event-handler in Application.cfc. The fact that OnRESTRequest is not single-threaded is the root-cause.

 

It turns out that, in our application, multiple REST requests sharing the same URI do occasionally have access to onRESTRequest. Some of the code in onRESTRequest involves flushing a servlet response outputstream to the client, as follows:

 

var out = response.getOutputStream();     
out.write(toBinary(toBase64(content)));       
out.flush();       
out.close();

 

As onRESTRequest is not single-threaded, two such responses sometimes create a race-condition. One of them is flushed to the client, as expected. But parts of the second response (for example, headers) may not be flushed or may be flushed in the wrong order. The result is a 502 Bad Gateway error.

 

Workaround:

Use a ColdFusion flush after the servlet flush.

 

var out = response.getOutputStream();     
out.write(toBinary(toBase64(content)));       
out.flush();       
out.close();

// Ensure ColdFusion flushes every response
cfflush();

 

 

BKBK
Community Expert
BKBKCommunity ExpertAuthorCorrect answer
Community Expert
December 30, 2020

In the end, we found an even neater alternative to the above solution, namely:

if (isResponseContentBinary) {
    // Stream binary content
    writeoutput(toString(binaryResponse));
    cfflush();
} else {
    // Return REST response
    restSetResponse(response);
    return response;
}
Charlie Arehart
Community Expert
Community Expert
October 23, 2019

BKBK, you say this is since applying update 5. Did you first try to revert to whatever update you were on before, to prove is this is indeed "the cause"? And if you may not want to do that, have you tried using whatever isapi_redirect.dll you had from BEFORE the update 5? Sadly, the CF wsconfig "update" feature does not save a backup (I have a feature request for that).

 

And in doing the manual update to the dll, did you restart IIS after that? 

 

Was it failing the same way after update 5 but before the new dll? Had you tried to upate the connector AS PER THE UPDATED CONNECTOR THAT CAME with update 5? Or did you perhaps skip that step? What update were you on before?

 

Finally, it's possible that your problem is NOT about the update at all. I assume the site was working fine before the update, right? Did you do ANYTHING else after applying the CF update? Did you update the connector (with the wsconfig, or not)? Did you change anything in IIS? Did you perhaps "remove" and re-add a connector, instead of using the "update" feature of the wsconfig tool?

 

Answers to any of the above could provide a clue as to what is amiss. I appreciate folks often just wish to "get the solution", but as you are finding this is a more challenging problems for you, and it may take a bit more digging to resolve. (I will assume you are not interested in a consulting session to try to address this, if the above does not help. But I will say to other readers who may find this that I can often help solve such things, and perhaps in just minutes when I can actually see what's amiss on your server.)

 

Hope something there helps.

/Charlie (troubleshooter, carehart. org)
ashwinia70450482
Participating Frequently
December 29, 2020

Hi Charlie:

We have the same problem, in one of our cf servers, but restarting the service will fix it. It occurs off and on, so now way i can reproduce this. 

ashwinia70450482
Participating Frequently
December 30, 2020

We will not be able to solve your problem readily with only that info. You may have problems in cf (having nothing to do with the connector) , or you may have need of connector tuning. You may have failed to update the connector. These are things we can't easily diagnose remotely.

 

I know you just want the problem solved, and likely preferably here for free. I do offer hundreds of replies per year for free, but I just don't think that will work well for this. Perhaps others will throw out many things to try.

 

But my sense is that this will be best and most effectively resolved via a direct remote screenshare session. If you're interested, see the link in my signature (including my satisfaction guarantee). We might get to the bottom of things in less than a half hour. 


Charlie:

I have updated the connector again, it fixed the problem.