Highlighted

How to troubleshoot a 502 error?

New Here ,
May 25, 2017

Copy link to clipboard

Copied

How do I begin to troubleshoot a 502 error?

CF11/Windows 2012r2/IIS8

FusionReactor installed

I connect to twilio to send sms messages, and have a page setup to collect their callbacks when messages are sent.  They are sending me notifications that at random, they are receiving 502 errors when posting to the callback page.

<html>

   <head>

   <title>502 Bad Gateway</title>

   </head>

   <body>

   <h1>Bad Gateway</h1>An upstream server returned an invalid response.

   </body>

</html>

The callback page receives the info and writes to a db.  A call that usually takes 15ms to process on average.

Sometimes they may hit my server with 200 call back requests pretty rapidly.  At random, 1, multiple or none of those requests may result in a 502 error.

I have never physically seen a 502 error appear in the application.log, coldfusion-error.log, etc....

I also have isapi_redirect.log turned on and see nothing there that would indicate an error, just a bunch of lines that look like this:

[Thu May 25 07:16:12.393 2017] [14632:20188] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 23 of max reuse connection 250 and total endpoint count 250

[Thu May 25 07:16:12.393 2017] [14632:11256] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 24 of max reuse connection 250 and total endpoint count 250

[Thu May 25 07:16:12.394 2017] [14632:13928] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 25 of max reuse connection 250 and total endpoint count 250

[Thu May 25 07:16:12.394 2017] [14632:3120] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 26 of max reuse connection 250 and total endpoint count 250

[Thu May 25 09:42:43.780 2017] [14632:19260] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 27 of max reuse connection 250 and total endpoint count 250

[Thu May 25 09:42:44.581 2017] [14632:10920] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 28 of max reuse connection 250 and total endpoint count 250

[Thu May 25 10:00:14.724 2017] [14632:17416] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 29 of max reuse connection 250 and total endpoint count 250

[Thu May 25 10:00:14.736 2017] [14632:12616] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 30 of max reuse connection 250 and total endpoint count 250

[Thu May 25 10:26:00.375 2017] [14632:17416] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 31 of max reuse connection 250 and total endpoint count 250

I've checked fusion reactor and I don't see any increases in traffic or any red flags that indicate anything unusual going on with the server.  Heap and connections are well below their limits.  DB usage is minimal.

None of my users on the server have ever reported seeing a 502 error when browsing the apps.

The only thing I can see that may be somewhat related is that the frequency increased once I applied HF11-00011.

Thanks

TOPICS
Getting started

Views

1.0K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

How to troubleshoot a 502 error?

New Here ,
May 25, 2017

Copy link to clipboard

Copied

How do I begin to troubleshoot a 502 error?

CF11/Windows 2012r2/IIS8

FusionReactor installed

I connect to twilio to send sms messages, and have a page setup to collect their callbacks when messages are sent.  They are sending me notifications that at random, they are receiving 502 errors when posting to the callback page.

<html>

   <head>

   <title>502 Bad Gateway</title>

   </head>

   <body>

   <h1>Bad Gateway</h1>An upstream server returned an invalid response.

   </body>

</html>

The callback page receives the info and writes to a db.  A call that usually takes 15ms to process on average.

Sometimes they may hit my server with 200 call back requests pretty rapidly.  At random, 1, multiple or none of those requests may result in a 502 error.

I have never physically seen a 502 error appear in the application.log, coldfusion-error.log, etc....

I also have isapi_redirect.log turned on and see nothing there that would indicate an error, just a bunch of lines that look like this:

[Thu May 25 07:16:12.393 2017] [14632:20188] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 23 of max reuse connection 250 and total endpoint count 250

[Thu May 25 07:16:12.393 2017] [14632:11256] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 24 of max reuse connection 250 and total endpoint count 250

[Thu May 25 07:16:12.394 2017] [14632:13928] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 25 of max reuse connection 250 and total endpoint count 250

[Thu May 25 07:16:12.394 2017] [14632:3120] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 26 of max reuse connection 250 and total endpoint count 250

[Thu May 25 09:42:43.780 2017] [14632:19260] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 27 of max reuse connection 250 and total endpoint count 250

[Thu May 25 09:42:44.581 2017] [14632:10920] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 28 of max reuse connection 250 and total endpoint count 250

[Thu May 25 10:00:14.724 2017] [14632:17416] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 29 of max reuse connection 250 and total endpoint count 250

[Thu May 25 10:00:14.736 2017] [14632:12616] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 30 of max reuse connection 250 and total endpoint count 250

[Thu May 25 10:26:00.375 2017] [14632:17416] [info] ajp_process_callback::jk_ajp_common.c (2244): (cfusion) current reuse count is 31 of max reuse connection 250 and total endpoint count 250

I've checked fusion reactor and I don't see any increases in traffic or any red flags that indicate anything unusual going on with the server.  Heap and connections are well below their limits.  DB usage is minimal.

None of my users on the server have ever reported seeing a 502 error when browsing the apps.

The only thing I can see that may be somewhat related is that the frequency increased once I applied HF11-00011.

Thanks

TOPICS
Getting started

Views

1.0K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
May 25, 2017 0
BKBK LATEST
Adobe Community Professional ,
Jun 03, 2017

Copy link to clipboard

Copied

From your description, it seems that the client (Twilio) sent requests to the CFM page at moments when the ColdFusion server was unable to process them. I would image that ColdFusion would have been busy writing to the database.

A possible solution is to use asynchronous callbacks. Design their code to wait until the relevant server processes are complete.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 03, 2017 0