Highlighted

Coldfusion 10 Enterprise, IIS 8.5 503 errors

Community Beginner ,
Jun 30, 2014

Copy link to clipboard

Copied

We are upgrading certain applications in our environment to Coldfusion 10 and Server 2012 R2.

Everything in the QUA environment worked great, no issues.  However, when I put the new servers in production, I started seeing random 503 issues from the servers. (the application pools were stopped) - Restarting the application pool revived the site without a need to restart coldfusion.  In the interim of finding a fix, I had a powershell script checking the status of my app pool hourly and if it is down, I have it restart it and email me.

When I started looking into this, I did some tuning of my IIS connectors based on this article: Tuning ColdFusion 10 IIS Connector configuration — Adobe ColdFusion Blog - These particular servers are configured with one instance of coldfusion but separate connectors for each of the sites. Since implementing those changes last week, the 503 errors have only occurred one other time over the past weekend. Looking in the isapi_redirect.log file, I see this error that corresponds to each time the application pools crashed:

[error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

Just wondering what the above is actually referencing and if my issue is truly related to load?

Views

1.2K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Coldfusion 10 Enterprise, IIS 8.5 503 errors

Community Beginner ,
Jun 30, 2014

Copy link to clipboard

Copied

We are upgrading certain applications in our environment to Coldfusion 10 and Server 2012 R2.

Everything in the QUA environment worked great, no issues.  However, when I put the new servers in production, I started seeing random 503 issues from the servers. (the application pools were stopped) - Restarting the application pool revived the site without a need to restart coldfusion.  In the interim of finding a fix, I had a powershell script checking the status of my app pool hourly and if it is down, I have it restart it and email me.

When I started looking into this, I did some tuning of my IIS connectors based on this article: Tuning ColdFusion 10 IIS Connector configuration — Adobe ColdFusion Blog - These particular servers are configured with one instance of coldfusion but separate connectors for each of the sites. Since implementing those changes last week, the 503 errors have only occurred one other time over the past weekend. Looking in the isapi_redirect.log file, I see this error that corresponds to each time the application pools crashed:

[error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

Just wondering what the above is actually referencing and if my issue is truly related to load?

Views

1.2K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Jun 30, 2014 0
Adobe Employee ,
Jun 30, 2014

Copy link to clipboard

Copied

Can you refer to ColdFusion 11 IIS Connector Tuning — Adobe ColdFusion Blog and see if it helps.

Regards,

Anit Kumar

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 30, 2014 1
Community Beginner ,
Jul 07, 2014

Copy link to clipboard

Copied

Anit,

I've followed the Coldfusion 11 guide.  I still experienced a 503 error yesterday morning at 9:00am.  This is a time that there was little to no traffic.  My powershell script caught the worker pool down and restarted it.  In th isapi_redirect.log file on that server, I start see errors about a 1/2 hour before the script ran.

[Sun Jul 06 08:29:19.403 2014] [1616:5380] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:29:19.409 2014] [1616:5380] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:29:24.071 2014] [5776:1688] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:29:24.074 2014] [5776:1688] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:29:24.080 2014] [5776:1688] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:30:07.131 2014] [2992:1268] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:30:07.134 2014] [2992:1268] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:30:07.140 2014] [2992:1268] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:30:08.769 2014] [5312:3088] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:30:08.772 2014] [5312:3088] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:30:08.779 2014] [5312:3088] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:30:09.045 2014] [5312:4788] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 1 of max reuse connection 270 and total endpoint count 800

[Sun Jul 06 08:30:09.503 2014] [5260:4696] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:30:09.506 2014] [5260:4696] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:30:09.512 2014] [5260:4696] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:30:10.519 2014] [5496:3568] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:30:10.522 2014] [5496:3568] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:30:10.528 2014] [5496:3568] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:30:10.704 2014] [5496:4116] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 1 of max reuse connection 270 and total endpoint count 800

[Sun Jul 06 08:30:10.967 2014] [5496:1780] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 2 of max reuse connection 270 and total endpoint count 800

[Sun Jul 06 08:30:11.218 2014] [3000:4560] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:30:11.221 2014] [3000:4560] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:30:11.227 2014] [3000:4560] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:30:11.970 2014] [1224:5360] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:30:11.973 2014] [1224:5360] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:30:11.979 2014] [1224:5360] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:31:14.751 2014] [1600:2008] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:31:14.755 2014] [1600:2008] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:31:14.761 2014] [1600:2008] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 08:31:15.715 2014] [900:2120] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 08:31:15.719 2014] [900:2120] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 08:31:15.725 2014] [900:2120] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 09:00:47.181 2014] [2016:1780] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 09:00:47.185 2014] [2016:1780] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 09:00:47.192 2014] [2016:1780] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 09:00:48.801 2014] [2452:1632] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 09:00:48.803 2014] [2452:1632] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Sun Jul 06 09:00:48.810 2014] [2452:1632] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 09:01:20.975 2014] [3256:5492] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Jul 06 09:01:20.983 2014] [3256:5492] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Jul 06 09:01:23.720 2014] [3256:5492] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 1 of max reuse connection 270 and total endpoint count 800

[Sun Jul 06 09:01:46.739 2014] [3256:1724] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 2 of max reuse connection 270 and total endpoint count 800

[Sun Jul 06 09:02:15.222 2014] [3256:5244] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 3 of max reuse connection 270 and total endpoint count 800

If I read the guide correctly, I should now probably up my total connections.  However, Looking at the metrics for this site during the 8:00 - 9:02 hour, there were only 126 transactions across two servers.  Can you give me any information on what this error, "[error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly." actually means? 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 07, 2014 0
Community Beginner ,
Jul 07, 2014

Copy link to clipboard

Copied

The issue occurred again this afternoon.  I will be attempting some additional tuning tonight on the connectors. Again, at the time of the 503 errors, this was in the connector logs:

[Mon Jul 07 13:54:50.963 2014] [3256:5672] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (2), cping (0), send (0)

[Mon Jul 07 13:57:47.587 2014] [3256:1724] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (1), cping (0), send (0)

[Mon Jul 07 14:01:26.575 2014] [3568:5232] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Jul 07 14:01:26.579 2014] [3568:5232] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.

[Mon Jul 07 14:01:26.587 2014] [3568:5232] [info] init_jk::jk_isapi_plugin.c (2829): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Jul 07 14:01:26.688 2014] [3568:2160] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 1 of max reuse connection 270 and total endpoint count 800

[Mon Jul 07 14:01:27.702 2014] [3256:1804] [info] TerminateFilter::jk_isapi_plugin.c (2359): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Jul 07 14:01:28.060 2014] [3568:2160] [info] ajp_process_callback::jk_ajp_common.c (2066): current reuse count is 2 of max reuse connection 270 and total endpoint count 800

[Mon Jul 07 14:01:30.472 2014] [1256:2100] [info] init_jk::jk_isapi_plugin.c (2650): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Jul 07 14:01:30.482 2014] [1256:2100] [error] init_jk::jk_isapi_plugin.c (2795): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.


Can you explain the error message?  I just want to make sure the connector pool tuning is the right path to go down for this.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 07, 2014 0
Adobe Employee ,
Jul 07, 2014

Copy link to clipboard

Copied

This is a tomcat error. This comes usually, if you are running multiple applications under a site with each having their own App pool?

Are you running multiple applications within a site?

Regards,

Anit Kumar

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 07, 2014 0
Community Beginner ,
Jul 07, 2014

Copy link to clipboard

Copied

Hi Anit, thanks for the reply.

Yes, this site has one .Net application that runs from within the main coldfusion site.  So, it does run under it's own application pool.  The coldfusion application is the primary site and the only access to this is through that.  So, what can be done to resolve those types of issues?   I cannot separate this from the site and it was never an issue when this was on Coldfusion 9.01 and JRUN.  Any advise here?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 07, 2014 0
Adobe Employee ,
Jul 08, 2014

Copy link to clipboard

Copied

This issue is due to Tomcat and was not there in JRun.

Regards,

Anit Kumar

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 08, 2014 0
zeejayy LATEST
Explorer ,
Jul 29, 2014

Copy link to clipboard

Copied

We are having the the same issue. CF 10 Standard Edition / IIS 8.5 on windows server 2012 R2

I found this open bug https://bugbase.adobe.com/index.cfm?event=bug&id=3530880

Which is exactly the steps we have gone through and pretty close to the same settings we used.

We are also running a CF 10 site with a .net application running under on it's own separate app pool.

Is there any known solution for this? The best we have been able to do is set the error tolerance

for the application higher. This doesn't resolve the issue but it does help.

We have gone through all the connector tuning as well.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 29, 2014 0