Performance Tuning

[Guest]

I have 1 CF 10 server 64 bit running on IIS 7.5 on a Win 2008 R2 VM w/ 4 Gig Ram on 4Ghz of processing. The box hosts 5 websites with not too much traffic.  I've been experiencing Service unavailable errors about once per day and only on certain websites. I restart the CF App Service and all is well again.  All 4 of the websites have the same code with just a few variables difference to get different data.

I've made some adjustments in the server. For example I set min and max JVM Heap size to 2048.

Any ideas what I can do to get things running with more stability?

[Anit_Kumar]

You may have to tune your CF connector appropriately to make your website(s) works flawlessly. You can refer to http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

[carl type3]

Setting minimum values can be just as important as maximum. Suggest set min JVM heap to half max so that would be 1024m. You can do well on 64 bit to increase MaxPermSize from 192m to 356m and while your dealing with permanent generation set an initial or minimum setting PermSize=192m. Well that is a start not exactly tuning since you would need to do some JVM logging to know for sure what values suit best.

Check logs for errors or warnings CF10\cfusion\log CF-EVENT and CF-OUT log. Indeed the Tomcat connector may not need looking at so check CF10\config\wsconfig\1\ISAPI log for errors.

Is CF10 patched to update7 and had WSCONFIG tool run to remove and add IIS connector?

HTH, Carl.

[Guest]

Thanks Carl,

My JVM Heap sizes are now set to Min = 1024 and Max = 2048

And my JVM args are now...

-server -XX:MaxPermSize=356m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random

I'm not sure where to set the minimum PermSize=192m as you mentioned?

I'll wait for your response on that before I do a restart.

I am patched to Version 10,0,7,283649 so I'm good there. I did run through the article that Anit Kumar has mentioned and to say the least, things did not go smoothly.

Also, I have not much experience looking at the log files for much other than some general app debugging but I'll go have a look.

Jack

[Anit_Kumar]

The -XX:MaxPermSize is the PermSize.

[Guest]

Thanks Anit!

I restarted with the new settings.

Now I'm going to focus on the Webserver Configuration Set up now...

I have...

[localhost:cfusion] Internet Information Server (IIS) : All

[localhost:cfusion] Internet Information Server (IIS) : www.website1.com

[localhost:cfusion] Internet Information Server (IIS) : www.website2.com

[localhost:cfusion] Internet Information Server (IIS) : www.website3.com

And I know that isn't how it is supposed to be set up with 5 or so websites but this really gave me problems and I could not get all sites to work at the same time any other way.

[Anit_Kumar]

If all the websites are ColdFusion based, then you can create a connector with "ALL". You can tune your connectors following the document link pasted above.

Alternatively, you can create individual connectors and then just review the workers.properties file for the same.

[carl type3]

Jack Regarding PermGen part of your JVM alter this:
-server -XX:MaxPermSize=356m -XX:+UseParallelGC etc
to:
-server -XX:PermSize=192m -XX:MaxPermSize=356m -XX:+UseParallelGC etc

That sets the minimum or initial setting for PermGen to 192m else it is 64m if not defined.

> looking at the log files
Check "CF-OUT log" for GC or heap error / warning messages.
Check "ISAPI log" for [error] messages.

Check "CF-EVENT log" for AJP error /warning messages.

Did you perform the manual step mentioned here?
http://helpx.adobe.com/coldfusion/kb/coldfusion-security-hotfix-apsb12-25.html
This bit -
After applying this hot fix, reconfigure the connectors using the wsconfig tool. Find the wsconfig.exe at {cf_install_home}/{CF Instance}/runtime/bin. To reconfigure the connectors, remove the IIS connector listed and use the add option to configure the latest IIS connector.

Again Carl.

PS sorry for shorthand it is Saturday morning for me.

[Guest]

Thanks again Carl.

Wow - Just got home on Friday here. Wish I already had a full nights sleep.

When I originally ran the update - I disconnected and reconfigured my webserver connections as instructed. But it was a nightmare because when I reconnected them using ALL Sites... Many of my websites would not come back up. Resource not found error. That is why I ended up with such a wierd Connector configuration through trial and error to get all the sites working again.

Regarding the logs...

My logs are riddled with tons of entries since September 20, 2012 when I upgraded to CF 10. Here is a sampling of some odd entries which I found scanning things over...

I do not see an ISAPI log?

ColdFusion-out.log

Feb 22, 2013 18:51:25 PM Information [scheduler-1] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 482918920 Total memory: 1017774080 Active Sessions: 1

Security.log

"Information","ajp-bio-8012-exec-17","02/20/13","16:03:48",,"An error occurred while fetching element from authcache." (About 8x/day)

Scheduler.log

"Information","DefaultQuartzScheduler_QuartzSchedulerThread","02/22/13","18:11:25",,"Task WEBSITE.GET_RSS.CFM misfired."

Metrics.log

"Information","scheduler-0","02/22/13","00:04:27",,"Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: 2 Bytes received: null Bytes sent: null Free memory: 1336036024 Total memory: 2063794176 Active Sessions: 4"

( *** There are 950 of these entries for today )

[carl type3]

Jack, Nothing to worry there in ColdFusion-out.log, Security.log, Scheduler.log or Metrics.log.

"ISAPI log" is my short hand for the isapi_redirect.log file in CF10\config\wsconfig\N where N is magic number eg 1 assigned by WSCONFIG tool. Any [error] lines in isapi_redirect.log?

Does CF10\cfusion\logs coldfusion-error.log have any ajp messages other than:
Stopping ProtocolHandler ["ajp-xxx-nnnn"]
Initializing ProtocolHandler ["ajp-xxx-nnnn"]
Starting ProtocolHandler ["ajp-xxx-nnnn"]
that can not be associated with a scheduled CF IIS restart?

HTH, Carl.

[Guest]

Regarding the asapi_redirect.log files, there are plenty of errors. I don't know where to begin. Here are a few links to copies of my asapi_redirect.log files...

https://docs.google.com/file/d/0B1-0aakdVIO7VFExV1V2ZGc2NkU/edit?usp=sharing

https://docs.google.com/file/d/0B1-0aakdVIO7cklabERNdWRsWUk/edit?usp=sharing

https://docs.google.com/file/d/0B1-0aakdVIO7ZVZGQldsbkRGUHc/edit?usp=sharing

https://docs.google.com/file/d/0B1-0aakdVIO7QVpRR2xHRFU4QzQ/edit?usp=sharing

Here is a sampling of the error log. Towards the end I have the entire day on Friday...

https://docs.google.com/file/d/0B1-0aakdVIO7U1NNN0JCLWNHdkE/edit?usp=sharing

I do some development on this box so there is plenty of activity.

Thanks again Carl.

Jack

[Guest]

Alright the Error happened again. The error was left for about 2 hours and 4 of 5 websites on the server were like this.

Service Temporary Unavailable!

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Jakarta/ISAPI/isapi_redirector/1.2.32 ()

**** Nothing but starting and stopping references in the Coldfusion Error log...

**** Coldfusion out log

Feb 24, 2013 08:45:38 AM Information [scheduler-2] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 661995560 Total memory: 1048444928 Active Sessions: 2

**** Here is the ISAPA log...

[Sun Feb 24 09:15:56.322 2013] [3488:1484] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Sun Feb 24 09:15:56.541 2013] [3416:2836] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Sun Feb 24 09:15:56.541 2013] [3416:2836] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Sun Feb 24 09:17:06.358 2013] [3416:5640] [info] jk_open_socket::jk_connect.c (626): connect to 127.0.0.1:8012 failed (errno=61)

[Sun Feb 24 09:17:06.358 2013] [3416:5640] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1047): Failed opening socket to (127.0.0.1:8012) (errno=61)

[Sun Feb 24 09:17:06.358 2013] [3416:5640] [error] ajp_send_request::jk_ajp_common.c (1669): (cfusion) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=61)

[Sun Feb 24 09:17:06.373 2013] [3416:5640] [info] ajp_service::jk_ajp_common.c (2684): (cfusion) sending request to tomcat failed (recoverable), because of error during request sending (attempt=1)

[Sun Feb 24 09:17:07.561 2013] [3416:5640] [info] jk_open_socket::jk_connect.c (626): connect to 127.0.0.1:8012 failed (errno=61)

[Sun Feb 24 09:17:07.561 2013] [3416:5640] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1047): Failed opening socket to (127.0.0.1:8012) (errno=61)

[Sun Feb 24 09:17:07.608 2013] [3416:5640] [error] ajp_send_request::jk_ajp_common.c (1669): (cfusion) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=61)

[Sun Feb 24 09:17:07.608 2013] [3416:5640] [info] ajp_service::jk_ajp_common.c (2684): (cfusion) sending request to tomcat failed (recoverable), because of error during request sending (attempt=2)

[Sun Feb 24 09:17:07.623 2013] [3416:5640] [error] ajp_service::jk_ajp_common.c (2703): (cfusion) connecting to tomcat failed.

[Sun Feb 24 09:17:07.623 2013] [3416:5640] [error] HttpExtensionProc::jk_isapi_plugin.c (2293): service() failed with http error 503

[carl type3]

Using Windows Task Manager | Processes tab | View | Select Columns | Memory - Commit Size | Show all users

How much memory commit size does coldfusion.exe consume when system fails?

>Nothing but starting and stopping references in the Coldfusion Error log

This could be OK or bad. Are the stopping references the CF10 App Service restart you are performing to make the system function since I would not expect a stopping and starting in here otherwise?

ISAPI log seems to indicate Tomcat ISAPI DLL redirector connector is unable to connect with Tomcat AJP connector at the CF10 worker end. So perhaps your server.xml syntax CF10\cfusion\runtime\conf AJP section would be interesting eg:

<!--  Define an AJP 1.3 Connector on port 8009-->

<Connector port="8012" protocol="AJP/1.3" redirectPort="8445" tomcatAuthentication="false" />

Regards, Carl.

[Guest]

Alright one of my 5 websites is down again.

application.log

"Error","ajp-bio-8012-exec-77","02/25/13","09:49:58",,""

coldfusion-error.log

is clean

coldfusion-out.log

Feb 25, 2013 12:00:31 PM Information [scheduler-2] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 455965136 Total memory: 1046740992 Active Sessions: 10

Feb 25, 2013 12:01:31 PM Information [scheduler-1] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 466892856 Total memory: 1045954560 Active Sessions: 11

Feb 25, 2013 12:02:31 PM Information [scheduler-2] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 400167992 Total memory: 1048051712 Active Sessions: 11

Feb 25, 2013 12:03:31 PM Information [scheduler-0] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 603792408 Total memory: 1047265280 Active Sessions: 11

Feb 25, 2013 12:04:31 PM Information [scheduler-2] - Max threads: null Current thread count: null Current thread busy: null Max processing time: null Request count: null Error count: null Bytes received: null Bytes sent: null Free memory: 593062152 Total memory: 1049296896 Active Sessions: 10

Without restarting the service to fix this I can see the following 3 points:

1. coldfusion.exe Memory commit size consumption...

    - Memory (private working set) = 1,297,856 K

    - Commit Size = 1,383,172 K

2. server.xml syntax check. Note AJP connector line,,,

<Server port="8007" shutdown="SHUTDOWN">

  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on">

  </Listener>

  <Listener className="org.apache.catalina.core.JasperListener">

  </Listener>

  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener">

  </Listener>

  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener">

  </Listener>

  <GlobalNamingResources>

    <Resource description="User database that can be updated and saved" name="UserDatabase" pathname="conf/tomcat-users.xml" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" type="org.apache.catalina.UserDatabase" auth="Container">

    </Resource>

  </GlobalNamingResources>

  <Service name="Catalina">

    <Executor name="tomcatThreadPool" minSpareThreads="4" maxThreads="50" namePrefix="catalina-exec-">

    </Executor>

    <Connector port="8012" protocol="AJP/1.3" redirectPort="8445" maxThreads="50" tomcatAuthentication="false">

    </Connector>

    <Engine jvmRoute="cfusion" name="Catalina" defaultHost="localhost">

      <Realm className="org.apache.catalina.realm.LockOutRealm">

        <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase">

        </Realm>

      </Realm>

      <Host name="localhost" autoDeploy="false" unpackWARs="true" appBase="webapps">

        <Valve pattern="%h %l %u %t &quot;%r&quot; %s %b" directory="logs" prefix="localhost_access_log." className="org.apache.catalina.valves.AccessLogValve" suffix=".txt" resolveHosts="false">

        </Valve>

      </Host>

    </Engine>

  </Service>

</Server>

3.

isapi_redirect.log on Connection [1]

[Mon Feb 25 02:23:59.463 2013] [3416:1728] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (14), cping (0), send (0)

isapi_redirect.log on Connection [2]

[Mon Feb 25 00:54:29.760 2013] [5128:5680] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 00:54:29.775 2013] [5128:5680] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 00:54:29.822 2013] [5128:5680] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 01:15:30.637 2013] [5128:4184] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 07:15:07.518 2013] [2208:5960] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 07:15:07.596 2013] [2208:5960] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 07:15:07.752 2013] [2208:5960] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 07:39:08.423 2013] [2208:1320] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 08:08:41.201 2013] [4480:5100] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 08:08:41.201 2013] [4480:5100] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 08:08:41.279 2013] [4480:5100] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 08:41:42.151 2013] [4480:3688] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 09:22:20.901 2013] [5896:5380] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 09:22:20.901 2013] [5896:5380] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 09:22:21.276 2013] [5896:5380] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 10:47:28.509 2013] [5896:5760] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 2 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 11:15:22.304 2013] [5896:1708] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

isapi_redirect.log on Connection [3]

[Mon Feb 25 02:23:12.506 2013] [4048:5756] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (11), cping (0), send (0)

[Mon Feb 25 03:22:57.283 2013] [4048:3524] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 12 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:23:00.752 2013] [4048:5232] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 13 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:23:01.189 2013] [4048:6048] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 14 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:23:01.846 2013] [4048:892] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 15 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:23:01.939 2013] [4048:4496] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 16 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:23:02.830 2013] [4048:3696] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 17 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:23:03.924 2013] [4048:3348] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 18 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:30:03.591 2013] [4048:5812] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 19 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:30:04.107 2013] [4048:4984] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 20 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:30:06.201 2013] [4048:5144] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 21 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:30:06.357 2013] [4048:5816] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 22 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:30:06.748 2013] [4048:4136] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 23 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:30:06.888 2013] [4048:5384] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 24 of max reuse connection 250 and total endpoint count 250

isapi_redirect.log on Connection [4]

[Mon Feb 25 00:11:25.688 2013] [5760:5744] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 00:11:25.688 2013] [5760:5744] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 00:11:25.782 2013] [5760:5744] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 00:13:26.430 2013] [5760:5744] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 2 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 02:36:27.214 2013] [5760:1208] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 02:49:36.217 2013] [4888:5416] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 02:49:36.217 2013] [4888:5416] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 02:49:38.842 2013] [4888:2796] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 03:16:37.243 2013] [4888:5064] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 03:20:10.241 2013] [5868:2608] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 03:20:10.241 2013] [5868:2608] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 03:20:10.366 2013] [5868:2608] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 04:30:11.478 2013] [5868:3936] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 05:00:26.063 2013] [5860:3728] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 05:00:26.079 2013] [5860:3728] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 05:00:26.235 2013] [5860:3728] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 05:45:27.221 2013] [5860:1460] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 05:49:57.722 2013] [5200:4124] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 05:49:57.738 2013] [5200:4124] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 05:49:58.019 2013] [5200:4124] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 05:49:58.019 2013] [5200:5852] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 2 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:32:31.167 2013] [5200:5228] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 3 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:32:31.167 2013] [5200:5852] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 4 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:40:23.494 2013] [5200:4948] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 5 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:41:12.388 2013] [5200:3364] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 6 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:42:40.096 2013] [5200:5852] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 7 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:44:12.556 2013] [5200:4948] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 8 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:45:32.529 2013] [5200:3364] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 9 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 06:46:13.391 2013] [5200:4464] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 10 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 08:39:59.113 2013] [5200:5736] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 09:05:05.100 2013] [5272:5724] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 09:05:05.100 2013] [5272:5724] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 09:05:05.210 2013] [5272:5724] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 09:28:06.158 2013] [5272:4468] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 09:37:16.396 2013] [2208:4812] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 09:37:16.396 2013] [2208:4812] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 09:37:16.646 2013] [2208:4812] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

[Mon Feb 25 09:58:17.445 2013] [2208:3608] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping

[Mon Feb 25 10:03:03.792 2013] [4936:1936] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()

[Mon Feb 25 10:03:03.792 2013] [4936:1936] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

[Mon Feb 25 10:03:03.932 2013] [4936:1936] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

I'm going to have to restart now,

Jack

[carl type3]

CF10 Java is 64 bit? You see CF RAM is 1.3Gb and on 32 bit Java there is a logical limit between 1.3Gb and 1.4Gb depending on some variables.
Check via CF10admin > Server Settings > Settings Summary > Java VM Name =
Java HotSpot(TM) 64-Bit Server VM  or Java HotSpot(TM) Server VM   ?

Metrics logging not showing anything useful. Suggest disable that for now. You should gain two outcomes from this:
-take the overhead of logging away recording two log files (metrics and coldfusion-out)
-with less other detail in coldfusion-out.log perhaps something more pertinent to situation will be easier to notice in that log

CF10 tomcat is also logging with the Host Valve enabled. Anything useful to gain from the CF10\cfusion\runtime\logs\localhost_access_log.2013-02-25.txt? If you are not getting any useful diagnosis from Host Valve log perhaps turn that off so it does not have any overhead effecting your overall performance.

EG server.xml part becomes removing Valve:

<Host name="localhost" autoDeploy="false" unpackWARs="true" appBase="webapps">
</Host>

Consider the Connector part of your server.xml:

<Connector port="8012" protocol="AJP/1.3" redirectPort="8445" maxThreads="50" tomcatAuthentication="false">

Tomcat documentation says:

http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html

The maximum number of request processing threads to be created by this Connector, which therefore determines the maximum number of simultaneous requests that can be handled. If not specified, this attribute is set to 200.

Any reason why you have tuned Tomcat AJP Connector down from default (200 to 50)? I would think in your situation more than default would be better. EG:

<Connector port="8012" protocol="AJP/1.3" redirectPort="8445" maxThreads="400" tomcatAuthentication="false">

Keep in mind the earlier posting by Anit refer to:

http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

You could do well to add "time out" settings via server.xml and workers.properties. Also for workers.properties you can add worker.cfusion.connection_pool_size = 400 .

CF requires restart to apply server.xml change. IIS & WWW require restart to apply workers.properties.

ISAPI logs look fine. Some web sites more busy than others.

Well a lot there. In short:
-is CF10 Java 32 or 64 bit?
-disable Metrics logging
-with Metrics off any useful details in coldfusion-out.log?
-anything useful from Valve log?
-nothing to gain from Valve turn it off
-alter server.xml to more than default maxThreads="400"
-ISAPI logs are OK

Then latter on another day:
-pending outcome from above consider "time out" for server.xml and workers.properties and other changes to workers.properties for pool.

HTH, Carl.

[BKBK]

gammajring wrote:

My JVM Heap sizes are now set to Min = 1024 and Max = 2048

To be on the safe side,  I would, for 32 bit Java, use Min = 1024 and Max = 1024.