ColdFusion 10 intermittent "service not available"

[Ripples-NhPkgs]

Have been programming over 30 years and CF in its various forms since 1998, but have to say this upgrade is NOT straightforward and after 3 very frustrating days I've finally reverted to CF9.

CF10 Standard upgrade from CF9. Running on dedicated IIS 7.5 on WIN 2008 R2 Server. All 64bit.

Upgrade installation worked correctly. RDS was selected as was the Upgrade all IIS sites.

Initially CF10 failed to start and checking IIS showed the connectors had NOT been installed. CF10 Administrator just showed raw machine code.

Used the CF10 Web Server Configuration tool on "ALL Sites" - no difference.

Removed "All sites" and then installed the connectors against individual sites. This worked - but then I noticed each site was running the CF9 connectors!

Removed each. Then ran the CF9 Web Server Configuration tool to remove all CF9 connectors. Confirmed they had been removed in IIS, then ran CF10 Web Server Configuration tool to add the connectors and all worked well.

This morning I rebooted the server to confirm CF had "installed" correctly, 3 of my 5 sites too 10 minutes before they could be accessed. Then after 30 minutes I'd get a simple white screen with "service unavailable".

Can't afford to have our sites and clients sites intermittent so have reverted to the CF9 connectors and took CF10 offline.

Hopefully someone else has had similar experience and can shed some light on this issue.

Thanks

Peter

[Charlie Arehart]

Lee and Byron, you may have what Byron says are the same issues, but your solutions may be different.

First, Lee, you say you just applied update 7. Had you applied any CF10 updates prior to that? If so, had you re-run the web server configuration tool (to remove and re-add your connections)? That’s mandatory for updates 3-5.

What many often don’t notice, though, is that if they may have “skipped” those updates and just applied a later one (like the latest, 7), then they STILL have to remember to rerun the web server config tool after that later update. The technote and update text (in the updates feature) do “recommend” it, but the challenge is that they don’t require it because technically it’s not required if you’re going from, say, update 6 to update 7.

But again, if you are either applying this as your first update, or had skipped one of those earlier ones, or had applied them but had NOT rerun the web server config tool, then you are currently running with the old connector. And THAT could be the cause of your problem.

Look at the isapi_redirect.dll file in each subdirectory of C:\ColdFusion10\config\wsconfig\ (meaning, C:\ColdFusion10\config\wsconfig\1, C:\ColdFusion10\config\wsconfig\2, and so on). Each of these represents a connection of IIS to an instance. You may have only one, you may have many such numbered subfolders. Look at the date of that file: if it’s not dated November 8 2012 (or later) then it’s not been updated. The connector in each subfolder must be updated.

(And you run the tool for each instance from within that instance, such as C:\ColdFusion10\cfusion\instance1\bin\wsconfig.exe. Again, don’t forget to launch the tool with “run as administrator”, if on Windows Vista, 7 or 2008.)

Finally, Byron, the reason I have proposed you may have the “same issue but different solution” is that you say you “Installed CF10 on a fresh win 2k r2”. If by that you mean you had not applied any updates at all, then you have 2 things to do: apply the latest update (7) and then run the web server config tool to reconfigure any connections you may have made during the installer.

Let us know if this may have helped either of you (or other readers, seeing this in the future.)

/charlie

[Lee_Bartelme]

I think on our dev box, it made me do this when updating to hot fix 6. I don't remember specifically doing this on our prod box.  I tried searching for wsconfig.exe but didn't find it. Then I looked on adobes site on how to do this and the steps were more than confusing. I have no clue if I completed this step. I restarted the machine hoping it would complete this for me.  I'll check the date on the file and if it's older than what you said, look into doing that step again.  I swear my dev box prompted me to do this, but I don't remember the prod box doing so. If I remember right, the wsconfig tool just opens a single box witha single field right. I have no idea what I need to do at with the tool as there is little/no documentation on the subject. Can you point me in the right direction. I think on the dev box I just left whatever was in there and moved on.

[Charlie Arehart]

No, Lee, it will not “complete it for you”. And I agree that if one looks at the CF docs (in the “Admin and Config” as well as “Install” guides) there is no real step-by-step guide to running the web server config tool. That is surprising. It focuses only on how to “start” it, not how to “use it”. I suppose some could argue, “it’s a gui, so it’s obvious”, but I would agree that someone not familiar with it would not know for sure how to work with it, and what all the buttons are about.

(And those who think Lee is being whiny, consider also that someone who merely installed CF and chose to connect it to IIS at the time would NOT have previously seen this web server config tool. They only see it now, when the CF10 update calls for them to re-build the web server configuration. So this problem is perhaps more significant than it’s ever been for CFers without lots of CF admin/config experience.)

But yes, Lee, you would need to run it (with that “run as administrator” option, if you see one when you right-click on it), and you would remove and then re-add whatever connections you see (taking note of what they were, if more than one, in terms of what “site” was connected to what “instance”, again if you run more than one instance of CF, as is supported in the Enterprise/Trial/Developer edition). A restart of CF won’t do it for you.

Let us know if that helps.

/charlie

[Lee_Bartelme]

I updated the connectors last week, just had a crash 5 minuets ago.

As a side note, I added a scheduled task on my development machine that would "ping" each of our applications every 20 minuets. If the server returned anything other than a 200 status, I have it send me an email. I had this script running for a for a few weeks before I updated my connector and the server never crashed. I stopped the scheduled task on the same day I updated the connector, wanting to see if it was my task that was keeping the server alive, or if this config would truly fix the issue.

Oddly enough, when the application pools just stopped, I do not see the same getRealPathFromConn() error. This is in my error log at the time of the event:

Feb 19, 2013 2:40:56 PM org.apache.catalina.core.StandardServer await

INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance.

Feb 19, 2013 2:40:56 PM org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["ajp-bio-8012"]

Feb 19, 2013 2:40:56 PM org.apache.catalina.core.StandardService stopInternal

INFO: Stopping service Catalina

Feb 19, 2013 2:41:58 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

Feb 19, 2013 2:42:01 PM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-bio-8012"]

Feb 19, 2013 2:42:01 PM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Feb 19, 2013 2:42:01 PM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.23

Feb 19, 2013 2:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

Feb 19, 2013 2:42:13 PM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 20.4-b02

Feb 19, 2013 2:42:21 PM org.apache.catalina.core.ApplicationContext log

INFO: CFMxmlServlet: Macromedia Flex Build: 87315.134646

Feb 19, 2013 2:42:26 PM org.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler ["ajp-bio-8012"]

Feb 19, 2013 2:42:27 PM com.adobe.coldfusion.launcher.Launcher run

INFO: Server startup in 41013 ms

I don't see an error. The first event there I believe (valid shutdown command) is when I hit restart on the windows machine. No errors. So what the heck? I had previously increased my IIS rapid fail to 50 and in event viewer of course there are 50 of them. Basically these two over and over again:

• A process serving application pool 'domain.com' suffered a fatal communication error with the Windows Process Activation Service. The process id was '4924'. The data field contains the error number.
• A process serving application pool 'domain.com' terminated unexpectedly. The process id was '10336'. The process exit code was '0xff'.

And at 2:36 Application pool 'domain.com' is being automatically disabled due to a series of failures in the process(es) serving that application pool.

There are no coldfusion errors thrown today at all before 2:40, essentially when I restarted the server. Nothing in the exception log. I feel like all my CF update did was hide it from the log as the problem remains. I've turned my development scheduled task again in an attempt to keep the applcations running hoping there will be no issues. Any thoughts on this. I've been doing a lot of reading and it seems that the IIS 7.5, Win Server 2008, CF10 combo is entirely unstable.

The isapi_redirect.dll located in Coldfusion10\config\wsconfig\1 is indeed dated 11/8/2012 1:18 PM.

[carl type3]

Lee, Maybe some of the CF10\config\wsconfig\1\isapi_redirect.log around 2:40pm to 1 hour before would be interesting?

Probably less so but still could be good to check anything of concern in that time frame for CF10\cfusion\logs\coldfusion-out.log?

HTH, Carl.

[Lee_Bartelme]

So in the isapi_redirect.log there are alot of these at 14:34. And I mean A LOT, but they exist throughout:

• [Tue Feb 19 14:34:02.404 2013] [9372:10556] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
• [Tue Feb 19 14:34:02.404 2013] [9372:10556] [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly
• [Tue Feb 19 14:34:02.420 2013] [9372:10556] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
• [Tue Feb 19 14:36:04.818 2013] [12000:11812] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250 (LESS OF THESE)

After the restart sets of these:

• [Tue Feb 19 14:41:00.206 2013] [9720:11712] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping
• [Tue Feb 19 14:42:03.081 2013] [2944:2988] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
• [Tue Feb 19 14:42:28.805 2013] [2624:3044] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250

There is nothing in the outlog that looks out of the ordinary or that looks like an error.

[carl type3]

Lee, case it helps here is an extraction of what ISAPI log looks like while working normally take from CF stop to load commencing.

[Tue Feb 19 12:10:34.919 2013] [2596:1912] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping
[Tue Feb 19 13:25:26.442 2013] [5888:5868] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
[Tue Feb 19 13:25:26.473 2013] [5888:5868] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
[Tue Feb 19 13:25:26.677 2013] [5888:5868] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250
[Tue Feb 19 13:25:30.677 2013] [5888:3148] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 2 of max reuse connection 250 and total endpoint count 250
[Tue Feb 19 13:25:30.692 2013] [5888:5868] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 3 of max reuse connection 250 and total endpoint count 250
[Tue Feb 19 13:53:50.503 2013] [5888:3124] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (3), cping (0), send (0)
[Tue Feb 19 13:59:04.607 2013] [5888:2804] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 4 of max reuse connection 250 and total endpoint count 250
[Tue Feb 19 13:59:05.060 2013] [5888:3124] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 5 of max reuse connection 250 and total endpoint count 250
[Tue Feb 19 13:59:05.107 2013] [5888:4892] [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 6 of max reuse connection 250 and total endpoint count 250

> [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
I would think that odd if a stop did not proceed it. Did IIS WWW CF stop?
> [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly
"Error" usually means there is a problem that needs to be resolved. I have not come across this one yet. Suggest follow that error detail perhaps will lead to solution.
> [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized
There we have part of a start-up process.
> [info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 1 of max reuse connection 250 and total endpoint count 250
Normal usage. Depending on load value for "reuse count is" will increment.

While you can get more ISAPI log details via changing isapi_redirect.properties altering log_level=info to debug I would not recommend it in a production environment. If you know what the process is that leads up to crash and can simulate that process occurring then log_level=debug can be useful.

Again, Carl.

[carl type3]

@Lee
>INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path

I don't think that will necessarily be causing a problem and you may not be using APR. Find information about APR here:

http://tomcat.apache.org/native-doc/index.html

You could download APR extract tcnative-1.dll (take care there is Win 32 and 64 bit) and copy that to CF10\cfusion\lib then restart CF10 Application Service. You will notice the log details change to EG:

org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library
org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false],
random [true].
org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-apr-8012"]
etc

[Lee_Bartelme]

After Googling " [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly" I have come across others with the same issue dealing with the redirect.dll file. They speak of patching the file, but I don't think that is an option for me, as this dll should have to come from Adobe/CF correct?

Should I be running my applicatin pools with Enable 32-bit set to true. They are set to false currently. This really seems like an Adobe CF/IIS issue doesn't it. What the hell is going on. I read in one thread they think this is happening durring applicatin recycle at some point or possibly due to load. My scheduled task that hits my applications every 20 minuets I think keeps all application/worker processes from shutting down, thus keeping the application pools alive, but my uneducation may prove it's all in my head.

I have no idea if we are using APR or how to even tell. Should I just download the file and see what happenes. Is there some sort of drawback to doing so? Any harm that can be done? My thinking is this is a load issue as it NEVER happens on our development machine.

[carl type3]

As you say we can’t just go get the current Tomcat Connector (1.2.37) extract ISAPI DLL and use that. CF have given us a modified 1.2.32 Connector and I guess CF Dev team have good reasons for doing so.

Having an error is a help, finding what the error really means can be a bit like following bread crumb trail. What to recommend I am not sure yet. Perhaps your workers.properties syntax would be interesting. I have been using JMX with jconsole to monitor load at the CF end of the Connector and found that helpful. Let me know if you want more details on that.

Apply APR DDL if you want. I have on some CF10 environments with APR DLL applied but for now I think time spent on APR will distract from what the real problem is. Perhaps another day. I noticed it in the thread and so thought I would pass on what I know.

[Lee_Bartelme]

Thanks for your input. I'm a programmer assigned to also keep the server running if you know what I mean, so my skills as far as a Server Admin are pretty basic. I know what I am doing to get things done, but as far as real debugging/troubleshooting on the server side of things, I am lacking. I feel like my only solution is to wait for the next CF patch. I think my scheduled task keeps things running for now, and while not ideal, the server hasn't crashed while running the script yet.

[carl type3]

Lee - know where your coming from.

With the ISAPI log mention of "error" and "load balance" curious what is syntax of CF10\config\wsconfig\1\workers.properties EG:

worker.list=cfusion

worker.cfusion.type=ajp13
worker.cfusion.host=localhost
worker.cfusion.port=8012
worker.cfusion.max_reuse_connections=250

Read this informative blog, I know there is a lot there:

http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

With your mention of running schedule task to keep things running that can point to timeout issues. You might do well to add time out syntax. IIS WWW and CF need restarting to apply changes. EG

workers.properties add -
worker.cfusion.connection_pool_timeout = 60

server.xml CF10\cfusion\runtime\conf alter AJP section -
<Connector port="8012" protocol="AJP/1.3" redirectPort="8445" tomcatAuthentication="false" connectionTimeout="60000" />

HTH again, Carl.

[Lee_Bartelme]

My workers.properties looks exactly as you have displayed. Should I try to corespond my connection_pool_timeout with my application timeout in my CF applications? Is 60 = 60 minuets? I'll read through your link in the meantime and hopefully it will answer my question of if I should have to edit the XML and the workers.properties or if it's an either or type of thing. Thanks again for your expertise.

[carl type3]

workers.properties connection_pool_timeout = 60 is in seconds where as server.xml AJP connectionTimeout="60000" is in milliseconds. General recommendation is the time outs should match and references offer 60 seconds or 600 (10 minutes) for the ISAPI pool and AJP threads. I would not set to 60 minutes however as always do your own testing to know what value suits best. You are welcome - but I am no expert.

[carl type3]

@Lee it has been a while. How is that going for you? Regards, Carl.

[Lee_Bartelme]

I'm afraid to jinx it all. I did the changes discuessed above last Monday, (8 days ago) and haven't had a crash yet. One of our applications did crash, but it was an unrelated issue that we've had in the past. So I had to restart the server anyway, but I didn't want to declare a win until it had at least been 2 weeks without a server restart. However I am cautiously optomistic.

I am pretty pissed that I was required to do this and Adobe itself didn't guide me to this solution. I know they changed their underlying technology with the CF10 update, and you can't predict all issues, but having IIS 7.5 running multiple sites under the default ALL connection seems like it would be pretty common and an easy bug to find. I hope in a couple of weeks I can mark my issue as fixed.

[carl type3]

Agree still too early to celebrate but even a few days of uptime is good considering the fail rate. Keep us posted good or bad, I got a few more tricks when it comes to CF10 tomcat connectors. Cheers, Carl.

[Lee_Bartelme]

So this morning at 7:20 event viewer is showing a series of Wanring: process servering application pool blah blah errors. I didn't even know this happened as I guess it didn't happen more than 50 times in a quick enough span to trigger the Rapid Fail thing in IIS. A coworker asked me about it as he was receiving 503 errors during the almost hour the application pool was stopped, but I guess it worked itself out and everything was back up and running by 8 when I got in.

isapi_redirect log just has a series of :

• [Tue Mar 12 07:20:00.047 2013] [8852:9092] [info] ajp_send_request::jk_ajp_common.c (1658): (cfusion) all endpoints are disconnected, detected by connect check (2), cping (0), send (0)
• [Tue Mar 12 07:20:19.297 2013] [8852:4708] [info] TerminateFilter::jk_isapi_plugin.c (2343): Jakarta/ISAPI/isapi_redirector/1.2.32 () stopping
• [Tue Mar 12 07:20:20.093 2013] [7924:3552] [info] init_jk::jk_isapi_plugin.c (2634): Starting Jakarta/ISAPI/isapi_redirector/1.2.32 ()
• [Tue Mar 12 07:20:20.108 2013] [7924:3552] [error] init_jk::jk_isapi_plugin.c (2779): Initializing shm:(null) errno=-1. Load balancing workers will not function properly.
• [Tue Mar 12 07:20:20.108 2013] [7924:3552] [info] init_jk::jk_isapi_plugin.c (2813): Jakarta/ISAPI/isapi_redirector/1.2.32 () initialized

I have two getRealPathFromConn exceptions in the CF-error log but these happened at 8:00. The IIS warnings started at 7:20 and stopped at 8:05.

On one hand I am happy the pools didn't stop indefinetly but on the other hand it's obvious the problem is still not fixed. Even stranger, I went skiing yesterday and since I know I was going to be away, I turned on my script that "seemingly" was keeping the applications alive by pinging them every 20 minuets... Obviously that was an illusion that it was helping anything.

I am so close to going back to CF9 on this. If it wasn't such a pain to switch both our servers, I'd probably have already done it by now.

[carl type3]

Lee so ISAPI log still has mention of "error" and "load balance". At least that is consistent. I have still not seen this elsewhere but figure that is only clue as to problem. I gather symptom has changed where by things ran slow appeared crashed but recovered in about 1 hour period.

Would be interesting to know how much memory and cpu was being consumed by coldfusion.exe via Windows TASKMANAGER and if that was different to other times when system is working normally.

So readers and followers of matter are all on the "same page" perhaps by way of “sanity check” how your (1) workers.properties syntax  (2) server.xml AJP section (3) jvm.config JVM arguments portion (4) CF10 patch level from CF10admin I button (5) properties file size and date of ISAPI.DLL would all be good to know EG:

1) CF10\config\wsconfig\N\ workers.properties -

worker.list=cfusion

worker.cfusion.type=ajp13

worker.cfusion.host=localhost

worker.cfusion.port=8012

worker.cfusion.max_reuse_connections=250

worker.cfusion.connection_pool_timeout = 600

2) CF10\cfusion\runtime\conf\ server.xml AJP section -

<Connector port="8012" protocol="AJP/1.3" redirectPort="8445" tomcatAuthentication="false"

connectionTimeout="600000" />

3) CF10\cfusion\bin\ jvm.config Arguments to VM section -

java.args=-server -Xmx512m -XX:MaxPermSize=192m -XX:+UseParallelGC -Xbatch -D "not worry about the remainder for now"

4) CF10admin System Information 

Server Product  ColdFusion 

Version  10,0,8,284032 

Tomcat Version  7.0.23.0 

Edition  Standard    

Serial Number  deleted

Operating System  Windows Server 2012   

OS Version  6.2   

Update Level  /D:/ColdFusion10/cfusion/lib/updates/chf10000008.jar   

Adobe Driver Version  4.1 (Build 0001)   

JVM Details 

Java Version  1.7.0_15   

Java Vendor  Oracle Corporation   

Java Vendor URL  http://java.oracle.com/ 

Java Home  d:\ColdFusion10\jre   

Java File Encoding  Cp1252   

Java Default Locale  en_US   

File Separator  \   

Path Separator  ;   

Line Separator  Chr(13) 

User Name  SERVER$   

User Home  D:\   

User Dir  d:\ColdFusion10\cfusion\bin   

Java VM Specification Version  1.7   

Java VM Specification Vendor  Oracle Corporation   

Java VM Specification Name  Java Virtual Machine Specification   

Java VM Version  23.7-b01   

Java VM Vendor  Oracle Corporation   

Java VM Name  Java HotSpot(TM) 64-Bit Server VM   

5) CF10\config\wsconfig\N\isapi_redirect.dll  properties -

380 KB (389,120 bytes)

Date modified 8/11/2012

One other thing. Can you PM me I want to send you a slide deck that I am preparing on CF10TOMCAT. It is work in progress so for now I do not want to share more widely yet.

Regards, Carl.

[Guest]

This problem is plaguing me too.  I know it is related ot the ISAPI Connector.  We have a lot of IIS->Tomcat installations using the connector.  The 1.2.32 version caused us similar stability problems and we had to upgrade.  It seems strange that they aren't issuing udpates or supplying the code so that we can patch.  If you paid close attention to the Rollup package 5, it replaces the isapi_redirect.dll file if you rerun wsconfig.  They left the version number the same, but the file size is different by 1byte.  This update patched a problem that would cause the connector to crash on large XML posts.  I hope there is an answer soon.

[Guest]

Could having ASP.net 4 installed play a role in this.  I have at least one server that hasn't crashed and it doesn't have .net 4 installed.  This server is not server a lot of content, so I wasn't certain if .net 4 is a related issue.

[Lee_Bartelme]

We've got a development server that has never crashed once. I assume it's a load issue. If your sites don't serve a lot of users you may never have a problem. My two servers are identical in almost every way except for load and number of running sites. We have 20+ sites on the production server and only 4 on the development server. I have since followed more of Carl's suggesstions, namely increasing memory and PermSize in the jvn.config as well as setting max reuse and min connection pool size. I did the change on the 13th and so far we've gone 36 hours or so without incident. The longest I've run since this started without incident is 7 or so days. If I make it past 2 weeks I'm going to celebrate. (I believe ASP.NET was always installed on our server, but shortly before our server ever crashed we did install a 3rd party ASP.NET web application. It runs on it's own application pool however, and once we set it up it has never crashed. It does however pull a single resource (an image) from the website that is running in the pool that fails... This asp.net web application IS NOT running on our development server.... Hmmmm....)

[Guest]

Our server that keeps crashing is a staging server.  Hardly any use at all.   The difference between our Dev and Stage is Dev has ASP.net 2 and Stage has ASP.net 4.  ASP.net 4 adds two ISAPI filters and a bunch of handler mappings.  The interesting thing is that .net 4 adds wildard handler mappings *.  So everything will pass through that handler mapping, not just asp pages.  I tried removing the handler mappings to see if that makes a difference. 

[Lee_Bartelme]

That is very interesting. How can I tell what version of ASP.net I have installed? The application we use that is ASP.net is not crucially important and may go away. I would almost be willing to uninstall/disable ASP.net or whatever I can need to do to test it. If that is indeed the problem, Adobe needs to step up to the plate and freaking do something. Please let me know if this indeed fixes your problem. Like I said I made some changes 3 days ago. and if that appears to fix the issue I will surely report back. Either way, whatever is the cause Adobe needs to address it and create an actual fix apart from uninstalling something as crucial as ASP.net or guess configuring various files on the server. CF10 has been out for a while to have wide spread problems like this in my opinion.

[isites-85260]

Hi, I have noticed this thread went quiet.  I'm wondering if any of you worked out the issue with the connector (Tomcat) hanging.  In my case it's running on IIS 7.5.  Happens like clockwork when McAfee site scan hits the site which uses up a ton of connections in no time flat.  I always have to restart services to bring it back.  I have update 9 installed and have checked the connector versions.  If I'm correct, the 64 bit connector hasn't changed since before update 5 (380KB 11/8/12 1:18PM).  I am trying to increase the max re-use connections (500) to see if that helps but it really seems as if Tomcat is just not cutting it and we too are cosidering NOT moving forward with deployments on CF10 and sticking with JRUN and CF9, which runs like a dream BTW.

If any of you have found the solution or a decent work around let me know?  Did it wind up being ASP.NET 4? Eager to hear back!

-Matt