ColdFusion 10 intermittent "service not available"

Explorer ,
Jun 14, 2012 Jun 14, 2012

Copy link to clipboard

Copied

Have been programming over 30 years and CF in its various forms since 1998, but have to say this upgrade is NOT straightforward and after 3 very frustrating days I've finally reverted to CF9.

CF10 Standard upgrade from CF9. Running on dedicated IIS 7.5 on WIN 2008 R2 Server. All 64bit.

Upgrade installation worked correctly. RDS was selected as was the Upgrade all IIS sites.

Initially CF10 failed to start and checking IIS showed the connectors had NOT been installed. CF10 Administrator just showed raw machine code.

Used the CF10 Web Server Configuration tool on "ALL Sites" - no difference.

Removed "All sites" and then installed the connectors against individual sites. This worked - but then I noticed each site was running the CF9 connectors!

Removed each. Then ran the CF9 Web Server Configuration tool to remove all CF9 connectors. Confirmed they had been removed in IIS, then ran CF10 Web Server Configuration tool to add the connectors and all worked well.

This morning I rebooted the server to confirm CF had "installed" correctly, 3 of my 5 sites too 10 minutes before they could be accessed. Then after 30 minutes I'd get a simple white screen with "service unavailable".

Can't afford to have our sites and clients sites intermittent so have reverted to the CF9 connectors and took CF10 offline.

Hopefully someone else has had similar experience and can shed some light on this issue.

Thanks

Peter

Views

42.5K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
replies 101 Replies 101
Explorer ,
Jun 16, 2012 Jun 16, 2012

Copy link to clipboard

Copied

Update for others hitting issues with CF10 and IIS on Win 2008 R2 64 Bit.....

Thought I'd try a clean install and not an upgrade to CF9. I restored the server boot drive to a pre CF10 4installation state. Unistalled all connectors then unistalled CF9. Rebooted, then installed a clean version of CF10.  Hit some IIS issues as the default document and cfm connectors did not flow down to "all sites". I corrected by 'resetting to parent' for these settings. Then manually configured settings in CF Admin.

All worked correctly UNTIL I rebooted and then hit the "service not available" again with a footnote of  "jakarta/isapi/isapi_redirector/1.2.32"

Searching Google I found other users hitting the same issue in that this Tomcat connector fails to load normally on IIS start. Not sure if its a 64bit issue, but as I mentioned this server is running 64bit Win 2008R2, IIS 7.5 and CF10 64 bit. I'd have thought this was a very common server configuration.

CF9 64 bit had no issues.

Sadly I see no responses from Adobe.

Have given up with CF10.

Hopefully Adobe can come up with a fix soon!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Jun 18, 2012 Jun 18, 2012

Copy link to clipboard

Copied

Had similar issues and attempted install/reinstall.  Found that the Handler Mappings were completely wrong.  Changed them to point to the correct location and all the problems disappeared. 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jun 18, 2012 Jun 18, 2012

Copy link to clipboard

Copied

Thanks Stephen!

I also found mappings were still pointing at CF9 connectors after the standrad installtion and changed them using the CF10 Connector Tool. Confirmed they were correctly pointing to CF10 connectors and all worked well until a reboot when the connectors intermittently "dropped out" and I get the "service unavailable" screen.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jun 20, 2012 Jun 20, 2012

Copy link to clipboard

Copied

Update 6/20/2012

Have been troubleshooting this connector issue daily unfortunately still without success.

Have tried the upgrade from CF9 to CF10. Removed CF9 completely and installed a fresh copy of CF10. All with the same intermittent issue that the connector drops out on some web sites after a reboot. I have confirmed in IIS that the default documents include index.cfm and the handlers for CFM etc are pointing to the new Tomcat redirect connector.

Sites work perfectly once setup, but one or two will drop out after every reboot. Mostly they just hang with a blank page showing. Sometimes I'll get "service unavailable".

If I look at my CF logs there no error messages. If I look at my IIS logs it just shows a huge delivery time while I get the connector renewed for that site.

There's only 4 sites on this server and the failed ones can be any. No consistency there.

Have tried removing all connectors in CF9 before installing CF10 and running the CF10 connector tool but still get the same results.

Tried installing the All Sites connector - same result. Tried individual site connectors with and without the All Sites Connector - same result.

Have confirmed I can get the failed site up and running by using the connector tool to remove the bad site connector and then re-install it. Site remain good until a reboot.

Restarting the WWW Publishing Service does not cause any problems

Restarting all CCF10 Services also does not cause any problems.

Install was from ColdFusion_10_WWEJ_win64.exe dated 5/3/2012 that came direct from Adobe on disc - so can't blame corrupt download.

As I mentioned before, I know CF very well having been programming full time with it since 1998. I always keep the server updated to the latest software and while there's always quirks to work around, I confess this one is proving very frustrating.

Hopefully Adobe will step in soon!

Peter

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Jun 20, 2012 Jun 20, 2012

Copy link to clipboard

Copied

Knock on wood, but I haven't experienced this issue yet.  My guess is that it is more the Apache integration.  Are all the services restarting?  If this is happening repeatedly, I would file a bug over at https://bugbase.adobe.com/

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jun 20, 2012 Jun 20, 2012

Copy link to clipboard

Copied

Thanks again Stephen!

I did raise a bug report #3216317

First response said it could not be reproduced.

I did find a similar issue at http://forums.adobe.com/message/4465029 I also see lots of "nul null error in the coldfusion-error.log but seems to be a different cause

+++++++++++++++++++++

Jun 20, 2012 1:19:54 PM org.apache.tomcat.util.http.Parameters processParameters

INFO: Invalid chunk starting at byte [9] and ending at byte [9] with a value of [null] ignored

+++++++++++++++++++++++++++++++++++

However these errors are continuous whereas the site stall occurs immediately after a reboot and is cleared by rerunning the connector tool for CF10 on the bad site.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Aug 03, 2012 Aug 03, 2012

Copy link to clipboard

Copied

 

UPDATE 8/3/2012

I spent the past week trying to troubleshoot this issue in a little more depth.

Have used the Web Server Configuration Tool in all combinations of "ALL" or individual sites or both. If I remove all connectors and start afresh, selecting "ALL" does not always flow down the CF file Handlers to the individual sites. Only way this seemed to work is if I reset the handlers for each individual site to "revert to parent".

However, even when connectors could be confirmed to exist correctly through IIS Manager, this intermittent failure for all sites to start after a reboot continued.

After much testing I've found I can resolve the problem every time by waiting a couple of minutes after a reboot and then restarting the IIS Service. Subsequently the server works well. CF10 is fast and I see no issues.

I did find a couple of entries in the C:\CF10\cfusion\logs\coldfusion-error.log that may offer a clue

+++++++++++++++++++++++++++++++++++++++++++++++++++

 

During CF Start

several....

*******************************

 

Jul 31, 2012 1:17:51 PM org.apache.tomcat.util.http.Parameters processParameters

INFO: Invalid chunk starting at byte [0] and ending at byte [0] with a value of [null] ignored

********************************

 

and....

*********************************

 

Jul 31, 2012 1:22:23 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\CF10\\cfusion\lib;C:\\CF10\\cfusion\jintegra\bin;C:\\CF10\\cfusion\jintegra\bin\international;C:\\CF10\\cfusion\lib\oosdk\classes\win

**********************************

 

and....

**********************************

 

Jul 31, 2012 1:22:36 PM com.adobe.coldfusion.launcher.Launcher run

INFO: Server startup in 14761 ms

Error: Could not load mediaLib accelerator wrapper classes. Continuing in pure Java mode.

Occurs in: com.sun.media.jai.mlib.MediaLibAccessor

com.sun.media.jai.mlib.MediaLibLoadException

***********************************

 

and several times.....

***********************************

 

Jul 31, 2012 2:15:47 PM org.apache.coyote.ajp.AjpProcessor getRealPathFromServer

SEVERE: Error in getRealPathFromConn

java.net.SocketException: Connection reset by peer: socket write error

at java.net.SocketOutputStream.socketWrite0(Native Method)

at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)

at java.net.SocketOutputStream.write(SocketOutputStream.java:124)

at org.apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:391)

at org.apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1112)

at org.apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)

at org.apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)

at org.apache.catalina.core.StandardContext.getRealPath(StandardContext.java:4475)

at org.apache.catalina.core.ApplicationContext.getRealPath(ApplicationContext.java:414)

at org.apache.catalina.core.ApplicationContextFacade.getRealPath(ApplicationContextFacade.java:335)

at coldfusion.runtime.ServletContextWrapper.doGetRealPath(ServletContextWrapper.java:179)

at coldfusion.runtime.ServletContextWrapper._doGetRealPath(ServletContextWrapper.java:116)

at coldfusion.runtime.ServletContextWrapper.getRealPath(ServletContextWrapper.java:98)

at coldfusion.runtime.TemplateProxyFactory.getFullName(TemplateProxyFactory.java:1092)

at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:202)

at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)

at coldfusion.runtime.TemplateProxyFactory.resolveFile(TemplateProxyFactory.java:120)

at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:138)

at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:84)

at coldfusion.runtime.AppEventInvoker.<init>(AppEventInvoker.java:64)

at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:226)

at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)

at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)

at coldfusion.filter.PathFilter.invoke(PathFilter.java:112)

at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)

at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)

at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)

at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)

at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)

at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)

at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)

at coldfusion.CfmServlet.service(CfmServlet.java:204)

at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)

at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)

at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:414)

at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:203)

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)

at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

***************************************

I

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Aug 05, 2012 Aug 05, 2012

Copy link to clipboard

Copied

Re Durring CF start:
org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\CF10\\etc

Was discussed in this post:

http://forums.adobe.com/message/4446569#4446569

Maybe the details in that post to resolve that message will partly provide solution to your issue.

HTH, Carl.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Aug 06, 2012 Aug 06, 2012

Copy link to clipboard

Copied

Thanks Carl,

I appreciate the pointer. I did come across the post during my search for any clues, but following the conclusion that the library wasn't necessary I decided to not to complicate the CF10 install.

My current concern was the "SEVERE: Error in getRealPathFromConn" entries which I'm hoping someone out there may know what these are.

Thanks again

Peter

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Aug 06, 2012 Aug 06, 2012

Copy link to clipboard

Copied

Peter I recall seeing Jrun (CF6 7 8?) and getRealPathFromConn errors before but have not directly been involved with matter so it does not come to mind as to solution. Perhaps if you search without tomcat details you might find a useful lead since CF has run Jrun for longer than tomcat.

HTH again, Carl.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Aug 10, 2012 Aug 10, 2012

Copy link to clipboard

Copied

Thanks Carl a good lead... Coding error on undeclared variable caused the "SEVERE: Error in getRealPathFromConn" entry in the log. Despite clearing this error the failure of all sites to start after reboot persists. CF Logs show no errors duing or after the latest reboot. On the last reboot 2 out of 4 sites 'hung'. Tried the usual IIS service restart, but 1 site still hung. Shut down IIS service then restarted CF10 service which brought all 4 sites back up.

I do note that stopping the CF10 service takes considerably longer than previous CF version. Is this normal?

Peter

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Aug 12, 2012 Aug 12, 2012

Copy link to clipboard

Copied

Hi Peter

Have you figured the way to resolve your issue with cf10 yet? Or you have reverted back to cf 9. It seems we may face the similar issue. Can you please check your system error log to see whether you got some similar erroes as I did. I have posted the detail in

http://forums.adobe.com/message/4609125

My production server is really unstable at the moment and this forcing me to monitor the server 24/7 in case it hangs.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Sep 17, 2012 Sep 17, 2012

Copy link to clipboard

Copied

Ryan,

Sorry for the slow response...

After several weeks trying to resolve this CF10 problem full time the final staw came when the Tomcat Service crashed twice in one day leaving my customers and clients staring at an error page. Not to mention the loss in revenue.

I therefore restored CF9 earlier this month and am once again running on a stable configuration.

I received no response from Adobe as to what may be the cause, but would warn anybody considering upgrading to CF10 on a 64 bit Windows OS. It could be that a clean install is fine, but at this stage I can't see the improvement over CF9 worth the disruption of starting with a clean OS.

As I previously mentioned, I've been running CF since 1998 and this is the first issue I've been unable to resolve.

Peter

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Sep 17, 2012 Sep 17, 2012

Copy link to clipboard

Copied

Peter,

Thank you for your response.

There are two bugs which might associate to our issue have been submitted to Adobe and I believe they are all confirmed by Coldfusion Team. As far as I know, they are working on it at the moment, but unfortunately they are unable to put the fix in hotfix 1 they have released early this month or late last month.

https://bugbase.adobe.com/index.cfm?event=bug&id=3318104

https://bugbase.adobe.com/index.cfm?event=bug&id=3222748

We are waiting for the fix from Adobe, since we are not going to rollback to where we came from.

My temperate solution is increasing the number of error IIS can handle before shut itself off.

It is in application pool > Advance setting > Rapid-Faill Protection - Maximum Failures

Since I have found the IIS will shut itself off after 5 errors been sent from connector, by increasing the number here, it temperately solved my problem (but my event log still full of errors from connector)

Ryan

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 01, 2012 Oct 01, 2012

Copy link to clipboard

Copied

Ryan,

Just wanted to confirm that we experienced the issues with IIS listed in those bugs. I was able to get Adobe's attention via social media and we are testing out new jars that fix the issue with the isapi connector.

We ran for almost 24 hours before having to roll back to our CF9 servers on Friday. (We had to rollback because of this DateConvert() bug.) The new jars appear to fix the connector issue. Hopefully there will be an update soon to address this. But, since the instructions from adobe included dissconnecting sites and running wsconfig after replacing the wsconfig.jar, I'm not sure that a regular update will fix it automatically.

If you want to test the new jars, Rupesh has an offer to send the jars to people who would test them on this bug page in the notes. (rukumar (at) adobe (dot) com)

We have again added the CF10 servers into the pool and they are ramping up. Hope to be fully on CF10 by Tuesday.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 02, 2012 Oct 02, 2012

Copy link to clipboard

Copied

Hi Chris

Thanks for your reply and the information you have provided.

I am afraid I couldn't try the new jar due to the server is in production. I will wait the official patch for cf10 and hopefully it can resolve the issue troubles the community.

Thanks

Ryan

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Sep 22, 2012 Sep 22, 2012

Copy link to clipboard

Copied

Peter, I hear your frustration, and I have some thoughts.

I don't see anyone suggesting this yet: is it possible that this is all down to a permissions issue? I realize that at this point, you have likely uninstalled CF10 and may not care to futz with it anymore, but in case you may, or in case others may find this, here are a couple of suggestions.

First, did you notice how things would be ok until you rebooted? That leads me to think that something that was "changed" was really only changed in memory and could not be written to disk, so that on reboot, the "change" was "lost". I can't right now think what that may have been, but it sounds like a clue worth considering.

One thing I do know is that it may help when running the web server configuration tool if you first right-click on it and "run as administrator". This is one of those vagaries that arises from the increased security in Windows 2008. I do think that sometimes CF is getting an unfair black eye due to things like this.There are other apps which work funky if you don't "run as admin". Some even give no error, but still don't do what you think they should do.

Also, there are important differences in IIS 7 from how things worked before, which could also be related (assuming you are not using IIS 6 Compatibily mode). Some folks may have noticed that a web.config file may be written to the root of your IIS sites which have been configured for CF. That could be created by changes you make in the IIS interface, or by the CF web server config tool (or installer).

Well, as for the latter here's another potential gotcha: perhaps CF doesn't have permissions to that directory. For those who change the user under which CF runs, they may not consider any need to give write permissions for that user to their website docroot. If CF can't write to it, then I could see the web server config tool (or installer) not implementing things correctly.

Not all configurations of CF and IIS require creation of the web.config file, so I do realize some will say they see none there and yet things are working. Put briefly, the differences have to do with if you are or are not running IIS in IIS 6 compatibility mode. If you are not, then it works quite differently than it did before.

First, it seems that if you configure "all sites", CF configures things (like what you see in handler mappings) in an applicationhost.config file in \Windows\System32\inetsrv\config, and any changes from that made for a given site are written to its web.config. And second, if you configure a specific site, then it writes the needed settings to that site's web.config. In either case, again, if there are any challenges writing to that file, then settings won't be persisted. I grant that I'm still guessing a bit here, so I'll welcome thoughts from others.

BTW, this use of the web.config could also explain some of the confusion you were hitting when you had things configured for both CF 9 and CF10. Perhaps you may have had (as I did at one time) a site set to connect using CF9, and another set to connect using CF10, both pointing to the same web site docroot. That was always ok in CF traditionally, to test things on one server with different sites against different versions of CF.

But now (if not using IIS 6 Compat mode) then CF and IIS are using these config files, and they are NOT writing different ones for different versions. If 2 sites point to the same docroot, changes to each basically overwrite the other. It can definitely be confusing. 

And for many people, CF10 is the first release where they are deploying CF on IIS 7. You may know that CF 9.0 and below did not formally support it, but people found they could get it to work by enabling IIS 6 compatibility. And while 9.0.1 added full support for IIS 7, many just continued to use IIS 6 Compat when they upgraded from 9.0 to 9.0.1. I suspect now we have more people going to IIS 7 for the first time with 10, or maybe putting it on new boxes where they did not think to even turn on IIS 6 compat, and so they are experiencing the results of this new approach to things for the first time.

Again, some of what I offer here is speculation. A lot of this is new for all of us. I just wanted to offer these thoughts as I'd not see anyone else do so yet here. I appreciate the severe annoyance some have felt in things seeming "not to work". But I'll just say that there are people who are running CF10 fine, so it is possible to configure things correctly.

This is somewhat like the pain we went through in the move from CF 5 to CFMX (6), where things with Java were quite different and some even gave up on CF (or just never moved up past 5, or not for many years.) Then there's the added differene that CF10 is running on Tomcat rather than JRun, and Adobe did have to create a new connector (and couldn't use the stock one that comes with Tomcat). With so many moving parts, it can be hard to know for sure when something is a "bug", when something is just a misconfiguration, or a misunderstanding, or a failure to set proper permissions, or to run as administrator, and on and on.

As in that time fo the switch to CFMX, there may be growing pains but there will also be growing understanding as more and more people find and resolve problems such as these, and share their knowledge. It's little consolation for those who are on the front lines deploying the new release in production.

But I hope, Peter, that perhaps some of what I've shared here might either help solve problems or at least help get you further down the road to resolving them, again, if and when you may be willing to give deployment of CF10 another shot.


/Charlie (server troubleshooter, carehart.org)

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Sep 23, 2012 Sep 23, 2012

Copy link to clipboard

Copied

Hi Charlie,

Thanks for the detailed response. Well written and very much appreciated.

I do agree that the change between CF9 and 10 is similar to the introduction of CFMX. The complexities of getting this to work across different operating systems must be daunting task to the developer.

I've been messing with code long enough to remember the introduction of PC DOS, MS DOS and Windows. Took a few years before processing power made Windows usable, but I've stuck with that OS in its various forms from those early days. I try to take advantage of the latest OS and software versions wherever possible.

Similarly I've been with CF since '98 (version 2?). Upgrades have gone smoothly, but occaisionally a new server has given me the chance to install a fresh copy onto a fresh copy of Windows and SQL. Normally this would be my fix for this issue - upgrade to new server hardware and carry out a fresh install. Unfortunately time and costs mean this is not currently an option hence sticking with trying to fix this frustrating error for so long.

At the time of upgrade I was running the latest CF9 (9.0.1.274733). Yes, I was in IIS6 compatibility mode, and feel permissions were fine.

Just to clarify, after any reboot 1 or more of my 4 commercial sites would hang until the Tomcat error page appeared. Knowing this I'd check each site immediately after a reboot, then reboot IIS and occaisionally CF10 until all  4 sites appeared.

Initially I found the WEB Configuration Tool made a complete mess of the web.config files, so ended up using the tool to remove all sites and configuring individual but they were still very messy. Eventually I closed down CF10, and spent an hour or so manually configuing all sites in IIS to "revert to parent". Then I used the tool to configure the ALL SITES option. This produced much cleaner web.config files, but didn't resolve the problem.

Once I got all 4 sites running then they remained stable until the next reboot. Except that is for that last day when the Tomcat Server stopped twice withing 24 hours.

Yes, I did have CF9 and CF10 running in parallel initially, but during the troubleshooting, I completely uninstally CF10 AND CF9, before installing a fresh copy of CF10. Unfortunately this didn't help.

I have reverted to CF9 and will stick with it until I see sound evidence that CF10 is stable.

I do feel it's a connector issue between CF10/Tomcat and IIS. I'm on the 64 bit versions so can't speak for a 32 bit setup.

Again, many thanks for your response.

Best wishes

Peter

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Nov 09, 2012 Nov 09, 2012

Copy link to clipboard

Copied

Ripples, sorry to hear your pain.

We upgrade to CF 10 running Windows 2008 servers with IIS 7, connecting to MySQL 5 DBs. We put our highest profile customers on these new boxes and promised them a great, upgraded service. We tested, and tested each site was then put live on the new VPSs. Site failures became rampant. IIS got hung up several times a day. Restarting IIS solves the problem, but it is an absurd problem for a middle ware that is on version 10.

Seems that if there is only 1 site running there is no problem, but if mulitple sites run on the same server, IIS gets hung up.

I am glad to read that going back to CF 9 solved your problems. This is what we may do.

Charlie Arehart, above has some good suggestions it seems, and from what I can tell, it is really a matter of CF10 working propertly with IIS 7. Problem can be solved by running IIS 7 with IIS 6 compatibility enabled?

If anyone has come up with a stable config of the above specs - would love to hear it

Thanks,

james

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Nov 21, 2012 Nov 21, 2012

Copy link to clipboard

Copied

James,

Update 4 had the connector fix in it. If you apply Update 4 and use wsconfig to disconnect and reconnect your instances to IIS 7, you should be good to go. The crashing should stop.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Nov 26, 2012 Nov 26, 2012

Copy link to clipboard

Copied

Chris,

Thank you for the note. We are have applied this fix, and so far so good.

We also had to update a setting in CF admin setting for: “Maximum number of simultaneous Template requests”. The default for CF 10 is set to 25. The default install of our local copy of CF 10 has this set to 25. Our box was set to 10, as it was upgraded from 8 and 9 and we think this was left over from CF 8 and 9.

However, according to Adobe, when the limit for Template requests is reached, the system is supposed to queue the rest. In our case it does not queue, it stops. So there is still some concern. 

We also don’t see how a request is equal to site activity for 1 site. We thought that “worker processes” would be related to a number of different processes that happened when web templates run.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Feb 19, 2013 Feb 19, 2013

Copy link to clipboard

Copied

Chris,

Thank you for the note. We are have applied this fix, and so far so good.

We also had to update a setting in CF admin setting for: “Maximum number of simultaneous Template requests”. The default for CF 10 is set to 25. The default install of our local copy of CF 10 has this set to 25. Our box was set to 10, as it was upgraded from 8 and 9 and we think this was left over from CF 8 and 9.

However, according to Adobe, when the limit for Template requests is reached, the system is supposed to queue the rest. In our case it does not queue, it stops. So there is still some concern.

We also don’t see how a request is equal to site activity for 1 site. We thought that “worker processes” would be related to a number of different processes that happened when web templates run.

Ours is set to 0. Is that bad/incorrect. Does 0 in this case mean no limit?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jan 25, 2013 Jan 25, 2013

Copy link to clipboard

Copied

Just updated Coldfusion to latest version (Hotfix 7). We are still having this issue. System runs for a week or so and then our main application pool dies with this error over and over:

SEVERE: Error in getRealPathFromConn
java
.net.SocketException: Connection reset
    at java
.net.SocketInputStream.read(SocketInputStream.java:168)
at org
.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:319)
at org
.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:445)
at org
.apache.coyote.ajp.AjpProcessor.readString(AjpProcessor.java:425)
at org
.apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:407)
at org
.apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1154)
at org
.apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)
at org
.apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)
at org
.apache.catalina.core.StandardContext.getRealPath(StandardContext.java:4475)
at org
.apache.catalina.core.ApplicationContext.getRealPath(ApplicationContext.java:414)
at org
.apache.catalina.core.ApplicationContextFacade.getRealPath(ApplicationContextFacade.java:335)
at coldfusion
.runtime.ServletContextWrapper.doGetRealPath(ServletContextWrapper.java:179)
at coldfusion
.runtime.ServletContextWrapper._doGetRealPath(ServletContextWrapper.java:116)
at coldfusion
.runtime.ServletContextWrapper.getRealPath(ServletContextWrapper.java:98)
at coldfusion
.runtime.TemplateProxyFactory.getFullName(TemplateProxyFactory.java:1092)
at coldfusion
.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:202)
at coldfusion
.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)
at coldfusion
.runtime.TemplateProxyFactory.resolveFile(TemplateProxyFactory.java:120)
at coldfusion
.cfc.CFCProxy.<init>(CFCProxy.java:138)
at coldfusion
.cfc.CFCProxy.<init>(CFCProxy.java:84)
at coldfusion
.runtime.AppEventInvoker.<init>(AppEventInvoker.java:64)
at coldfusion
.filter.ApplicationFilter.invoke(ApplicationFilter.java:226)
at coldfusion
.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion
.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion
.filter.PathFilter.invoke(PathFilter.java:112)
at coldfusion
.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)
at coldfusion
.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
at coldfusion
.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion
.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion
.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion
.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion
.filter.CachingFilter.invoke(CachingFilter.java:62)
at coldfusion
.CfmServlet.service(CfmServlet.java:219)
at coldfusion
.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at org
.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org
.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at coldfusion
.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
at coldfusion
.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at org
.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org
.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org
.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at org
.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
at org
.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org
.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at org
.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org
.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)
at org
.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org
.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:414)
at org
.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:204)
at org
.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
at org
.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)
at java
.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java
.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java
.lang.Thread.run(Thread.java:662)

While the server error is the same each time, the trace is different with each one. The first one always looks like the above, and the rest of them after it look similar to the below where the connection is reset by peer. However the full traces and paths to different files changes with each error usually. Sometimes three or four in a row of the same, sometimes just one. Either way, there are like 40 or 50 in a row.

Jan 25, 2013 8:10:04 AM org.apache.coyote.ajp.AjpProcessor getRealPathFromServer

SEVERE: Error in getRealPathFromConn

java.net.SocketException: Connection reset by peer: socket write error

          at java.net.SocketOutputStream.socketWrite0(Native Method)

          at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)

          at java.net.SocketOutputStream.write(SocketOutputStream.java:124)

          at org.apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:401)

          at org.apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1154)

          at org.apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)

          at org.apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)

          at org.apache.catalina.core.StandardContext.getRealPath(StandardContext.java:4475)

          at org.apache.catalina.core.ApplicationContext.getRealPath(ApplicationContext.java:414)

          at org.apache.catalina.core.ApplicationContextFacade.getRealPath(ApplicationContextFacade.java:335)

          at coldfusion.runtime.ServletContextWrapper.doGetRealPath(ServletContextWrapper.java:184)

          at coldfusion.runtime.ServletContextWrapper._doGetRealPath(ServletContextWrapper.java:116)

          at coldfusion.runtime.ServletContextWrapper.getRealPath(ServletContextWrapper.java:98)

          at coldfusion.runtime.RuntimeServiceImpl.getRealPath(RuntimeServiceImpl.java:1046)

          at coldfusion.runtime.RuntimeServiceImpl.getRealPath(RuntimeServiceImpl.java:1060)

          at coldfusion.filter.FusionContext.getRealPath(FusionContext.java:858)

          at coldfusion.filter.FusionContext.getRealPath(FusionContext.java:830)

          at coldfusion.runtime.RuntimeServiceImpl.resolveTemplatePath(RuntimeServiceImpl.java:794)

          at coldfusion.runtime.TemplateProxyFactory.getResolvedFile(TemplateProxyFactory.java:1263)

          at coldfusion.runtime.TemplateProxyFactory.getTemplateFileHelper(TemplateProxyFactory.java:1571)

          at coldfusion.cfc.ComponentProxyFactory.getProxy(ComponentProxyFactory.java:51)

          at coldfusion.runtime.CFPage.CreateObject(CFPage.java:5560)

          at coldfusion.runtime.CFPage.CreateObject(CFPage.java:5575)

          at cfApplication2ecfc851739670.runPage(D:\www\facilitator\Application.cfc:30)

          at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:244)

          at coldfusion.filter.SilentFilter.invoke(SilentFilter.java:47)

          at coldfusion.runtime.TemplateProxyFactory.resolveComponentHelper(TemplateProxyFactory.java:538)

          at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:234)

          at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)

          at coldfusion.runtime.TemplateProxyFactory.resolveFile(TemplateProxyFactory.java:120)

          at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:138)

          at coldfusion.cfc.CFCProxy.<init>(CFCProxy.java:84)

          at coldfusion.runtime.AppEventInvoker.<init>(AppEventInvoker.java:64)

          at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:226)

          at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)

          at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)

          at coldfusion.filter.PathFilter.invoke(PathFilter.java:112)

          at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)

          at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)

          at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)

          at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)

          at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)

          at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)

          at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)

          at coldfusion.CfmServlet.service(CfmServlet.java:219)

          at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)

          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)

          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

          at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)

          at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)

          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)

          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)

          at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)

          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)

          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)

          at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)

          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)

          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:414)

          at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:204)

          at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)

          at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)

          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

          at java.lang.Thread.run(Thread.java:662)

This is a critical error and is really bugging the crap out of me. Strange thing is, as far as I know, this has not happened until the last month or so. We've been running CF10 on IIS7 for about 6 months. The first appearance of the error is on Aug 7, however it only fired once. The on Nov 5 once. Then Jan 10 twice. Then on Jan12 it started firing the error repeatedly in a row causing IIS to stop the pool. I updated CF shortly after that to attempt to fix the issue, but it remains an issue. Also, this error has occured 0 times in our development machine. The machines are for all intents and purposes identical. (HDD/RAM is different but nothing else)

The only other thing I can think of, you can slap me for my ignorance, is there are other things in the log file:

A bunch of these:

Jan 18, 2013 8:13:59 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks

SEVERE: The web application [/] created a ThreadLocal with key of type [coldfusion.util.DateUtils$1] (value [coldfusion.util.DateUtils$1@cde21de]) and a value of type [java.util.GregorianCalendar] (value [java.util.GregorianCalendar[time=1613372400000,areFieldsSet=true,areAllFieldsSet=true,lenient=false,zone=sun.util.calendar.ZoneInfo[id="America/Denver",offset=-25200000,dstSavings=3600000,useDaylight=true,transitions=157,lastRule=java.util.SimpleTimeZone[id=America/Denver,offset=-25200000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2021,MONTH=1,WEEK_OF_YEAR=8,WEEK_OF_MONTH=3,DAY_OF_MONTH=15,DAY_OF_YEAR=46,DAY_OF_WEEK=2,DAY_OF_WEEK_IN_MONTH=3,AM_PM=0,HOUR=0,HOUR_OF_DAY=0,MINUTE=0,SECOND=0,MILLISECOND=0,ZONE_OFFSET=-25200000,DST_OFFSET=0]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

And These:

Jan 17, 2013 10:42:07 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\\ColdFusion10\\cfusion\lib;C:\\ColdFusion10\\cfusion\jintegra\bin;C:\\ColdFusion10\\cfusion\jintegra\bin\international;C:\\ColdFusion10\\cfusion\lib\oosdk\classes\win

You won't suprise me if you say these items may be related but I really don't know what those two other errors mean. I've tried googling them with no luck on what to do about them.

Also looking at the exception log, usually around the same time I get an error stating that Coldfusion can't find something. Usually at the top of the stack the method its running is:

at coldfusion.runtime.TemplateProxyFactory.getResolvedFile(TemplateProxyFactory.java:1344)

At the time I am usually running something to get the path to create a component or something like that if I follow the trace down. This seems like it is the result of the main error that Coldfusion can't the path. Usually these exceptions are found just in the application.cfc or when the application is initializing. Just so everyone is aware, all our web files are on our 😧 drive not the system drive where Coldfusion is installed. It's not an external drive. This is a virtural server hosted by our IT department.

All of the IIS6 Compatability components are there in the Role Services. Do I need to "turn them on?"

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Feb 11, 2013 Feb 11, 2013

Copy link to clipboard

Copied

@Lee

FWIW I am having the exact same issue.  Installed CF10 on a fresh win 2k r2 box IIS7.5 running a Mura instance.  Developed the site over the last 6 months or so.  Ran into an issue with the site returning blank pages with a few errors.  I did see all of the ones you describe above.  Fiddled with the JVM settings to no avail.

I hope adding my voice to the crowd gathering here helps get things fixed.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines