Application errors in cf log even though website stopped in IIS

[Dstharding]

Hi,

About two weeks ago, Coldfusion 8.01 starting crashing on my windows 2003 server. It freezes up about 3 times a day, and takes all other non-cf sites in iis down with it. Once I restart the coldfusion application server, they all come back up.

I have no idea what is causing this, since the server, exception and application logs don't show any consistent errors at times of crash. The ONE thing I can find that is strange and repetitive is the fact that there is a website throwing tons of file not found errors all throughout every day in the application log. I went into IIS and stopped the website that was getting all of the errors, and the application.log errors for that site continue!

I can't figure out how this could be happening! Any ideas? Even if someone were trying to access the site that is stopped in IIS, those errors shouldnt ever make it to the coldfusion server.

Thanks!

Davey

P.S. If anyone has any idea what could be causing the lock-ups, any help is appreciated. I have installed the current hot fixes, and no help...

[Charlie Arehart]

Davey, you've really raised 3 issues. Let me offer some help with them:

1a) as for your trying to find the causes of lockups, I'll assert you're looking in the wrong logs (the "server, exception and application logs" that you mention). Instead, check out the more important \runtime\logs (or \logs\, if on Multiserver.) The -out log there will often have far more valuable information on why a server is crashing, and it's vital that you not just look "just prior to the crash", but instead find the first error that seems to start occurring before things go south. (Sometimes, once one problem happens, others follow and they can mask the real root problem.)

In many cases, the problem of CF crashing can come down to a memory error, but it may not be what you think. I talk about that (as well as more on the points above) in a blog entry: http://www.carehart.org/blog/client/index.cfm/2010/11/3/when_memory_problems_arent_what_they_seem_part_1

1b) Also, it can be vital to know what, if any (and how many) requests are running at the time leading up to the crash. There are many ways to do that, with various levels of detail, depending on what version of CF you're running (Standard or Enterprise) and if Enterprise, what form of deployment: Server, Multiserver, or war/war.

For those on CF 8/9 Enterprise, the Server Monitor can be very helpful.

For those on any release, in Standard or Enterprise Server deployment, the command-line CFSTAT tool (or same info in Windows Perfmon) can be very helpful. Google about it or see the CF Docs to learn more.

In any version of CF since 6, whether Standard or Enterprise, Server or Multiserver, the JRun Metrics can be helpful. Again, google or see the docs to learn more.

Finally, those on any version of CF since 6, whether Standard or Enterprise, Server or Multiserver, can also use third party monitoring tools like FusionReactor and SeeFusion. Where the CFSTAT, Perfmon, and JRun Metrics only provide gross numbers about how many requests are running, FR and SF (like the CF Server Monitor) can tell you exactly WHAT requests are running (each request's name, how long it's been running, the IP address of the sender, the user-agent, and much more, including details about what queries have run in each request, if that feature is enabled.)

2) You ask how a site can still be causing errors to the CF logs even with the site stopped. Well, maybe the requests aren't coming in via IIS. Maybe they're happening based on a scheduled task, or perhaps via the built-in web server in CF (though that would be somewhat harder to happen easily).

In addition to the tools above that can show you details about what requests are running, you can also check your web server logs to get more details about what requests have run (and FusionReactor also logs every request that runs, with still more detail than the web server logs).

There has to be an explanation. Things don't happen without a cause. Since this is what I do for a living (help people solve CF problems), I can tell you that the number of things that can happen is surprising, but then again nearly all problems are easily solved once you know where to look and can connect the dots. That's where sometimes experience certainly helps. In that regard, I'll say that if you decide you'd rather have someone "looking over your shoulder" while you try to sort through things, you can find more about my services in the signature below. But as you can see, I and others are happy to help for free here. Sometimes, it's just more expedient to get help directly. Totally your call.

Hope the above may be enough to help get you going.

/charlie arehart

charlie@carehart.org

Providing on-demand troubleshooting services for CF and CFBuilder

at http://www.carehart.org/consulting