Skip to main content
A
Anonymous
July 20, 2013
Question

Coldfusion 10 enterprise - mod_jk keeps crashing

  • July 20, 2013
  • 1 reply
  • 4235 views

Environment:

Windows server 2003

CF10 enterprise - update 11 installed

Apache 2.2 with mod_ssl

Error

After about 8 hours of use eventually the logs will start showing this message and no further requests to coldfusion are possible. Static assets are still served by apache however.

[Sun Jul 21 01:00:11 2013] [1796:1092] [error] ajp_service::jk_ajp_common.c (2470): Failed allocating AJP message buffer

[Sun Jul 21 01:00:11 2013] [1796:1092] [info] jk_handler::mod_jk.c (2748): Service error=-5 for worker=cfusion

[Sun Jul 21 01:01:36 2013] [1796:3492] [error] ajp_service::jk_ajp_common.c (2455): Failed allocating AJP message buffer

[Sun Jul 21 01:01:36 2013] [1796:3492] [info] jk_handler::mod_jk.c (2748): Service error=-5 for worker=cfusion

[Sun Jul 21 01:01:37 2013] [1796:2488] [error] ajp_service::jk_ajp_common.c (2455): Failed allocating AJP message buffer

[Sun Jul 21 01:01:37 2013] [1796:2488] [info] jk_handler::mod_jk.c (2748): Service error=-5 for worker=cfusion

Steps taken to try and address

  • Have reinstalled both apache and coldfusion.
  • Have reinstalled the apache connector after fully patching coldfusion
  • Have attempted to tune the workers.properties / server.xml files (e.g http://www.webtrenches.com/post.cfm/resolve-stability-problems-and-speed-up-coldfusion-10)
  • Have tried both multiple instances and single instance mode
  • Have even attempted to get mod_proxy_ajp working, but because this server hosts approximately 20 websites I would need to be able to set up virtual hosts in the web.xml, however for some reason coldfusions version of tomcat this is not working as expected so I get 404's on all index.cfm requests when browsing with a virtual host proxying via the ajp port. Same goes from straight mod_proxying.

I think I am out of ideas short of swtiching to IIS. But have reservations based on the fact CF10 had workers problems with IIS as well and this is win2k3 meaning the horrible iis6 and no native rewrite rules.

This topic has been closed for replies.

1 reply

C
carl type3
Brainiac
July 21, 2013

I don't know what is going wrong for you there and CF with apache is not my environment however some things come to mind you can try.

Are those errors reported in "mod_jk" log (cf10\config\wsconfig\N\) file? Any java errors in "cf-out" or "cf-error" logs (cf10\cfusion\logs)? Reason I ask is tomcat connector is reporting a buffer issue, tomcat uses java and sometimes one of the java memory buffers (heap new perm) could be reaching a limit.

Something else that might be worth trying is add tomcat native library to use tomcat AJP apr rather than AJP bio. Weekend for me so if you want some more details on that reply and I will respond when I am at work.

I favour applying similar values as those listed in the link you mention. One point of contention with those values is the worker.cfusion.connection_pool_timeout=60 in mod_jk properties and connectionTimeout= "60000" in server.xml. Some references suggest a value of 10 minutes not 1 minute so that would be pool_timeout=600 and connectionTimeout= "600000". Note you need to restart CF and apache to apply those tomcat changes.

HTH, Carl.

A
Anonymous
July 21, 2013

The errors are in mod_jk.log and there are no errors in out and error logs on any of the instances. The cf logs just stop at the time of the outage in cf (meaning no more requests are making it to server).

Memory seems fine in fusion reactor. But your mention of it being a memory issue aligns with what I have seen when googling the error. I could try increasing permgen to see if that helps, the main heap seems ok though.

Will try the 10 minute timeout.

And yeah would be interested in the native tomcat library option. I have a mixture of railo and acf clients and have no issues with railo clients on mod_jk/w2k3/apache 2.2 (though the publically released mod_jk, not the custom adobe one). So I find this very odd.

A
Anonymous
July 23, 2013

Thanks for your observations. Well indeed that would appear not overly stressed at all.

Using RDP or console Windows TASKMANGER | Processes tab, how much memory Commit Size (you might have to add that column and tick show process for all users) does the coldfusion.exe grow to?

Also might be interesting to know is CF10 using java 6 or 7 (CFadmin > System Information | Java Version and Java VM Name)?

I hope the native tomcat library (APR) helps. Carl.


*sigh*

Last successful request 18:03:11.40 Memory 34%

Server uptime was approx 6 hours 30 mins at crash time

Java Version 1.7.0_15

http://grab.by/oHJ4

jconsole overview

http://grab.by/oHHu

connector overview

http://grab.by/oHHy

coldfusion error

http://grab.by/oHHO

Coldfusion out log stops at 16:25 (approx 96 mins before requests cease) a rotate occurred but new file is 0 byte.

mod_jk.log

http://grab.by/oHHY

task manager

http://grab.by/oHI4

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices