Skip to main content
P
pwp69
Inspiring
September 29, 2008
Question

CF 8 JVM memory is not being garbage collected.

  • September 29, 2008
  • 10 replies
  • 3294 views
I am baffled by something I am seeing on my QA server. I have an app that we load tested but when the test completed the JVM memory used was not released. I used CF Server Monitor to watch the memory usage and sometimes it spiked to the max and either the app failed or I got timeout exceptions.

This is the only app running on this server and the testing completed over an hour ago but the memory has not been released yet.

CF Admin settings:
Maximum JVM Heap Size (MB) 512

The CF Server JVM Setting arguments include: -server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC

I found a script that uses java.lang.Runtime and java.lang.management.managementFactory that dumps a JVM memory usage profile The latest dump follows:

JVM Monitor - ColdFusion Server - Enterprise v8,0,1,195765
JVM Memory Monitor - struct
Heap Memory Usage - Committed 481 MB
Heap Memory Usage - Initial 0.00 MB
Heap Memory Usage - Max 493 MB
Heap Memory Usage - Used 437 MB
JVM - Free Memory 44.0 MB
JVM - Max Memory 493 MB
JVM - Total Memory 481 MB
JVM - Used Memory 449 MB
Memory Pool - Code Cache - Used 8.80 MB
Memory Pool - PS Eden Space - Used 6.37 MB
Memory Pool - PS Old Gen - Used 428 MB
Memory Pool - PS Perm Gen - Used 52.4 MB
Memory Pool - PS Survivor Space - Used 3.50 MB
Non-Heap Memory Usage - Committed 62.8 MB
Non-Heap Memory Usage - Initial 18.3 MB
Non-Heap Memory Usage - Max 240 MB
Non-Heap Memory Usage - Used 61.2 MB

According to the CF Server Monitor JVM memory usage builds up to 477 MB then the app fails or timesout.
Session Scope memory usage: 0.27 KB
Application Scope memory usage: 1.370 KB
Server Scope memory usage: 3.12 KB

Since the test ended JVM Memory Usage has dropped back to 438 MB?

Besides CFAdmin nothing else is running on this CF Server. I've read several other memory related topics but none of them have helped.

Can someone tell me why the memory isn't being release? How can I further troubleshoot the problem?

Thx

pwp
This topic has been closed for replies.

10 replies

BKBK
Community Expert
BKBKCommunity Expert
Community Expert
October 1, 2008
Adam cameron wrote:
My point was that your rule of thumb:

maximum heap size(Xmx) = RAM(in MB) / (2 * number of servers using the JVM)
Is not a very good one. Plug 4GB RAM (so a small server) and
one CF instance into that equation. Your rule suggests I should
be allocating 2GB to the heap. Which - as you yourself pointed
out - won't work.

There are always limits, of course. We still use Newton everyday, even though relativity has shown him to be incorrect at high speeds

A rule of thumb is just that: a rule of thumb. In these things, one is usually looking for an order of magnitude estimate. It is meant to give you an indication of the kind of value to expect or of how far you can go before the cookie begins to crumble. It is more art than exact science.

The elaborate, one-gig-is-stable example you give is just one such rule of thumb. It will crumble at 1 GB RAM. Also, though your machine fell flat on 1.5 GB, there are 32-bit machines that successfully run on 1.5 GB heap size with, say, 4 GB RAM. However, that does not discount your experience.

There could be other factors involved. For example, the type of application can have an influence on the workable maximum heap size. Hence my suggestion for PWP69 to look into queries.




N
Newsgroup_User
Inspiring
October 1, 2008
> Adam Cameron wrote:
> The maximum stable heap size I've managed to get is around
> 1.0-1.2GB, on a win32 system. On Solaris (running a 32-bit JVM),
> about 1.4GB. It *seems* like GC doesn't actually clear out RAM
> properly if more than that much RAM is being addressed.
>
> Yes, there is a well-known
> http://kb.adobe.com/selfservice/viewContent.do?externalId=tn_19359&sliceId=1

Not really what I was talking about. One might be able to get the CF
instance to *start* with 1.8GB allocated to the heap, but it won't actually
work. I've managed to get a server to idle for a reasonable length of time
on 1.5GB, but as soon as the thing started to ramp up, it face-planted,
once it started actually trying to *use* the higher end of the RAM
allocated to it. At 1.2GB, it'll seem to run OK for a reasonable length of
time, but eventually it starts leaking memory; at around 1GB, it was pretty
stable.

Hence my comment about it being *stable* at that allocation. Not that "it
simply won't start if more than 1.8GB is allocated to it".

My point was that your rule of thumb:

maximum heap size(Xmx) = RAM(in MB) / (2 * number of servers using the
JVM)

Is not a very good one. Plug 4GB RAM (so a small server) and one CF
instance into that equation. Your rule suggests I should be allocating 2GB
to the heap. Which - as you yourself pointed out - won't work.

--
Adam
BKBK
Community Expert
BKBKCommunity Expert
Community Expert
October 1, 2008
PWP69 wrote:
I have been monitoring JVM Memory Usage with CF Server Monitor.
Session, Server and Application scope memory usage all seem within reason. They increase and decrease as expected.
JVM memory usage climbs and may decline a little but never enough for the app to run for long.

What database server does the application use? How many queries are there in the application?

I ask because there could also be the problem of query objects that the garbage collector fails to collect. This is a known problem for SQL Server. One of the solutions was to set Max Pooled Statements to zero in the application's datasource settings in the Coldfusion Administrator.

BKBK
Community Expert
BKBKCommunity Expert
Community Expert
October 1, 2008
Adam Cameron wrote:
The maximum stable heap size I've managed to get is around
1.0-1.2GB, on a win32 system. On Solaris (running a 32-bit JVM),
about 1.4GB. It *seems* like GC doesn't actually clear out RAM
properly if more than that much RAM is being addressed.

Yes, there is a well-known 1.8 GB heap-size limit for 32-bit processors

I think your PERM size seems quite big in relation to your
heap size. But that's not based on any specific knowledge,
just the ratio I've seen in various different configurations.

Reminds me of another rule of thumb: set the initial heap size(Xms) and the maximum heap size(Xmx) to the same value.

P
pwp69Author
Inspiring
October 1, 2008
Thx, BKBK.

No. I didn't know the rule of thumb for setting maximum heap size but I did increase to 1024 MB and it still eventually ran out.

I have been monitoring JVM Memory Usage with CF Server Monitor.

Session, Server and Application scope memory usage all seem within reason. They increase and decrease as expected.

JVM memory usage climbs and may decline a little but never enough for the app to run for long.

Clicking the Run GC button has no affect.

I'm following up on ksmith's link but can't find the jrockit JVM or the mission control tool set to download. Apparently since Oracle and BEA merged things have changed some.

thx

pwp
N
Newsgroup_User
Inspiring
October 1, 2008
> OK. Then I suppose you would know the rule of thumb:
> maximum heap size(Xmx) = RAM(in MB) / (2 * number of servers using the JVM)

The maximum stable heap size I've managed to get is around 1.0-1.2GB, on a
win32 system. On Solaris (running a 32-bit JVM), about 1.4GB. It *seems*
like GC doesn't actually clear out RAM properly if more than that much RAM
is being addressed.

I'ev no experience with 64-bit JVMs.

What version JVM is in use here? And OS?

I think your PERM size seems quite big in relation to your heap size. But
that's not based on any specific knowledge, just the ratio I've seen in
various different configurations.

I suppose there was a reason to change from the defaults? Can you detail
why the settings are what they are?

--
Adam
BKBK
Community Expert
BKBKCommunity Expert
Community Expert
October 1, 2008
>I know I can increase the memory available to the JVM
>using the startup params...
OK. Then I suppose you would know the rule of thumb:
maximum heap size(Xmx) = RAM(in MB) / (2 * number of servers using the JVM)

>...when I do [increase the memory] it may take a bit longer but
>the system eventually crashes as well.
There may well be something wrong with Coldfusion. But then again there might just be an application that is generating an infinite loop or rogue threads. You have to rule this out.

The Administrator interface in Coldfusion 8 has the tools you need to find out what's causing the high memory usage. Just follow these steps:
1) Open the Coldfusion Administrator and go to the Server Monitor page.
2) Press the button Launch Server Monitor. The server monitoring interface opens.
3) Click on the Statistics tab and then on Memory Usage.

Study the various kinds of memory usage for clues. You might also want to see what happens when you click on the Run GC button in the Memory Usage Summary.


A
Anonymous
September 30, 2008
Use the ColdFusion server monitor memory page or review http://www.schierberl.com/cfblog/index.cfm/2006/10/12/ColdFusion_memoryLeak_profiler. I believe Sun offers similar capabilites to jrockit now, too.
P
pwp69Author
Inspiring
September 30, 2008
4GB.

I know I can increase the memory available to the JVM using the startup params but when I do it may take a bit longer but the system eventually crashes as well.

thx

pwp
BKBK
Community Expert
BKBKCommunity Expert
Community Expert
September 30, 2008
What is your machine's RAM?

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices