CFML gateway : java.lang.OutOfMemoryError

[obouillaud]

I though that CFML gateways were designed for long running tasks, but my tests tends to show the opposite :
whatever I do, it always crash with a java.lang.OutOfMemoryError

I made a mass emailing gateway, following the CDFJ article (
june 2005, volume 7, issue 6 : the asynchronous CFML gateway)

that is to say :
- a classic cfm page that defines the needed struct (arguments for the gateway cfc) then a call to the gateway.
- the gateway cfc is very basic : it makes a query to retrieve email adresses from a database then loop over the query and for each adress send an email.

I use cfoutput query + a cfmail for each email because i need to make a bit of other things in the loop (every 100 email sent i made another "update" query to keep an information of what email were sent.

I have about 150,000 emails to send but plan to send more (1,000,000. I'm not a spammer, all adresses are opted-in).

The first time it crashed around 25,000 emails.
I tried to maximize the JVM settings (maxperm, maxheap, ... to 512MB then 1024MB), it then crashed around 55,000, but whatever I do it still crashs before the end of the task. The server is a dual xeon 3.0 ghz with 2 GB of RAM and is running Coldfusion 7.0.1 (updater 2) on windows 2003 std.

I also replace the cfmal tag by cffile tag (for each adress i write then delete a small file, to simulate activity without really sending tons of emails ... and remove the "every 100 loops update a counter" query... it still crashs (but around 360,000 loops).

Just in case, I tried the cfsetting timeout in the cfc, the timeout in the struct passed to the cfc... no effect.

So I'm a bit confused about the aibility to handle long running tasks...

The exception.log files contains :
"Error","Thread-13","04/20/06","17:26:58",,"Error invoking CFC for gateway emailblaster: null"
java.lang.OutOfMemoryError


And here is a part of the cfc code :

[BKBK]

The "funny" thing I notice, is that yesterday the crash was around 412,000, today it's 355,000 loops.
Coincides with one of my pet hypotheses. There could be a memory-intensive code-block or process, outside of the component, that the JVM runs while the component code is busy. Where the loop crashes would then depend, for example, on when the rogue-process started and when its memory usage peaked.

[BKBK]

I wonder whether one could use a thread-dump utility to get information about any memory-intensive processes that the JVM might be running parallel to the gateway component. Sysinternals' Process Explorer is another handy investigative tool for memory usage.

[obouillaud]

"One limitation that you need to be aware of is that you cannot use this utility from Terminal Services. "

I can only use the server using TS... the server is 50km far from me :-(

I will try on a local dev server... but the config is quite different (slower cpu, only 1GB of ram, database on the same server than CF...) but why not