Known Participant

Question

CFML gateway : java.lang.OutOfMemoryError

Forum|Forum|19 years ago
April 21, 2006
20 replies
2661 views

I though that CFML gateways were designed for long running tasks, but my tests tends to show the opposite :
whatever I do, it always crash with a java.lang.OutOfMemoryError

I made a mass emailing gateway, following the CDFJ article (
june 2005, volume 7, issue 6 : the asynchronous CFML gateway)

that is to say :
- a classic cfm page that defines the needed struct (arguments for the gateway cfc) then a call to the gateway.
- the gateway cfc is very basic : it makes a query to retrieve email adresses from a database then loop over the query and for each adress send an email.

I use cfoutput query + a cfmail for each email because i need to make a bit of other things in the loop (every 100 email sent i made another "update" query to keep an information of what email were sent.

I have about 150,000 emails to send but plan to send more (1,000,000. I'm not a spammer, all adresses are opted-in).

The first time it crashed around 25,000 emails.
I tried to maximize the JVM settings (maxperm, maxheap, ... to 512MB then 1024MB), it then crashed around 55,000, but whatever I do it still crashs before the end of the task. The server is a dual xeon 3.0 ghz with 2 GB of RAM and is running Coldfusion 7.0.1 (updater 2) on windows 2003 std.

I also replace the cfmal tag by cffile tag (for each adress i write then delete a small file, to simulate activity without really sending tons of emails

... and remove the "every 100 loops update a counter" query... it still crashs (but around 360,000 loops).

Just in case, I tried the cfsetting timeout in the cfc, the timeout in the struct passed to the cfc... no effect.

So I'm a bit confused about the aibility to handle long running tasks...

The exception.log files contains :
"Error","Thread-13","04/20/06","17:26:58",,"Error invoking CFC for gateway emailblaster: null"
java.lang.OutOfMemoryError

And here is a part of the cfc code :

Advanced techniques

This topic has been closed for replies.

O

obouillaudAuthor

Known Participant

"One limitation that you need to be aware of is that you cannot use this utility from Terminal Services. "

I can only use the server using TS... the server is 50km far from me :-(

I will try on a local dev server... but the config is quite different (slower cpu, only 1GB of ram, database on the same server than CF...) but why not

BKBK

Community Expert

I wonder whether one could use a thread-dump utility to get information about any memory-intensive processes that the JVM might be running parallel to the gateway component. Sysinternals' Process Explorer is another handy investigative tool for memory usage.

BKBK

Community Expert

The "funny" thing I notice, is that yesterday the crash was around 412,000, today it's 355,000 loops.
Coincides with one of my pet hypotheses. There could be a memory-intensive code-block or process, outside of the component, that the JVM runs while the component code is busy. Where the loop crashes would then depend, for example, on when the rogue-process started and when its memory usage peaked.

O

obouillaudAuthor

Known Participant

- I removed the cfsettinf timeout : no changes.

- I removed non ascii chars... no changes (crast at 355,000).

- I changed the JVM settings... no changes.

- there is one clean call to the gateway : <cfset status = SendGatewayMessage("emailblaster", emaildata)>

- the settings in the cfadmin were : 10 threads / max 250,000 events in queue. I tried 20 / max 1,000,000.... no changes (crash at 355,000 loops)

The "funny" thing I notice, is that yesterday the crash was around 412,000, today it's 355,000 loops.
I restarted Coldfusion to have a fresh new jrun... and restarted the task : same crash at 355,000 loops.

What does it mean ? Maybe it's something related to the database content, an email adresss that can cause problems... As the query sorts the result by email alphabetically, It's possible to have between yesterday and today a news adress that causes problems.
So, I add a new thing : I don't delete the txt file after 355,000 loops. to see the last email processed and be sure it crashed every time at the same row. The first time I tried it crashed at 355,849
The last adress processed was "jkcay.ofutueq AT wanadoo.fr" (I change the order of some letters to avoid displaying the real adress) : no space, no accent, nothing special :-(
The next adress was "jkcay.cancoise-freromau AT wanadoo.fr" (same process with the letters). One more time, nothing noticeable !
The previous and following 10 adresses have nothing special either.

I retried the test a second time ... it crashed at 355,950. But this time (as I do my test on our live server, and there were new emails inserted between the two tests

the last email processed was one before the previous test.
I mean that the last processed email the second time was eleven rows before the last one processed the first time.

So... I'm quite sure now it's not relative the the content of the database. It seems more something like a timeout or the number of objects in the memory, ...

BKBK

Community Expert

It's an achievement that the component code executes successfully, and in just a few seconds. In my opinion, it helps in the process of elimination.

For example, if the empty loop hung or also caused an outOfMemory error, then it would mean some external process associated with the gateway is manufacturing objects in the background. In fact, I'm currently looking for any sources of live objects and also for a way to gracefully free memory.

N

Newsgroup_User

Inspiring

> huuu ???
> a loop that does nothing isn't very useful ... even if it doesn't crash !-)

I have, in the past, got Java out-of-memory errors with templates that have
non-ascii characters in them, although the last time I saw this would have
been CFMX6.0, I think (although that could well be because I make sure I
don't have any in there, any more!)

What happens if you run your CFFILE version of the loop, but first out any
characters from your comments that have graves, acutes or cedillas.

This seems like a dumb suggestion, but it's easy to test, and removes it as
a possibility. I - too - see nothing in your code that raises a red flag.

--
Adam

O

obouillaudAuthor

Known Participant

huuu ???
a loop that does nothing isn't very useful ... even if it doesn't crash !-)

BKBK

Community Expert

I would also the do following, even if only to eliminate them as possible suspects

- Verify that there is just one, and only one, clean call to SendGatewayMessage()

- Set initial heap size = max heap size = 1024 MB and maxPermSize = 256 MB
(reference: ColdFusion MX: Tips for performance and scalability )

- delete the tag, <cfsetting>, from the component code (It is related to the debugging process, a major memory consumer, and so is a suspect by association)

- experiment with a larger value of "Event Gateway Processing Threads" (in Coldfusion Administrator => Event Gateways => Settings)

BKBK

Community Expert

We've won one battle.

O

obouillaudAuthor

Known Participant

Ok, sorry I didn't undertood.

I tried with nothing in the loop. It takes just a few seconds to complete the whole task, and I have no error. I looped over 802,052 adresses, the total in my database.

I can try with more (by creating a false table with more than one million of rows) but I think there will be no change.

O

obouillaudAuthor

Known Participant

I want to be able to send 1,000,000 emails without error. (My database is 800,000 persons, so I want to be able to contact them if needed

BKBK

Community Expert

What I mean is, how many loops do you get when there is no code in the loop?

Show more replies

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded