run large file processing batches in scheduled task is failing

We have a system that receieves real time messages via an event gateway. We file all these messages onto disk as small TXT files. I am trying to get our staging environment to load these TXT file overnight to bring staging back up to date with live. We are about 3 weeks behind at this point with about 45k messages a day.

We have a CFM page that gets a list of all the TXT files, reads each one in and then uses a set of CFC objects to create an object structure from the message (the message has various subcomponents/combinations) it then calls a serialise function on the object which serialises the object into a database, it passes the serialise call down to all the subcomponents.

This page seems to run fine if I run it manually. I can run blocks of 10k messages in comfortably as long as I'm willing to sit and watch it. This takes about an hour.

I wanted to schedule this page overnight so it would process the messages at 3am and thus be up to date by the time we came in about 8.

When I run the page as a scheduled task it gets to just over 1000 messages and crashes with a JVM out of memory error

ROOT CAUSE: 
java.lang.OutOfMemoryError: GC overhead limit exceeded

javax.servlet.ServletException: ROOT CAUSE: 
java.lang.OutOfMemoryError: GC overhead limit exceeded

     at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:70)
     at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
     at jrun.servlet.FilterChain.service(FilterChain.java:101)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
     at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
     at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
     at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
     at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

Now I can't let this run as it also takes down the gateways that are responding to live messaging and while we can catch up its not really very practical.

I am resetting the object at the bottom of each loop using <CFSET msgObj = JavaCast("null","")> in the hope it wouldn't consume tons of memory, but obviously this isn't wokring. I am saving the scheduled task output to a file so I can see what happened but I don't really understand why its behaving so differently when i run it as a scheduled tasks to when i run the page in a browser,

Anyone have any pointers as to what i might try?