Skip to main content
B
BrianTech3
Participating Frequently
March 20, 2009
Question

Problem Writing Large Files

  • March 20, 2009
  • 3 replies
  • 755 views
Hi all,

Now that I got the read problem under wraps, I'm trying to write some data from a database to a text file that has fixed width records. I'm looping through and doing appends with CFFile. Around 22MB it bombs out saying I've run out of heap memory. Is there a better way I should be trying to write it, other than using CFFile?

Thank you
    This topic has been closed for replies.

    3 replies

    ilssac
    ilssac
    Inspiring
    April 13, 2009

    You may want to try using some Java objects to write this large file out, they can be more efficient and less memory hungry then the CFML versions.

    Here is a good looking link that describes this in more detail I got with a Google Search for ColdFusion Java file object


    http://www.bennadel.com/blog/305-ColdFusion-CFFile-vs-Java-java-io-BufferedOutputStream.htm

    B
    balumohan
    Inspiring
    April 13, 2009

    will this be solved by increasing the ???

    tclaremont
    tclaremont
    Inspiring
    April 13, 2009

    I have found that modifying the heap size can be tricky... and frustrating. Use the following code to see where your stats are right now, and refresh this page periodically while your CFFILE routine is running:

    <cfoutput>

    <cfset runtime = CreateObject("java", "java.lang.Runtime").getRuntime()>
    <cfset freeMemory = runtime.freeMemory() / 1024 / 1024>  <cfset totalMemory = runtime.totalMemory() / 1024 / 1024>  <cfset maxMemory = runtime.maxMemory() / 1024 / 1024>

    Max Memory Available to JVM: #Round(maxMemory)#MB<br>  Total Memory Allocated: #Round(totalMemory)#MB<br>  Free Allocated Memory: #Round(freeMemory)#MB

    <br><br>
    </cfoutput>

    Run this page (above) and note the stats. Then, while the CFFILE process is running, refresh the stats page periodically and note what is happening to your memory. Then, once the CFFILE page either crashes or completed, run the stats page a final time and note the results.

    Post back with what you find.

    A
    Anonymous
    March 21, 2009
    Hi,

    When I write out to a file, I use the MOD functionality. In the example below, I grap 25 rows, them write them to the file. Modify the number after the MOD to keep your memory in the safe zone. The last piece of the code dumps out any remaining rows (say you have 5 left over, but can't get to the full 25). The WriteCSV is my query that is writing out the records to the table.

    cfwild
    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices