Skip to main content
Participant
February 26, 2009
Question

Running out of memory building csv file

  • February 26, 2009
  • 3 replies
  • 391 views
I'm attempting to write a script that does a query on my database. It will generally be working with about 10,000 - 15,000 records. It then checks to see if a certain file exists. If it does, it will add the record to an array. When its done looping over all the records, it takes the array that was created and outputs a csv file (usually with about 5,000 - 10,000 lines). But... before that ever happens, it runs out of memory. What can I do to make it not run out of memory?
    This topic has been closed for replies.

    3 replies

    Inspiring
    February 28, 2009
    quote:

    Originally posted by: nozavroni
    I'm attempting to write a script that does a query on my database. It will generally be working with about 10,000 - 15,000 records. It then checks to see if a certain file exists.

    Sounds pretty inefficient to me. Is there no way you can modify the query so that it only selects the records for which the file exists?
    nozavroniAuthor
    Participant
    February 28, 2009
    It was actually being caused by making a cfhttp request within the loop with GET. I instead changed it to HEAD, and it works beautifully. Thanks anyway :)
    dave_jf
    Participating Frequently
    February 27, 2009
    What you may want to do is to output the array in to a file little by little. Try outputting 500 or so rows at a time. This should help remove the memory issue.