• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

How to handle > 200mb data file using ColdFusion

New Here ,
Sep 12, 2019 Sep 12, 2019

Copy link to clipboard

Copied

Hello ColdFusion Expert,

I have a question regarding flat file processing using ColdFusion.

Currently at our company we used webMethod to process big data file( > 200MB) - it is working fine and fast but due to the maintenance cost is to high we decided to use ColdFusion. The question is how to process big data file using ColdFusion without consume high memory that will effect the application performance?

Views

279

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 12, 2019 Sep 12, 2019

Copy link to clipboard

Copied

It depends on how you will be reading it. Do you know yet? You can read such files with cfhttp (yes, pointing to a csv file, and it can turn that into a query).  Or you can use CFFILE and its various actions, some which read the entire file, some which read it line by line. Same for the file* function equivalents to that.

 

But either way, 200m is nothing to a CF running on, say, a 1gig heap--unless of course other things runnign within CF are using up that 1g heap. Usually it would not be so, that "all the memory is already used". And many shops run with 2, 4, 8, or far more gigs of heap.

 

So this may well be no issue for you to handle. Only testing will tell.  And good monitoring tools (like FusionReactor, or JVM tools, or others) can help you understand how memory is used during and after that request processing.


/Charlie (troubleshooter, carehart.org)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Sep 17, 2019 Sep 17, 2019

Copy link to clipboard

Copied

LATEST
Thank you for the answer. really helpful.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Resources
Documentation