Highlighted

How to handle > 200mb data file using ColdFusion

New Here ,
Sep 12, 2019

Copy link to clipboard

Copied

Hello ColdFusion Expert,

I have a question regarding flat file processing using ColdFusion.

Currently at our company we used webMethod to process big data file( > 200MB) - it is working fine and fast but due to the maintenance cost is to high we decided to use ColdFusion. The question is how to process big data file using ColdFusion without consume high memory that will effect the application performance?

Views

143

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

How to handle > 200mb data file using ColdFusion

New Here ,
Sep 12, 2019

Copy link to clipboard

Copied

Hello ColdFusion Expert,

I have a question regarding flat file processing using ColdFusion.

Currently at our company we used webMethod to process big data file( > 200MB) - it is working fine and fast but due to the maintenance cost is to high we decided to use ColdFusion. The question is how to process big data file using ColdFusion without consume high memory that will effect the application performance?

Views

144

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Sep 12, 2019 0
Adobe Community Professional ,
Sep 12, 2019

Copy link to clipboard

Copied

It depends on how you will be reading it. Do you know yet? You can read such files with cfhttp (yes, pointing to a csv file, and it can turn that into a query).  Or you can use CFFILE and its various actions, some which read the entire file, some which read it line by line. Same for the file* function equivalents to that.

 

But either way, 200m is nothing to a CF running on, say, a 1gig heap--unless of course other things runnign within CF are using up that 1g heap. Usually it would not be so, that "all the memory is already used". And many shops run with 2, 4, 8, or far more gigs of heap.

 

So this may well be no issue for you to handle. Only testing will tell.  And good monitoring tools (like FusionReactor, or JVM tools, or others) can help you understand how memory is used during and after that request processing.

/Charlie (server troubleshooter, carehart.org)

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 12, 2019 0
New Here ,
Sep 17, 2019

Copy link to clipboard

Copied

Thank you for the answer. really helpful.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 17, 2019 0