We have CF2016 Enterprise Edition on a server. We use cfcontent tag to allow users to download a .CSV file they have exported from a report within our CF application. We have this problem on this server only and it is our only CF2016 server. We cannot install another version on this server.
The exported .CSV files may be from 2mb to 20mb in size. If file size is under 4mb it downloads fine. If file size is larger than that (approximately) the file will stop downloading around 4,700kb.
However, if I just put a direct link for a large .csv file into the home directory and call it directly via HTTP the file downloads with no problem. So it does not appear to be a firewall problem.
But, the same file will ALWAYS stall out when downloaded through our CF application. Also, if I create a simple page with a link to a CF template with a CFCONTENT tag in it then the small file will download but the larger file will stall out.
Depending on which of those approaches I use the download either just stops at 4,740kb and has part of the file OR I get an error message from the browser that was the download was interrupted. Tried different browsers from different locations.
At this point this appears to be a ColdFusion problem and we have done extensive checking of IIS, etc. The last interesting bit is that we get consistent related error messages in the log that contain the following error: The cause of this output exception was that: org.apache.catalina.connector.ClientAbortException: java.net.SocketTimeOutException.
I appreciate any feedback you may have.
The only thing I can think of would be to use CFSETTING requesttimeout="0" to disable the app from timing out during the download. Or does the download go pretty fast?
^ _ ^
Thanks for the suggestion.
The current setting for requesttimeout is far longer than the download time for files that are being saved as .CSV files. It is also the same as on our other servers that have different applications with the same download capability. But as I have never had a server where I set that value to 0 it is worth a try and/or doubling the current value.
The test file is 12,373 kb and (via the .CFM page) the browser reports the correct approximate download time of 3' 15". This matches the download time reported when I use the direct link to the file on the server. Unfortunately, the download through the .CFM files never last that long - timing out at about 2 minutes on most attempts with this size file.
The problem may not be cf, but instead tomcat underlying cf. Or the iis web connector BETWEEN cf and your iis.
So as a first test, have you tried creating a test page in the cf wwwroot, called via the built in web server (instead of iis), using the same port you use for the cf admin?
If that worked, it would point us away from cf or tomcat, to the cf/iis web connector or iis. If it did NOT work, that would point us clearly at CF and/or tomcat which underlies it.
There are many tunable knobs that could be at issue. Need to narrow things down.
Jerry, something to try: include the content-length header.
Here is an example to illustrate:
<cfset filepath="C:\Users\BKBK\Downloads\httrack_x64-3.49.2.exe"> <cffile action="readbinary" file="#filepath#" variable="httrackInstallationFile" > <cfset fileInfo=getFileInfo(filepath)><!--- a struct containing file properties ---> <cfheader name="Content-Disposition" value="attachment; filename=httrack_x64_install.exe"> <cfheader name="Content-Length" value="#fileInfo.size#"> <cfcontent type="application/octet-stream" variable="#httrackInstallationFile#"/>
Charlie Thank you for the response. I've been scouring your web-based pages on CF2016 release looking for clues so thanks for your blogging.
Due to security restrictions I am unable to access any site on the box by port number (.com:8503 for instance) due to the ports being locked down. I see the point of your suggestion so I tried the following: copied my file - for instance "EmpReport_102119_093000.csv" (14,000kb approx.) to the /wwwroot folder of IIS. This is a folder that is separate - on a different drive - from our ColdFusion 2016 installation. I then used a simple URL to call the file, for example: https://ourserver.company.com/EmpReport_102119_093000.csv. So, Charlie, I don't know that I've actually satisfied the requirements of your test but that's all I can do without involving a whole new level of the technology groups at my job site.
The download started correctly but stopped right at 4,864kb. Multiple attempts achieved the same result of the file being truncated at 4,864kb. I have tried exporting reports that go around the final record that downloads (possible bad characters rejected as problematic) or that comes after the final record in the download and not matter what I run into this maximum of approximately 4,864kb. All reports over that approximate size stop downloading. To reiterate we have several other applications that download perfectly well from the same code but none of the others are ColdFusion 2016.
BKBK - I did start to implement a test similar to your suggestion at end-of-day today. I had a small size gain improvement by adding the Content-Length to the header on Friday but did not do the readbinary and output the variable. I will give it a try on Wed. morning.
Regards all, Jerry
Jerry, thanks for the kind regards. As for your test, though,.no. It does not satisfy my goal, on 2 levels.
First, by not putting the file in cf's wwwroot (and not using cf's built in web server), you are precluding the sole reason I proposed it: to take iis out of the equation, so as To see WHERE the problem may be.
I realize why you feel you can't.
That said, the fact that you're showing now downloading directly a csv file proves this is NOT a CF problem. Cf does not serve csv files, so this seemingly MUST be an IIS issue, or perhaps something BEFORE IIS, but it's not CF. (I realize that originally you referred to sendimg the csv via cfcontent.)
In IIS, it could be the request filtering feature, a url rewrite, or something else.
It could be some anti-virus on the box or in your network, or a firewall, or even a load balancer.. Such tools sometimes implement protections beyond their basic expected functionality.
Hope that helps.
Ah, it seems I kind of got it backward! It might not have been clear in the original question so let me show this:
I have a .CSV file (not an XML file): EmpReport_102119_093000.csv I can put it in one of 2 places:
1, /ColdFusion2016/cfusion/wwwroot/ - not used for the application
2. /inetpub/wwwroot/myProject/ - virtual directory in IIS that has our CF application
Number 1 - I have to work with the main CF Admin/Network guy to allow me to do this. I believe this is where you want me to put the file for your test.
Number 2 - WORKS and downloads the entire 14,000kb file
I created a clickHereToDownload.cfm file that has a link to a file that has the CFCONTENT tag in it in the same directory/root as example number 2 and points to the same .CSV file. This one will always FAIL.
I hope this is more clear, if not my thanks all the same. I'll show this thread to a senior guy who does most of the Admin work and see what we can come up with tomorrow morning.
Yep, first I meant csv not xml, of course. I have corrected my reply.
Second, the main point was that you were showing accessing that file directly, not via cfcontent, which is why I said now cf was out of the picture. I have also better clarified that.
But now you do refer again to using cfcontent. Ok. But then you do it from the iis folder.
To be clear, I am proposing that the cfm file (doing the cfcontent) be placed in the cf wwwroot, and be called via the cf built in web server, to remove iis from the equation. (the location of the file being served via cfcontent should not matter).
Mallethead, the cffile action to use for CSV files is "read".
Very similar behavior is marked with Adobe Bug Tracker https://tracker.adobe.com/#/view/CF-3041699. However instead of “org.apache.catalina.connector.ClientAbortException: java.net.SocketTimeOutException” errors users received a "java.lang.OutOfMemoryError: Java heap space” error. Perhaps due to different CF versions.
Similar behavior also reported here: https://www.bennadel.com/blog/1228-eric-stevens-on-cfcontent-and-memory-usage-in-coldfusion-8.htm
This leads to a lot of questions to the behavior of CFCONTENT. In a CF/IIS configuration, what is the difference between serving files using CFCONTENT versus directly through IIS? Based on the bug tracker comments, it appears CFCONTENT loads the file into memory thus limiting file sizes based on JVM memory settings and available resources.
So I question, is CFCONTENT suitable for serving large files? Is this addressed in CF2018?
I'm not aware of any change to CFCONTENT about its basic operation in recent years (including back that CF8 reference you make). I would not be surprised if it DOES read in the entire file to be sent, rather than to somehow stream it. And yes, THAT would be a big difference vs asking for the file directly via IIS, which then would not involve CF at all.
And this is why I am pressing Jerry to get a test of doing that cfcontent from a CF page that is and another that is NOT running itself by way of IIS--to find out where any problem may be, whether in CF, in tomcat, in IIS, etc.
I am working on getting someone with more clearance/access than myself to do the test. Hopefully can nail it down.
Regarding zpm007's comment. We have several CF 2010 servers in production that are running downloads properly using the CFCONTENT tag.
This issue is on a CF 2016 server although there was a similar ticket opened with CF Support about a year ago regarding a CF2010 server. In that instance our people don't know why it started working correctly but it did. Just recently learned about this. These are not huge files - anywhere from 2mb to maybe 30mb.
While we await (hopefully) news of that testing of the CF built-in web server, I will point out for your sake that there was no CF2010. There has been CF2018 and 2016, and before that 11, 10, 9, and so on. (FWIW, CF10 was from 2012, and CF11 from 2014.)
I don't say this (merely) to be pedantic, but to help you in case you go looking for info. I'd honestly never seen it referred to this way, and it would limit the results you'd find. 🙂
Oh duh, of course you are correct. No CF 2010 - actually crossed my mind while I was making the list in the spreadsheet and kept thinking - Hmm, CF 2010. After CF 8 I have not worked in a job where I actually installed the applications myself. I tend to focus on CF and SQL now-a-days.