Highlighted

Large cffile uploadall dies

New Here ,
Aug 14, 2016

Copy link to clipboard

Copied

I have a need to upload video clips from football games to our website.  These clips vary from 5mb to 200mb and there can be up to 220 per game (though usually about 160).  The total size of all files is about 18gb.

I am using the htmp input type=file multiple option to get the file names and cffile uploadall to bring the files up to the server.  I actually have my code working but if I give it more than about 30 of these clips, it chokes and dies. 

These files are being uploaded by officiating crew evaluators who are not technical in any way.  And because of the file size, it can take several hours to upload.  What I am trying to do is have our website actually initiate the uploads rather than have them do it.  So I need to upload all files in the same transaction and just let it run as long as it needs to.

So apparently around 30 of these files or so, I must be hitting a wall.  Does anyone know what this wall is and how I might go about avoiding it?

Dave

Views

663

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Large cffile uploadall dies

New Here ,
Aug 14, 2016

Copy link to clipboard

Copied

I have a need to upload video clips from football games to our website.  These clips vary from 5mb to 200mb and there can be up to 220 per game (though usually about 160).  The total size of all files is about 18gb.

I am using the htmp input type=file multiple option to get the file names and cffile uploadall to bring the files up to the server.  I actually have my code working but if I give it more than about 30 of these clips, it chokes and dies. 

These files are being uploaded by officiating crew evaluators who are not technical in any way.  And because of the file size, it can take several hours to upload.  What I am trying to do is have our website actually initiate the uploads rather than have them do it.  So I need to upload all files in the same transaction and just let it run as long as it needs to.

So apparently around 30 of these files or so, I must be hitting a wall.  Does anyone know what this wall is and how I might go about avoiding it?

Dave

Views

664

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Aug 14, 2016 0
Adobe Community Professional ,
Aug 14, 2016

Copy link to clipboard

Copied

Try to upload in smaller batches. In any case, I have 2 suggestions.

You should:

1) Increase the request timeout values in the Coldfusion Administrator or, alternatively, use the cfsetting tag, to set the estimated upload time.

If your upload speed is, say, 3 MB/s, then you could assume an average of 1 MB/s, hence a request potentially lasting 18000 seconds for 18 GB.

2) Use the following attribute values in the cffileupload tag:

maxuploadsize="201" <!--- maximum size in MB allowed for upload file --->

maxFileSelect="220" <!--- maximum number of files allowed for upload --->

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 14, 2016 0
New Here ,
Aug 14, 2016

Copy link to clipboard

Copied

BKBK:

First, thanks for the reply.  So here is where I am at....I have a cffileupload tag in my form and it tries to do the upload.  The url file for the cffileupload exists but I am not sure I am doing it right.  Do you have any samples of a page using cffileupload and the url related file that it calls that I could look at?

Thanks!

Dave

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 14, 2016 0
Adobe Community Professional ,
Aug 14, 2016

Copy link to clipboard

Copied

multiUpload.cfm

<script type="text/javascript">

    var getUploadError = function(errorObject)

    {

        alert(ColdFusion.JSON.encode(errorObject));

    }

</script>

<cffileupload url="uploadSelectedFiles.cfm"

    maxuploadsize="10"

    onError="getUploadError"

    hideUploadButton = "no"

    progressbar="true"

    name="myupload"

    addButtonLabel = "Add File"

    clearButtonlabel = "Clear"

    uploadButtonlabel="Upload"

    width=600

    height=400

    title = "File Upload"

    maxFileSelect="4"/>

uploadSelectedFiles.cfm

<cffile action="uploadall" destination="c:\uploads\" nameconflict="overwrite" />

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 14, 2016 0
Jamo LATEST
Engaged ,
Aug 15, 2016

Copy link to clipboard

Copied

You didn't mention which web server or version of ColdFusion you are using.  CFFileUpload & CFFile are not the solution you are seeking with files that exceed 10mb.

Some web servers restrict the maximum file size of uploaded file attachments.  (For IIS 6 and IIS 7, the default maximum file upload size is 4 MB and 28.6 MB respectively.)  ColdFusion doesn't handle incoming large files very well either (IMHO) unless you do something called a "chunking upload" and avoid using CFFile altogether.

A Google search for "chunk" and "coldfusion" will return a blog article by Ben Nadel regarding how to use Plupload & Coldfusion together.

Chunking File Uploads With Plupload And ColdFusion

Chunking Amazon S3 File Uploads With Plupload And ColdFusion

I recommend checking out the Amazon S3 solution as there are ways to directly upload files without having to use ColdFusion.

I personally use this "jQuery File Upload" library with ColdFusion.  It's similar and it enabled our applications to allow uploads of 500mb+ files without impacting the stability of the server or other users.

jQuery File Upload Demo

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 15, 2016 0