Submitting and receiving a large number of form fields

Many thanks for the suggestions.

I modified the page to update each person individually, rather than all of the people in the category at once using ajax.  I did it in a way that meant that after each persons details were updated, then they dropped off the too do list by refreshing the list (again, using ajax).  This works well on the one hand, but as I expected, if the list of people is quite long (which is how we got to this problem in the first place), then loading the remaining records in the list takes bit of time, and as a result, slows things up.  I'm currently trying to figure out if I can group the records into logical smaller, and more manageable groups that will not ever come close to the limit of form fields, and do the updates that way.

Hi Ben, I have just implemented a solution for an old application to solve the same problem where a repeating set of fields can grow to 100's of sets in a few cases. What I implemneted was a two form solution. One for the repeating data display and a second used to post the data in JSON format in a single field. When the submit button is pressed Javascript is used to collect all the repeating data into a JSON dataset and stored in the hidden field for submitting. You end up with a single field to submit then you unpack the JSON on the server and process.

<form method="post" action="savedata.cfm">
<input type="hidden" name="jsondata">
<input name="submit" type="submit" value="Save" onclick="packData()">
</form>

Hi Ben5E9F,

My initial thoughts coincide with yours. I would, on that page, write code to collect the fields into a data-interchange format such as JSON or XML. The JSON or XML file is then uploaded using something like:

<form method="post" action="uploadFiles.cfm" enctype="multipart/form-data">
<input name="fileContents" type="file"><br>
<input name="submit" type="submit" value="Upload File">
</form>
 

I'll suggest some ideas that might help.  Some require JavaScript, but that is fairly easy to accomplish even without jQuery.  Google: javascript fetch post coldfusion

1. What about placing each group (Person1, Person2, etc) in a separate form?  In other words, you will have many forms on the page, but fewer fields within each form.  You could have a submit button for each group so that users can "save their progress" as they proceed through the entire form.  Simply, the form would submit, the page would reload, and then scroll to where you last saved.  Or, upon submit of that particular "sub-form", JavaScript could be used to send only that form's fields to your form-processing code.

2. When the entire form is completed, a button will fire a JavaScript script that loops through each form, and submits the fields to your form-processing code in smaller batches that won't exceed the limitations.

3. Rethink your user interface for managing this amount of data.  Should the submission of so many form fields be considered a single transaction?

4. Use a spreadsheet to layout your data, then, upload the spreadsheet and let CF process it.

There are a couple of things going on here. First, if you have that many form fields and want to pass them all to the server, you're not going to find a more terse way to do it than just using regular form fields. JSON doesn't require less room to describe the same data - it requires more!

Second, I'm having trouble imagining how many form fields you're passing at one time to have this problem. That must be one helluva form! Maybe you could redesign the form so it's not doing that, exactly. For example, you could present data in a grid, then let the user choose a single record for editing or creating, then just send that one change instead of the whole set. Of course, that's where JavaScript and optionally JSON might come into play. And of course, this would be a significant amount of work that you're not currently prepared to do.

Third, honestly, if you have a legitimate form that big, it's ok to increase the limit that's there just to define the size of a legitimate form. That's really all it is, and that's why you can change it. These constraints are movable exactly for this reason. Of course, that does make your server slightly more vulnerable to denial of service attacks, but that's the price of having a big form, and it's still just "slightly more vulnerable" - even if you increase it by 100x in most cases. Can you believe that for years we didn't have any limit at all?

Anyway, it seems like you can either just increase the form size or rewrite your form, so you should let your manager know that.

Dave Watts, Eidolon LLC

Ben, while someone may well offer a specific answer to your request (of how to handle a "large number" of form fields differently), I will share instead that concern  over raising that admin limit, while reasonable, can be over-stated.

First, note that the risk of raising it "too high" is not one that enables a break-in but a denial of service attack (and one which was not unique to CF), as explained in this post, 

https://www.petefreitag.com/item/801.cfm. 

And perhaps more important, note as well there that .net had chosen a limit of 1,000, while tomcat (which underlies CF) had chosen a limit of 10,000. So why did Adobe choose 200? There was never any explanation (when this decision was made in 2012). 

So should you really stress over finding a way to stay under that low limit? I'm just not sure it's worth the bother.

And if this issue is indeed only about a potential denial of service attack, do most sites really need to worry about that? 

Of course when it comes to security matters, some will be far more cautious than others. And I'm not here making any specific recommendations. I'm just sharing these observations, for an interested person to consider and research more.

And any new info on the whole  matter would of course be welcome, as it may come to light. 

But sure, if someone may offer an alternative way to pass your data and not hit this limit, it may well be of value to others as well.