Cf 2021 Issue with CFPDF and Flattening

[matthewl20646694]

I just recently upgraded to ColdFusion 2021 from ColdFusion 2016 and I have run into an odd issue.  I am populating a PDFForm with values using the CFPDFForm tag, and writing the PDF data to a variable.  Later I go to write out this PDF file to disk, flattening the form, and I get an error.  

<cfpdf action="write" source="PDFFormVariableName" destination="#pathToWritePDF#" flatten="yes" overwrite="true">

With this code, I get an error:

Error: Variables written as a file by specifying destination in a tag cannot be used again.

coldfusion.pdf.core.PDFException$PDFInvalidDocumentVariableException: An error occurred while reading source for the cfpdf tag. at coldfusion.pdf.PDFDocHandler.readInternal(PDFDocHandler.java:324) at coldfusion.pdf.PDFDocOperation.write(PDFDocOperation.java:427) at coldfusion.pdf.PDFDocOperation.write(PDFDocOperation.java:469) at coldfusion.tagext.pdf.PDFTag.doEndTag(PDFTag.java:1963) at ....  

If I change the code to write the PDF variable to disk, then load that file and flatten it, it works fine.
<cfpdf action="write" source="PDFFormVariableName" destination="#pathToWritePDF#" overwrite="true">
<cfpdf action="write" source="#formpath##outfile1#" destination="#pathToWritePDF#" flatten="yes" overwrite="true">

So there is no issue with the actual PDF file.  but now I have to write it to disk, read it from disk, flatten it and re-write it back to disk.  Versus onely writing to disk once.  

I did try doing the source as a variable, flatten with a distination of the same or different variable, with the same result.  Everything I have tried seems to show that I can not FLATTEN when the source is a PDF variable.  

The original Code worked fine in ColdFusion 2016 and 2018.  I do not have any other server at my disposal to test this on.  The server is ColdFusion 2021 Enterprise Edition with Update 4 installed.  The update was done this past week, CF 2016 was completely removed from the machine, and did a clean install of CF 2021.  I think that this needs to be submitted to Adobe as a bug, but I am not sure the process for that.  

I am open to suggestions on anything I might have missed or how to get this submitted to Adobe as a bug.  

Matthew

[Charlie Arehart]

Hi, Matthew. I've not experienced this or tried to recreate it (am reading this on a phone), but in fact my first thought was both that it could indeed be a bug AND that that bug might already be fixed and available.

So first, as for filing a bug or looking for them see tracker.adobe.com.

As for bug fixes that may exist, note also that being on cf2021 update 4 (from May) is not quite enough to ensure one has all the bug fixes that existed THEN. Sadly, Adobe opted in that May update AND the December update (3 for cf2021) to ONLY offer security fixes and NO bug fixes.

What's unfortunate is that the SEPTEMBER update (2 for cf2021 and 12 for cf2018) introduced several bugs, and though some had fixes created within weeks, NONE OF THOSE bug fixes were built into those updates in Jan or May (for either cf version). When I saw that in May z I was REALLY disappointed. It's the first time I've known them to forgo including known bug fixes (some of which affect a LARGE percentage of users) for several months. For example, a terrible bug with query of queries has had a fix since October!

Instead, one has to a) recognize their error as being related to that (not always obvious), b) consider that it could be a bug, c) go to tracker to look for it, d) if a fix is identified but it's hotfix jar is not offered there, they have to ask Adobe for it (and know how to, such as via cfsup@adobe.com) , then e) apply that hotfix to cf and restart. Finally, if somehow yet ANOTHER update may come out that does not include the fix, you have to put the special hotfix jar back in, as each update presumes to remove all such special hotfixes (because again typically such updates ARE supposed to include known recent fixes). It's been SO confusing that the last 2 updates have not.

And again, back to your issue Matthew, see if tracker has your bug and fix, as you may be caught up in this wash.

Otherwise, sure, file a bug report there. I'll add that the more you can do to make a test case totally stand on its own, the more readily you may get a response from Adobe or others the wanting to help.

Sorry for the diatribe, but I hope it helps you and others who may be following along. 

[matthewl20646694]

I do really appreciate your feedback. I did not know about the issue with the hotfixes , so I will keep that in mind as I work with my clients.  I have gone ahead and searched for an issue on the tracker, did not find any, and so I have submitted one there.  Hopefully they will be able to confirm the issue and get a fix in place.  I just hate having to write out a file to disk when I do not need to.  We have been working on improving the effeciency of our code by reducing how much we have to write out PDF files to diks, and do more in memory.  

It is also nice to know, that it is not something easy or should have known, "well in CF 2021 they changed the CFPDF tag to do...." and there is some known change that I was not aware of.  Knowing your background and the troubleshoot steps that I did, I do feel that I did not miss the obvious in this case.  

Thanks again.  

Matt

[Dave Watts]

MILDEST DIATRIBE EVAR

More seriously, even though I'd rather have existing hotfixes integrated into new releases, I suspect there's some good reason why Adobe isn't doing this. In my experience, Adobe isn't great at explaining why they're doing something even if they have a good reason why they're doing it. In this case, maybe some people aren't having problems with a feature, but if they have the hotfix installed it breaks something else. ColdFusion is complicated enough that I can see potential for that happening.

Dave Watts, Eidolon LLC

[James Moberg]

It's possible that you can't do both at the same time anymore. I've run into PDF-related issues in the past and started looking for actively-supported third-party alternatives that I could easily test outside of ColdFusion.  This "avoid-and-use-an-alternative" approach also means that we are unaffected by future bugs that are accidentally introduced (and then only fixed in newer versions or not released because it's not part of their release "schedule"). We also can leverage new/different features that may not be currently available in Adobe's built-in functions.

For flattening, separating and joining, we've had great success using PDFtk. We were forced to use this since built-in functions were unreliable and failed to successfully identify some PDF files created by WKHTMLTOPDF or provided by clients.  The functions would indicate that the PDF was "valid" and then throw an error when trying to perform an operation and complain that the PDF wasn't valid. (???)

To deal with unnecessary filesize bloat, we use GhostScript. This has enabled us to send many large dynamically-generated PDFs as small email attachments after being optimized to a much smaller filesize. (The slight reduction in quality hasn't been an issue.)

[BKBK]

It's possible that you can't do both at the same time anymore. I've run into PDF-related issues in the past and started looking for actively-supported third-party alternatives that I could easily test outside of ColdFusion.  This "avoid-and-use-an-alternative" approach also means that we are unaffected by future bugs that are accidentally introduced (and then only fixed in newer versions or not released because it's not part of their release "schedule"). We also can leverage new/different features that may not be currently available in Adobe's built-in functions.

For flattening, separating and joining, we've had great success using PDFtk. We were forced to use this since built-in functions were unreliable and failed to successfully identify some PDF files created by WKHTMLTOPDF or provided by clients.  The functions would indicate that the PDF was "valid" and then throw an error when trying to perform an operation and complain that the PDF wasn't valid. (???)

To deal with unnecessary filesize bloat, we use GhostScript. This has enabled us to send many large dynamically-generated PDFs as small email attachments after being optimized to a much smaller filesize. (The slight reduction in quality hasn't been an issue.)

By @James Moberg

@James Moberg , your first paragragh contains crucial advice