Generating Excel files suddenly throws exceptions

[WolfShade]

Good morning, all.

I have code that has been in place for at least 6 or 7 years that generates Excel files.  Depending upon the app, the Excel file is either A) made available to the user via download, or B) emailed to a pre-determined email address, possibly CC'ing others, as well.  It has had no issues for all this time, and no changes to the code have been made.

Recently, while creating a new section of a public-facing website, I copied and modified Excel generating code from one of these older sections, expecting it to work.  I've been having all kinds of issues, from files not being saved to the server, to simple strings added to a cell coming up as garbage.  Exceptions are being thrown.  So, I decided to check one of the earlier pages on the public site that creates Excel files, and lo and behold it's throwing an exception when it comes to creating the Excel file.  I have no idea how long this has been breaking, no one ever reported anything to my department.  But it's clearly breaking.  (UPDATE:  Someone just now reported an issue.)

The errors that are being reported are all the same: 

Message: org/apache/xalan/processor/TransformerFactoryImpl
Type: java.lang.ClassNotFoundException
Provider org/apache/xalan/processor/TransformerFactoryImpl not found

I found forum messages indicating that the xalan.jar file was missing.  I asked my SA about it, and she stated that xalan was never part of the install of any of our CF Servers.  So, it doesn't make sense to me that it not being part of the install would trigger this error.

For years this code has worked with very few issues.  I'm thinking some kind of server config change is the cause, but have no idea where to start looking.

Has anyone ever experienced anything like this?

V/r,

WolfShade

[WolfShade]

I just read about a new patch that is coming out.  I wonder if the following might be related?  Even though I'm told xalan was not part of the install process.

After installing and then uninstalling the last update, the cfspreadsheet function no longer works. The C:\ColdFusion2021\cfusion\lib\xalan.jar was removed during the uninstallation process, even though this JAR file was not added or modified as part of the original installation.

Just a thought.

WolfShade

[WolfShade]

I was recently informed that no patches have been uninstalled, so that's not it.

[WolfShade]

Is there some sort of java -D entry in CF Admin that can fix this?  I cannot figure this out.

[Charlie Arehart · Accepted solution]

Frustrating, of course. But there may be more to this than meets the eye. 

First, you say no updates were done. Let's not rely on what anyone says. What do you see in a dump of the server.coldfusion struct? I think it's the productversion key (am writing from my phone).

If it was updated, then some problems like this are solved if you delete the folders where cf saves compilation of cfml to Java (cfusion/wwwroot/WEB-INF/cfclasses) and where it saves package mgt artifacts (cfusion/bin/felix-cache). You can stop cf, delete those, and cf will recreate them (without performance impact). 

Even if cf was not updated, it's possible a package was. And someone may also have updated the Java that Cf uses, which can also call for these actions.

Finally, if these steps don't solve the problem, a next concern would be that there was a failure during updating of cf packages (again, whether anyone "knows" that was attempted or not). There are logs that could be explored to confirm things, but let's hear what you find about the above first (especially the reported version number). 

[WolfShade]

Thank you, Charlie.  I will start doing that, and report back.

WolfShade

UPDATE:  Product Version is 2021,0,20,330407

[matthew_0510 · Accepted solution]

The update for CF2021u20 (https://helpx.adobe.com/coldfusion/kb/coldfusion-2021-update-20.html) lists an issue with xalan.jar in the "Known Issues in the Update" section. The gist is that what you're describing could be caused by a missing xalan.jar file in [your-CF-root]\cfusion\lib folder.   

[WolfShade]

Hi, @matthew_0510 .

I did speak with my SA about that, and she said that xalan was never part of the installation of any of our CF Servers.

Do you think that just by pasting the xalan.jar file in its proper place will fix this?  I was of the impression that xalan was being deprecated/phased out.

WolfShade

[matthew_0510]

It's worth a shot; I do see that xalan.jar file in my environments and we are generating Excel files without issue. 

[WolfShade]

Sorry for the delay in replying.. I'm not getting email messages telling me that a reply has been made.

Do I need to do anything else?  Do I need to go into CF Admin and point to it?  Or will just copy/paste be sufficient?

V/r,

WolfShade

[BKBK]

Hi @WolfShade , Did you follow up on Charlie's suggestion to delete the Felix cache? If so, what was the result? Did the Xalan exception go away?

[WolfShade]

Hi, BKBK.

Not yet.. the SA and DBA are typically super-busy.  I sent the link for this thread to the SA.  Still waiting on a reply.  I just bumped the message.

WolfShade

[BKBK]

Hi, @WolfShade , thanks for the update.

[rayf32793838]

I'm seeign a similar problem with generating spreadsheets, but a with a different error message. Ths only started happening after CF 2021 Update 21 was installed.

java.lang.NoClassDefFoundError: com/zaxxer/sparsebits/SparseBitSet at org.apache.poi.ddf.EscherDggRecord.findNewDrawingGroupId(EscherDggRecord.java:304)

Ray

[spcooney]

Thanks so much for the thread. Updated one of our test environments and we ran across this issue while testing the patch.

We were getting a 500 error followed by a Caused by: java.lang.ClassNotFoundException: com.zaxxer.sparsebits.SparseBitSet not found by poi-5.4.1 [49].

Found this post and was able to go in and clear the cf classes as suggested (which has started to feel like the catch all solution post patches):

1. Stop the CF services.
2. Navigate to ColdFusion2023\cfusion\wwwroot\WEB-INF\cfclasses and remove all files (would be regenerated)
3. Navigate to ColdFusion2023\cfusion\bin\felix-cache and remove all files. (would be regenerated)
4. Start CF services and check the scenario.

I can confirm that in our case this worked.  After stopping, removing and restarted all tiers the export to excel and download work again as expected.

Thanks all for the input on this thread.

(Running: ColdFusion 2023 HF 15 on Windows)

[spcooney]

Potentially premature update on my part.  Though the class clear appears to have worked for one of our calls another similar call is broken and throws a different error after the initial 500.

Caused by: coldfusion.runtime.JSONUtils$JSONParseOverflowException: JSON parsing failure: Unexpected end of JSON string

The call of course works as expected on another yet to be patched tier.  Just thought I would update that the class clear may not have been a cure all.  Apologies again for the possible premature fix confirmation.

[WolfShade]

Unexpected end of JSON string.  I've seen that, before.  I'm assuming the string is being generated by CF.  Could be truncated data causing this?  Have you done a cfdump of the value?  Are you encoding the content so quotation marks and the like cannot break the string?

Still waiting to hear back from my SA on Charlie's suggested actions.

WolfShade

[spcooney]

The JSON object is being generated in SQL Server and sent back to the CF Function.  This is some code that definitely worked/works on HotFix 14.  we confirmed that it is in fact fully formatted JSON.

Thanks for the suggestion.  I hope the class cleanup works for you, it did fix my first issue. 

[spcooney]

Had a chance to catch up with another developer/admin colleague and we found that this JSON issue was merely coincidental as opposed to caused by the patch.  His work had gone to QA and ran into the patch issue where clearing the classes fixed it.  The next issue of the JSON object was caused by it being truncated because the datasource on that tier had not been set to allow CLOB (Enable long text retrieval (CLOB).).
Apologies for making it seem like a patch issue but thanks to the community for being here to vent as it did help me go through a lot of good troubleshooting steps.  Good luck to the OP on this thread I hope again that the class clear helps with your issue.

[WolfShade]

Yeah, I've been bitten by the non-CLOB bug, before.

Glad to hear that you got it all straightened out.

WolfShade

[BKBK]

Still waiting to hear back from my SA on Charlie's suggested actions.

By WolfShade

It is also vital to:

• move up from Update 20 to Update 21;
• ensure that the following Jar file is present:
  

  C:\ColdFusion2021\cfusion\lib\xalan.jar.​

• Alternatively, run the following code as a CFM page:
  

  <cfscript>
  xalanObject=createobject("java","org.apache.xalan.processor.TransformerFactoryImpl");	
  writedump(xalanObject);
  </cfscript>
  ​

[WolfShade]

Hi, BKBK.

We are on U21.  Got a Teams msg from our SA at 0400 this morning.  He had just finished applying a hotfix.  Apparently, a previous update had glitched the CFMAIL tag.  We had hundreds of emails in the /undlvrd folder; vast majority of those were error messages that I was expecting but never received.

He said he was taking PTO for today (who can blame him?), and I went back to the project I was working on.  Just trying to load the page gave an error.  Apparently spreadsheet is not installed, now. The page reads two Excel files for autosuggest fields in the form.

Still no word on whether or not Charlie's suggestions were enacted.

So, I have to wait until Monday when he returns to see about fixing that.

WolfShade

UDPATE:  I took your code suggestion for CFDUMPing the xalan object.  It's showing a red table with Class Name, Methods, and Fields.  So, I'm going to guess that he did put the xalan.jar file in place.

[BKBK]

Hi WolfShade, thanks for your answer.

UDPATE:  I took your code suggestion for CFDUMPing the xalan object.  It's showing a red table with Class Name, Methods, and Fields.  So, I'm going to guess that he did put the xalan.jar file in place.

By WolfShade

Good news! Let's hope that Monday will bring even better news.

[WolfShade]

As of right now, that error is still appearing:

Detail: You can install package through CLI package manager (E:/.../cfpm.bat) by running the command : install spreadsheet.

Message: The spreadsheet package is not installed.

.. followed by the file and linenumber where the error is happening, which is the first cfspreadsheet action="read" on the page.

I messaged the SA about it, he's online but (as usual) quite busy, so no notion of when this will happen.

WolfShade

[WolfShade]

Another error has just made itself known.

I commented out the spreadsheet code, just so the page would load.  Started filling out the form.

When 100% of the required fields are filled out, I'm using Javascript to load a homemade CAPTCHA.  When this happens, there's an exception thrown while loading the CAPTCHA.

Diagnostics: The image package is not installed.

So there are two packages missing, I'm assuming this happened while CF Server was being fixed by the hotfix.

WolfShade