Problem with ColdFusion 2018 Update 12 (Permissions Issue)

[albertogenty]

After installing this update, all IIS sites show the following message:

You do not have permission to view this directory or page.

No other changes to the server have been made.  This looks to be a permissions issue, but I cannot tell why.  I tried re-applying NTFS permissions from the root directory (c:\ColdFusion2018), but that did not work.  Also, when I try to uninstall the update using the following command:

java -jar C:\ColdFusion2018\cfusion\hf-updates\hf-2018-00012-328566\uninstall /uninstaller.jar

The command prompt returns the following error: 

Error: Unable to access jarfile C:\ColdFusion2018\cfusion\hf-updates\hf-2018-00012-328566\uninstall

I was able to uninstall the update manually by stopping the CF service, deleting the jar file and copying the files and folders back to the instance root directory.  I restarted the CF service and everything works fine.

Here are the server details:

Windows 2019, CF2018 Update 11

Any help is really appreciated.  Thanks!  Alberto

[Charlie Arehart]

Alberto, a few questions to try to help (or reach out for direct help in a consulting session):

• Had you or anyone there applied the auto lock down tool (new, and optional in cf2018)? If so, it imposes significant changes to permissions all over the place.
• How did you apply the update, in the cf admin or from the command line?
• Have you made a request (for a page via iis) on the server? Did it show more details? (If not, the iis settings for "error pages" may be hiding actual iis error details.) And I would wonder if the problem is in the request for the jakarta/isapi_redirect.dll that is called concurrently with any cfm page. If so, check the permissions for THAT for the iis app pool identity, or iis site anonymous user identity.

To be clear, I'm not aware of anything special about this update 12 that would cause your problems, but my first question may be why ANY update could have permissions issues. 

[albertogenty]

Hey Charlie, thanks for responding.  I agree that these updates should not affect permissions.  Below are my answers to your questions:

• We did use the auto-lockdown tool along with a manual review of what was done (and added a few extra changes).
• We have updated this server before and did not have a problem (Update 11).
• We have a custom error template in place, so IIS was responding.
• We did not enable detailed error messages.
• We have since restored the server from backup, so it's up and running again.

In any case, I'll check the jakarta/isapi_redirect.dll permissions, but given that lives in C:\ColdFusion2018, the permissions reset should have taken that out of play. 

[Charlie Arehart]

Permissions reset? 

[albertogenty]

In the original post, I mentioned that I reset NTFS permissions on c:\ColdFusion2018 and all subfolders and files.  It didn't make a difference.

[Charlie Arehart]

Well you say that after applying update 12, while having the problem, you "tried re-applying NTFS permissions from the root directory (c:\ColdFusion2018), but that did not work".

I realize you mean it did not help solve the error, so you reverted everything to get back to work. But you didn't and don't know the exact error that happened, only "You do not have permission to view this directory or page", which clearly was not enough to go on.

Perhaps you're reluctant to try again and hoped someone may recognize things. I don't, myself, beyond the above. Perhaps you'll want to wait for Adobe to find and fix some bug in the update related to this, but I've not heard of it from others.

And I think your evidence (that it's the update) is only circumstantial. If and when you may try again, do enable detailed iis logging and SEE what error it reports, if any. Then you may know the real problem and if it's about permissions you can look at exactly what it's complaining about (like that dll, rather than just the wsconfig folder, let alone the entire cf folder).

And if somehow you can tie that to the update, and it seems something Adobe should have "handled", then a bug report may be warranted. It could be something environmental, on your end, making it rather unique to you. 

Hope that may help, or that others will. 

[albertogenty]

Thanks for trying to help.  It's much appreciated.

Whether it's actually permissions or not, I completely disagree with your assessment that the update impact is circumstantial.  I can apply the update and it takes down the sites.  Without touching -anything- else, I can manually remove the update and the sites start working again.  Therfore, it is absolutely causal. 

All that said, I do plan to enable detailed errors to see what that exposes.  I have also submitted a ticket to Adobe, so let's see what they say.  

[Charlie Arehart]

By circumstantial I mean just because it happens WHEN you apply the update does not necessarily prove that it's caused BY the update. For example, there could be something in your environment (permissions setup) that has changed since your LAST update, such that had you done that before THAT update, this could have happened then also.

I realize thses kind of situations are frustrating, and it's generally easier to toss it to Adobe and say "it seems you broke it, see if you can fix it". Perhaps they will find something.

And the autolockdown tool really throws another wrench into things. (Had you applied it before or since the previous update?) 

I'm just saying sometimes the root cause of a problem is not what we might reasonably presume from. More to the point, if you may get no satisfaction from them, I do think there's at least  more you could do to understand the problem more precisely, if you may try again.

And yep, that's what I was indeed trying to help with. I'm like a first responder: the bell rings, I show up and try to render aid/get things working. I realize sometimes folks just prefer to revert and await Adobe's input. 

[albertogenty]

Charlie - I know what circumstantial means.  Nothing has changed in the environment.  This is 100% fact.  Moreover, and I say this respectully, I don't think what you are saying on this point is right.  If I can rollback an update and it starts working again, CLEARLY the update is doing something that it shouldn't.  Also, I am not "tossing" anything.  I am continuing to troubleshoot while getting help from Adobe.  As I mentioned before, the auto-lockdown was run BEFORE installing update 11.  Update 11 had no issuess.  

[Charlie Arehart]

I stand by all I've said. I realize you disagree, and think I'm being illogical. Let's let it go. 

I hope you may find satisfaction ultimately, whether from Adobe, or someone else here, or continued analysis (perhaps of the sort I asserted might help).

I and readers would certainly look forward to the conclusion. 

[albertogenty]

Agreed.  I will post our findings once I have something.

[albertogenty · Accepted solution]

Hi all,

My apologies for the delayed response. 

Second, a big shout out to Vikram at Adobe.  He was very helpful and responsive.  The issue was resolved a day or two after my last reply here. 

As it turned out, the issue was in fact the update and it affected other Adobe clients as well.

The short of it is that Adobe introduced a new version of Tomcat in this hotfix (hf-2018-00012-328566).  But, the update to the server.xml file did not remove an AJP configuration parameter that was introduced in a prior hotfix (hotfix hf-2018-00010-320417).  This parameter (requiredSecret, which was replaced by secret) is not supported in the new version of Tomcat.  

Per Vikram’s guidance, I removed the parameter, restarted the CF server service and that solved the issue.  

Again, thanks to everyone who tried to help.    

Alberto

[Charlie Arehart]

Alberto, thanks for sharing what you learned from Adobe. And it's great that you learned that the problem was the requiredSecret...and that update 12 changed the Tomcat within CF such that the connector now failed because of that requiredSecret.

But in the interest of helping others who may find this, I do feel it's important to correct/question something (and I hope we can get to the real bottom of things here).

You say that your work with Adobe leads you to conclude that the requiredSecret attribute "was introduced in a prior hotfix (hotfix hf-2018-00010-320417)".  But I can say with 100% confidence that CF2018 update 10 did NOT introduce the requiredSecret attribute into the AJP connector in the server.xml file. 

(To clarify for readers, when he refers to "hotfix hf-2018-00010-320417", that's simply the name of the folder that gets created in the CF hf-updates folder, once update 10 is applied.)

And I say that update 10 did not introduce this attribute, not only from experience (because I have not seen that with all the people I've helped do the updates--and also not EVERYONE who updates to update 12 has this problem--which was why I said things I did in previous comments).

But I also just now specifically demonstrated it on a CF2018 of mine. It was at update 4 (and the server.xml had neither a secret nor a requireSecret on the AJP line). I then applied update 10, and while it DID have a secret added, it did NOT add that requiredSecret attribute added.

To be clear for readers, it was indeed update 8 that had introduced the secret attribute (that Alberta also refer to), when the whole "ghostcat" debacle happened in early 2020. And the Tomcat team at that time deprecated the requiredSecret in favor of secret (and they added also a secretRequired boolean, which really confused matters for some).

Anyway, the bottom line is that it's NOT that update 10 which put that requiredSecret into that server.xml file, for you and for others. Or if it did (if you or Adobe may somehow prove that that it does), we have to ask again why it does not for EVERYONE?

And as I said in earlier comments, I'm still leaning toward the Auto Lockdown Tool being an "environmental" factor here. You confirmed you had run it. I've been wondering all along (since hearing also about the conflict in update 12 being the presence of any requiredSecret) if perhaps it was some version of the AutoLockdown tool which put that there, in the past. 

Then it would jive with what you and Adobe say, that its presence was simply ignored after  CF2018 update 8 until update 12 (which upgraded Tomcat and perhaps the connector). Could update 10 have added the requiredSecret ONLY for those who were had previously run the AutoLockdown tool? I suppose so, but that would seem REALLY odd, since again it was already deprecated as of CF2018's update 8. but it would NOT be odd that some past version of autolockdown tool ITSELF would have added that, and it laid dormant/ignored until update 12.

Again, I press all this because there ARE in fact people still experiencing this issue, who may find this, and it IS valuable for us to get to the bottom of things.  And for some it will of course be enough to hear merely "get rid of the requiredSecret", so again good on your for clarifying that point here. But others who applied update 10 and did NOT experience this should understand now that they are not alone. 🙂

And I'll understand if no one wants to beat this dead horse any further, if it turns out that what I say is so. But of course if I have anything wrong, I would want to be corrected, or would hope to remember to come back and share it. 

Finally, if you may be tempted to tweak your answer, Alberto, now that it's marked as "the answer", that could help some readers. I realize you may not want to, or you may say you can't edit your comments. Some can, while for some reason others cannot. I can, but I wouldn't presume to edit yours unless you told me to.

Hope all that's helpful to someone, without irking any others.

[albertogenty]

Charlie - There is nothing for you to correct, but I am happy to answer your question. 

First, I never claimed the issue affected EVERYONE, but it has affected some as confirmed by Adobe.

Second, I can't speak to why it did not happen to you or anyone else.  I am just communicating the facts about our case...  And facts will trump your confidence/experience (extensive as they are) everytime.  

Third, the Auto-Lockdown was run before update 10 was applied.  After update 10 was applied, everything worked fine.

FOURTH, IN OUR CASE, THE PARAMETER WAS 100% INTRODUCED IN UPDATE 10. 

I can prove this because it was NOT in the server.xml backup folder for update 10.  Then, I checked the server.xml file in the backup folder for update 11, and there it was.  🙂 

The correct answer remains as previously marked.

[albertogenty]

Charlie - To be fair, I just double checked something and wanted to share.  The time stamp for the update 10 backup folder precedes the timestamp for the lockdown folder. 

So, it looks like we ran the auto-lockdown AFTER update 10 was applied.  That being the case, it's possible that the auto-lockdown added the requiredSecret parameter.  That would explain the parameter showing up in the update 11 backuop folder as well.

In any case, I still maintain that update 12 should have removed the parameter if it was not supported with the new version of Tomcat.  

[Charlie Arehart]

Thanks, so much, Alberto. Great to have gotten to the bottom of things.

And sure, I agree that it would be great if update 12 or above would remove that arg if there, sure. You may want to run that by the Adobe folks you worked with, or create a new bug at tracker.adobe.com.

Until then, this thread could help many people. Again, thanks. 

[BKBK]

After installing this update, all IIS sites show the following message:

You do not have permission to view this directory or page.

No other changes to the server have been made.  This looks to be a permissions issue, but I cannot tell why.  I tried re-applying NTFS permissions from the root directory (c:\ColdFusion2018), but that did not work. 

By @albertogenty

I wouldn't tamper with NTFS permissions. You didn't have to before installing Update 12, so you shouldn't now. If you have made any such permissions changes, reverse them and go back to how things were.

... when I try to uninstall the update using the following command:

java -jar C:\ColdFusion2018\cfusion\hf-updates\hf-2018-00012-328566\uninstall /uninstaller.jar

The command prompt returns the following error: 

Error: Unable to access jarfile C:\ColdFusion2018\cfusion\hf-updates\hf-2018-00012-328566\uninstall

The result of a typographical error: you mistakenly typed a space character between "uninstall" and "/uninstaller.jar".

I was able to uninstall the update manually by stopping the CF service, deleting the jar file and copying the files and folders back to the instance root directory.  

This kind of manual "Uninstall+Reinstall" can cause problems at the best of times. For example, you might inadvertently uninstall/reinstall using a Java installation different from the one ColdFusion runs on. You could also miss adding or changing certain registry settings. You yourself have mentioned yet another possible source of problems: process and path permissions.

It is difficult, if not impossible, for a human to keep track of all the vital processes necessary during an Uninstall+Reinstall. Hence the need for installers.

Which brings me to my suggestion: use the updating tool in the ColdFusion Administrator to upgrade from Update 11 to Update 12. Before you do, check all your logs to make sure there are no show-stopping errors. Remember to also check the connector logs ( C:\ColdFusion2018\config\wsconfig\[MAGIC_NUMBER]\isapi_redirect.log ).