CF 2016 Upgrade / Migration Issues

[bw_bloodletter]

We're running an older CF10 instance on Windows Server 2012 that we finally got approval to upgrade.

The goal is to bring it in line with other instances of the same application, so we're upgrading to CF2016.

During the upgrade process, we chose "Configure my web server for ColdFusion 2016 (recommended)" as opposed to "Enable the built-in web server (coexist)".

The upgrade process went fine, with 1 reported error.  Looking into the log file, it was an error related to the connector, which made sense as we had left the old CF10 instance and conenctor in place.

We got onto CF Administrator, chose not to migrate anything (we do a full review of settings against a reference server manually), and verified some basic settings.

We went through the process of updating CF (to 11, then to 13), then stopping CF10, removing the connector for CF10, and adding the connector for CF2016.  This all went as expected.

We then set the CF2016 Application Server to run as the intended service account, made sure that account had all the correct permissions on various directories it needed to access, and restarted the VM.

CF itself came up as expected, and we were able to get back onto CF Admin and verify the updates had installed.

However, we were never able to get the website / connector working as expected.  Initially, IIS was throwing a 500 error.  Calling LoadLibraryEx on ISAPI filter c:\…isapi.dll failed.  Looking into it, various places suggested setting Enable 32-bit applications to true for the application pool.  Our working reference server did not have this set to true.  This got us past the 500 error, but we were then faces with a 401.3 error, reporting that the file couldn't be accessed due to ACL restrictions.  The ISAPI redirect dll file was referenced, as was the file that was actually requested (index.cfm or test.cfm).

We verified the CF service account had access to everything, as well as the IIS user account (IUSR).  We even tried creating a new test site and granting "Everyone" access to stuff.  We also noticed no ISAPI redirect log file was created, and the event viewer listed errors about not being able to load the ISAPI filter (for example, "Could not load all ISAPI filters for site 'DEFAULT WEB SITE'. Therefore site startup aborted.").

We verified that the module existed in the same location with the same filesize and timestamp as our reference server.  We verified that both the IUSR account and CF process had access to that folder.  We tried loading up a static html file, and that worked without issue.  We looked throughout everythign we could find in IIS related to the ISAPI module and the redirect to CFM files, and couldn't find anything meaningfully different from the reference server.  We dug into the IIS application.config and other config files, found an old reference to the ISAPI module in C:\Coldfusion10\, and removed it.  (There was another one already for CF2016.)

Nothing helped, and we eventually had to give up and roll back as our maintenance window was ending.

The only real difference between the two servers is that the reference server is running Windows Server 2016 while this older production server is running Windows Server 2012.  I don't have the broken server in front of me to test / dig into right now (since we rolled it back), but does anyone have any ideas?

We'd like to avoid having to roll an entire new VM to get this running, especially since there are other things configured on it (such as for shibbolized SSO access), but I was unable to find anything meaningful.

If we were to attempt this again, is there anything else we should try differently? Perhaps uninstall CF10 and delete the website from IIS first, then reboot, then install CF2016?  We don't care much about preserving the config of the site (minimal) or CF (we have a direct reference to pull from).  Custom libraries / tags / scheduled tasks / etc. are fairly minimal as well.

Thanks

[Charlie Arehart]

Wow, what an exhaustive post mortem, and great job covering the details. (I daresay, a troubleshooter after my own heart!)

So the loadlibrayex error, which you resolved via switching to 32bit: I suspect instead that was where you needed to add the iusr account to the cf config/wsconfig and its children folders.

I wonder (if you'd still had the server) if now swapping BACK to 64bit on the app pool would have solved it.

Of course, it's tough to say for sure. There are so many moving parts, and there could be some other impacting factor you're not mentioning. I will say that I have helped people while in that pickle and we've always solved it. So if you DO try again and this is not the solution, reach out for help at carehart.org/consulting and I may be able to help solve it quite quickly.

But if you or others do have a better answer to share before then, that would be great to see here. 

[bw_bloodletter · Accepted solution]

Thanks for the reply.

I was sketpicalof the 32 bit setting on the application pool, but I believe I did try granting the IUSR account full permission to entire C:\Coldfusion2016\ folder at one point, and it made no difference.

Looking at the reference server fresh today, I'm thinking it was possibly the lack of permissions for the "DefaultAppPool" (or whatever the app pool was on that server), though I do believe we also tried granting "Everyone" access at some point in the troubleshooting process.

If I get another maintenance window for this application I'll take another stab at it.

[Charlie Arehart]

Right, if you look closely, my point was that you may have done that (the IUSR privilege) early on, and then did the switch to 32-bit, the two together may have left it not working. Anyway, sure, give it a go when you can.

BTW, you COULD add perms for only the defaultapppool user, but the iusr is a group that covers that (if you don't need to be explicit per app pool). Also, you don't need to give it perms to ALL of CF, only the config/wsconfig and its children.

Finally, do beware that sometimes you will find that someone did perms to a lower-level folder, so that then your change on the higher-level will not be inherited. But beware the temptation to force perms down (telling it to override whatever may be below)--because that may also remove other exisiting perms that ARE needed. For instance, if you told it to force perms for the iusr down to all subfolders of config/wsconfig, make sure it doesn't remove the existing perms you have for the user running CF (assuming you changed CF to run as other than system/root).

[bw_bloodletter]

I realize I never followed up on this.

If I recall correctly, the issue was the lack of the DefaultAppPool permissions, and the 32-bit setting was not part of the actual solution.  When we reattempted this, everything worked without issue so we must have missed the DefaultAppPool permission somewhere, and the error messages that resulted led us down the wrong path with the 32-bit setting I mentioned earlier.