ColdFusion 2016 Add-On Service fails to start

[Guest]

Windows 2008 R2 server

CF9, CF11, & CF2016 installed (development server)

Java JRE 1.8.0_91

When I try to start the ColdFusion 2016 Add-On Service it fails immediately and does not load into memory.  I tried loading an admin command prompt and navigated to the Jetty folder and ran the command line directly from there too.

jetty.exe -zglaxservice ColdFusion2016Add-onServices

This results in no output and the service does not load.  However, if I load just jetty.exe by itself without command switches the server starts up and loads into memory.

• There are no new logs generated in the jetty/logs folder.
• Nothing besides the fact that it fails makes it into the Windows System or App logs.
• I loaded the Sysinternals Process Monitor tool and watched the execution of the jetty.exe and the folder structure and there are no Access Denied events.
• Failed to load under CF2016 gold, Update 1, and Update 2
• Shutdown CF9 & CF11 to make sure it wasn't a port violation
• Loading it via the command line fails
• Loading jetty.exe without command switches works

Any other suggestions on getting this working?  I've debated installing the features again on top of themselves using the dedicated Add-On installer.  I'm not sure how that impacts the system though.

[Guest]

After setting up another server I came across the same issue but found the change that caused it to fail.  In the CF Lockdown guides they have you create a new user account and then run the services as that account.  Well that same account that worked for CF9, CF10, & CF11 does not work for CF2016 for the ColdFusion2016Add-onServices.  If I return the service to the Local System account it runs again.  There must be something different configuration wise with the new Jetty being used in CF2016 that causes it to fail under the recommended configuration.

[Charlie Arehart]

Brentil, do you still have the problem, or have you since resolved it (I mean finding out what it would take to let it run with that non-system user)? Good catch on discerning the change back to the system account to be a temporary workaround.

I would find it surprising that that was the issue, since I assume that the user you created is set to run both CF and the add-on service, and that user would have been given permissions to the CF folder, which is where both CF and the add-on/jetty app runs....at least by default.

It is possible for one to install the add-on service using a separate installer (on the CF downloads page). That's meant primarily to install it on a machine that does NOT have CF (and where you want to run the Solr and other "add on" services in a separate box). If you may have run that on this CF machine, and it's running out of a different folder, then of course you need to give the non-system user permissions to THAT folder.

You can find out for sure: look at the properties for the add-on service (in the Windows Services panel). What's the value for the "path to executable"? If it's a folder other than your coldfusion2016 folder, then give that user permissions to that and try again.

Then again, you did say you got ProcMon and saw no failing file accesses. So that's indeed odd then that changing the users made things work, as it would beg the question of what it was that th other user could not do. That's a bit of a stumper, if these proofs above show it not to be what I'm proposing it might.

Do let us know.

[CaseyDougall]

I have the same problem right now on a new Windows 2012 RC2 server, fresh install and the add-on services stop as soon as you start them.

The ColdFusion 2016 Add-on Services service terminated unexpectedly

I've tried completely reinstalling these services as well and I've also applied ColdFusion 2016 Update 2 build 299200

ColdFusion 2016 Update 2 build 299200

I'm stuck as the error here isn't all that informative.

I haven't done anything in the lockdown guide yet either.

[Charlie Arehart]

Casey, are you still having your problem? Yours may or may not be the same as Brentil's. He had shared some info that kept me from proposing some other possible explanations. I have some of those below, but first some for you specifically:

1) First, did you try his solution (was your CF service and add-on service running as "local system" or as a different user? If a different user, did change it to local system fix things?)

2) Assuming that's not it, did you look for any log in the C:\ColdFusion2016\cfusion\jetty\logs, especially starting with std*, created on the date you tried to run it? Any errors in there?

3) Then look at your CF Admin, for the Solr Server link on the left, and click the "advanced settings" button on that page. In the field, "Solr Admin Port", what port does it show? That is the port that at least CF thinks this Add-on Service uses if it's running (and if you didn't change it in the Jetty config files, that's what it would be trying to start with). FWIW, the "Add-on service" runs both Solr and the PDFG service (the latter since CF11). It used to be called the "jetty service" in CF10, and simply the "solr service" in CF9".

It could be that something else (including an old version of CF, or something else entirely) is simply using that port already. A quick way to find if anything is using a given port is to use this command, assuming for instance that CF said the solr port was 8987:

netstat -ao | find "8987"

That at least lists IF something is using it, and the last value is the process ID which you could look up in Task Manager (if you don't see it, you can add that column to Task Mgr, but doing that is different for different versions of Windows, so I won't spell it out).

4) If something else IS using it, that would be the reason it can't start. If you can't change the OTHER thing already using that port, you could try to change the port used by this addon service. You DON'T change the port in the CF Admin, though, in order to "make the add-on service" use a new port. Instead, you edit Jetty config files. I can report that I find the port is set in both C:\ColdFusion0216\cfusion\jetty\start.ini and C:\ColdFusion2016\cfusion\jetty\etc\jetty.xml.

I can't find anything to definitively confirm which to change.

I would note that if you DO change it, you then WOULD need to change the CF Admin "Solr Server" page to point to that new port, and also the "PDF Service" page (new to the CF Admin in CF11 and 2016), where you would "edit"  any "pdf service manager" listed there pointing to that old port. I realize that's pretty fiddly stuff. Note the to IF's at the top of this point. Hopefully you can avoid this, but it may help someone else some day.

5) Finally, another very common reason that the "add on service" won't start is that someone has changed the JVM that CF uses (editing the jvm.config's java.home or the CF Admin's Java&JVM page to point to a new "java virtual machine path"). The problem is if they did NOT make the same change to ANOTHER file used by the addon service to point to the JVM . They do need to use the same JVM, it seems. The JVM location for the addon service is specified in C:\ColdFusion2016\cfusion\jetty\jetty.lax, on the line for "lax.nl.current.vm". The default location is this:

lax.nl.current.vm=C:\\ColdFusion2016\\jre\\bin\\javaw.exe

If CF's been told to use a new jvm in a new location, you need to modify this to point to that new location as well.

Let us know if any of this may help you, Casey. I realize it may not, but it was worth a shot, for you or others.

[michaelh53276323]

Hi folks,

for me nothing works from this solutions.

But after some hours i checked the backup folder from CF2016 Update 10 under

    C:\ColdFusion2016\cfusion\hf-updates\hf-2016-00010-314028\backup\jetty\lib

and there are many libs from jetty folder. And the jetty folder after update

     C:\ColdFusion2016\cfusion\jetty\lib

was empty.

I restore all subfolder from backupfolder into jetty folder and now the service would be start.

Maybe this helps for searching.

Have a nice day and take care.

Mike

[Charlie Arehart]

Mike, thanks for sharing your observation. it sounds like you had an error during the update process. It should of course not have left the lib folder empty (if indeed it was the cause). Good on you for finding the backup. For future reference, I have a post on how to look for indications of success or failure after applying CF updates, in case it may help any readers here:

Having problems after applying a CF update? What to check, and how to recover!

[bconner]

I'm having this issue as well on a 2016 install.  I applied the lockdown guide some time ago and am starting to move sites to it.  The Add-on Service will run if I give it an administrator account to use(!).  Had updated the JRE and updated the jetty.lax file to point to it.  Am using the same service account for the Application server and the Add-on Service.  The application server seems fine.

Any ideas?

--Ben

[bconner]

Ok, started the service with the administrator account and then monitored what it was doing with Procmon. It wants to write .tmp files into the c:\users\administrator\appdata\local\temp\ folder.  Example:

C:\Users\Administrator\AppData\Local\Temp\laxDAC9.tmp

I thought it couldn't write to the path of the account I would like to use (cfservice account) but I just checked the security permissions for C:\Users\cfservice\AppData\Local\Temp and the cfservice account has Full Control over that folder.

So I have no idea why it isn't starting when I switch the account to cfservice.  Hm.

--Ben

[lukester85]

Hey,

I have suffered this issue, the fix for me was to allow the 'cfusion' user ' list & read' permissions (not execute) to the D: drive.

For us, CF is installed in D,

Using ProcessMonitor:

Desired Access:

Read Data/List Directory, Synchronize

for CFUSION user.

To confirm my permissions:

D:\ColdFusion2016\ -- CFUSION = full control
D:\Java\ -- CFUSION = read & execute
D:\ -- CFUSION = list folder / read data (special)

Something is different between CF2016 and CF11, as didn't need to have CFUSION permissions on the drive at all before.

Hope this helps someone

Cheers
Luke

[Robert_Com99]

This fixed it for me as well. Thanks! I have seen this with some other programs over the years.