ColdFusion Server (CF 2016) suddenly will not start

[VincentL12]

I have been using ColdFusion 2016 for about 5 months without problem. Two days ago I suddenly could not access the built in server, receiving such messages as:

127.0.0.1 refused to connect. in Opera and ERR_CONNECTION_REFUSED in Chrome.

I thought it could be a firewall issue, since when booting up I saw a message saying that another app (Akamai NetSession)  was being blocked and that Windows Firewall had made some changes. However on checking the firewall, ColdFusion Server is being allowed through (as are Opera Chrome, etc).

I tried to start CFServer from a command prompt, but after navigating to the bin directory and typing ColdFusion.exe start I got no message of any kind.

I am on port 8501 with the server (8500 was used by and old version of CF11) but have never had any running problems.

I noted that the IIS start page appears on 127.0.0.1 when I checked the relevant options - so localhost is not completely disabled.

I am not an expert on server issues. Anyone any ideas? Thanks in advance for comments.

[carl type3]

Hi Vincent,

Try from CMD prompt as administrator:

-CD CF2016\cfusion\bin

-cfstart

Let me know what displays.

HTH, Carl.

[VincentL12]

The message shows:

Unable to initialise Security service  coldfusion.server.serviceException,,, WDDX packet parse error at line1 column 1  Content is not allowed in prolog... at coldfusion.security.SecurityManager.loadsecurity(SecurityManager.java1418) and similar error messages referencing java 1375, java 59, etc.

I recall downloading a Java update. Do I need to revert back to the previous version?

[Charlie Arehart]

Vincent, is that the FIRST message that appears on trying to start from the command line? or the last? If it's the last, what is the first? there may be a clue there.

Also, you say "CF won't start" but then you say that calls to the CF internal web server are failing. So if you look at Task Manager (if Windows, or the equivalent in Linux or OSX), is the coldfusion process running? If it is, then the issue is not that it's "not starting" but it's not working (subtle distinction). It may be useful to know, though.

As for your having installed Java recently, and that affecting either CF starting or working, yes, that could be the issue--maybe. If you had changed CF to point to a specific JVM, and you updated it, it may have removed key files from the JVM folder that CF WAS looking at. But because it was running the JVM update could not remove ALL files from that folder.

In this case, you'd want to tell CF to point to the NEW JVM folder. But of course, if you can't get to the CF Admin you may think you can't tell it to use the new JVM.

But if you look at your CF jvm.config file (in the coldfusion2016\cfusion\bin folder, or if on Enterprise, the bin folder of whatever instance you're trying to start if you have multiple instances), its first non-commented line will be for java.home. Does that point to the coldfusion2016\jre folder? or some other one? If another, then THAT was what CF WAS looking at (and still is trying), and you'd want to edit that to point to the NEW folder.

If you do edit it, watch that you need to use the same slashes you see listed for the current java.home value, and you must point it to the JRE folder of whatever JVM you're using (as your current value would do. Just be careful in updating that value to the new JVM location.)

I discuss all this in two extended posts with lots more to consider. And even if you think the above does not apply, you may still want to read them to be sure:

CF911: 'Help! I've updated the JVM which ColdFusion uses, and now it won't start!'

Why you should think twice about leaving on the "public JRE" option of the Java JDK installer

But your problem could be something else entirely. Please do answer the first question above, if the rest doesn't help.

And while we may be able to solve this via the forums here, sometimes there are just too many moving parts. In that case, there's no substitute for having someone instead look WITH you at your server to solve the problem, whatever the unique cause may be for you. I do that, as do others. You can find them at cf411.com/cfconsult. You may find it could be solved in less than an hour, so not necessarily a really expensive option. Some want a solution, some want to solve things on their own. I try to help either way. 🙂 Just wanted to put it out there for you and other readers to consider.

[VincentL12]

Charlie. Thanks for your detailed response.

The full logs are:

May 17, 2017 5:03:28 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\ColdFusion2016\cfusion\lib;C:\ColdFusion2016\cfusion\jintegra\bin;C:\ColdFusion2016\cfusion\jintegra\bin\international;C:\ColdFusion2016\cfusion\lib\oosdk\classes\win

May 17, 2017 5:03:29 PM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["http-nio-8501"]

May 17, 2017 5:03:29 PM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector

INFO: Using a shared selector for servlet write/read

May 17, 2017 5:03:29 PM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-nio-8016"]

May 17, 2017 5:03:29 PM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector

INFO: Using a shared selector for servlet write/read

May 17, 2017 5:03:29 PM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

May 17, 2017 5:03:29 PM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/8.0.32

May 17, 2017 5:03:31 PM org.apache.jasper.servlet.TldScanner scanJars

INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

05/17 17:03:32 INFO License Service: Flex 1.5 CF Edition enabled

05/17 17:03:32 INFO Starting Flex 1.5 CF Edition

May 17, 2017 5:03:32 PM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: Starting application services

May 17, 2017 5:03:32 PM org.apache.catalina.core.ApplicationContext log

INFO: ColdFusionStartUpServlet: ColdFusion: VM version = 25.72-b15

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Starting logging...

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Starting license...

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Developer Edition enabled

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Starting crypto...

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Installed JSafe JCE provider: Version 6.21 Crypto-J 6.2.1, EMC Corporation. JsafeJCE Security Provider (implements RSA, DSA, ECDSA, Diffie-Hellman, ECDH, AES, DES, Triple DES, DESX, RC2, RC4, RC5, PBE, MD2, MD5, RIPEMD160, SHA1, SHA224, SHA256, SHA384, SHA512, HMAC-MD5, HMAC-RIPEMD160, HMAC-SHA1, HMAC-SHA224, HMAC-SHA256, HMAC-SHA384, HMAC-SHA512, HMACDRBG, HASHDRBG, CTRDRBG, FIPS186PRNG, SHA1PRNG, MD5PRNG; RFC 3394, RFC 5649 AES Key Wrap; X.509 CertificateFactory; PKCS12, PKCS15 KeyStore; X.509V1, PKIX, PKIX-SuiteB, PKIX-SuiteBTLS CertPathValidators; X.509V1, PKIX, PKIX-SuiteB, PKIX-SuiteBTLS CertPathBuilders; LDAP, Collection CertStores)

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Starting security...

May 17, 2017 17:03:33 PM Error [localhost-startStop-1] - Unable to initialise Security service: coldfusion.server.ServiceException: coldfusion.wddx.WddxDeserializationException: WDDX packet parse error at line 1, column 1. Content is not allowed in prolog..

coldfusion.server.ServiceException: coldfusion.wddx.WddxDeserializationException: WDDX packet parse error at line 1, column 1. Content is not allowed in prolog..

        at coldfusion.security.SecurityManager.loadSecurity(SecurityManager.java:1418)

        at coldfusion.security.SecurityManager.load(SecurityManager.java:1375)

        at coldfusion.server.ServiceBase.start(ServiceBase.java:59)

        at coldfusion.security.SecurityManager.start(SecurityManager.java:1365)

        at coldfusion.server.CFService.setupSecurity(CFService.java:438)

        at coldfusion.server.CFService.start(CFService.java:560)

        at coldfusion.server.j2ee.CFStartUpServlet.startCFService(CFStartUpServlet.java:565)

        at coldfusion.server.j2ee.CFStartUpServlet.init(CFStartUpServlet.java:508)

        at javax.servlet.GenericServlet.init(GenericServlet.java:158)

        at coldfusion.bootstrap.ClassloaderHelper.initServletClass(ClassloaderHelper.java:121)

        at coldfusion.bootstrap.BootstrapServlet.init(BootstrapServlet.java:59)

        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)

        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)

        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)

        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5049)

        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5341)

        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)

        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408)

        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

May 17, 2017 17:03:33 PM Error [localhost-startStop-1] - Unable to initialise CFStartupServlet:coldfusion.wddx.WddxDeserializationException: WDDX packet parse error at line 1, column 1. Content is not allowed in prolog..

May 17, 2017 17:03:33 PM Information [localhost-startStop-1] - Shutting down servlet container...

I will look at what you suggest in the two extended posts, but in the meantime the above might throw a more focused light on the problem I have.

Vincent

[Charlie Arehart · Accepted solution]

Good news/bad news. The log confirms that the error you reported is indeed the first one in the log. And it explains the problem: a issue with C:\ColdFusion2016\cfusion\lib\neo-security.xml, which has a "parse error" at line 1, column 1.

I'm betting that if you open that file, you will find that it is empty. It shouldn't be. Or if it's not empty, it's mal-formed (not valid XML, or more specifically not valid WDDX-formatted XML).

The first question would be how it got empty or mal-formed. But that may take time to find and resolve.

But for now, you may want to check if the neo-security.bak file, in the same directory, is also empty or mal-formed. If it is not, and you renamed the other and renamed this to be neo-security.xml, and restarted CF, would things work (no more error, pages load)? But it may be borked also.

In that case, do you have any sort of backup of the file system that you could revert to?

Assuming you do not, sadly someone can't just "share theirs" with you because the info in the file is unique to each user. This file holds the CF admin security configuration settings (representing checkboxes checked, etc.). And anyone else's xml file would represent settings that they picked that you may set differently.

Even so, if you're desperate, here is the content of mine, from CF2016 also. (I wiped the value of my "userid.root.salt" value, as I'm pretty sure that's different on every install. It's not the password, but the salt value used to encrypt the password, stored in another file). Keep reading below this for a couple more thoughts:

<wddxPacket version='1.0'><header/><data><struct type='coldfusion.server.ConfigMap'><var name='AuthorizedUsers'><struct type='coldfusion.util.FastHashtable'></struct></var><var name='admin.userid.root'><string>admin</string></var><var name='CrossSiteScriptPatterns'><struct type='coldfusion.server.ConfigMap'><var name='&lt;\s*(object|embed|script|applet|meta)'><string>&lt;InvalidTag</string></var></struct></var><var name='admin.userid.root.salt'><string>00000000000000000000000000000000</string></var><var name='rds.enabled'><string>true</string></var><var name='allowconcurrentadminlogin'><boolean value='true'/></var><var name='allowedAdminIPList'><string></string></var><var name='contexts'><struct type='coldfusion.server.ConfigMap'><var name='/'><struct type='coldfusion.server.ConfigMap'></struct></var></struct></var><var name='admin.security.enabled'><boolean value='true'/></var><var name='admin.userid.required'><boolean value='false'/></var><var name='secureprofile.enabled'><boolean value='false'/></var><var name='rds.security.enabled'><string>true</string></var><var name='sbs.security.enabled'><boolean value='false'/></var><var name='rds.security.usesinglerdspassword'><boolean value='true'/></var></struct></data></wddxPacket>

You would want to stop CF, then edit your neo-security.xml putting this in its place. BEfore you do, read to the end for an edit you may want to make. (And if you're on Windows and edit it with Notepad and find you can't save the edits due to permissions issues, even though you ARE an Admin, this is a limit of Notepad. You could open it as Admin, or just save the changed file in your documents folder then copy/paste it with windows explorer, and THAT will prompt you to elevate your privileges to let you paste the file, while Notepad would not let you save it.)

Now, back to the file contents, because you would now have a different salt value, you'll find that your current CF Admin (and I think DSN) passwords will not work. So you'd find you could not login to the CF Admin. But note that you can set this element:

<var name='admin.security.enabled'><boolean value='true'/></var>

to

<var name='admin.security.enabled'><boolean value='false'/></var>

And now (after a restart) you could login, without any password. Then you could go to the CF Admin Security page, and modify the salt to some other value (than all 0's), and then enter a new password (and choose the drop down on that page to require a CF Admin password). Again, you may have to redo your DSN passwords. Try verifying one that worked. If it does not, just enter it again in the DSN settings page, and (because it will now be set with the salt you picked) it should verify now.

Let us know how it goes.    

[VincentL12]

Charlie. Many thanks for your latest response.

The neo-security.xml file was indeed empty.

I found a neo-security.xml file from last December and copied that across. I was then able to start the CF Server straight away.

However I could not open the CF Administrator, receiving an error "The DataSource file is not available". I then checked the neo-datasource.xml file and found this empty also. There is a neo-datasource.bak, but this was similarly empty. Further checks showed that the neo-drivers.xml, neo-secureprofile.xml and neo-icon.xml were also empty (no .bak versions exist). neo-document.xml has content.

I may have old versions of these files from CF10 or CF11 somewhere - it could take a while to find them.

What do you suggest is the best way forward?

[Charlie Arehart]

Well, you could install that same CF version on some other machine (or perhaps on this same machine) and bring the default neo-*.xml files over, to start from scratch. It may seem a tad painful, I know, but if you don't find the old versions, it may be the fastest option. (You may be considering reinstalling, but that's going to have FAR more impact, in loss of logs, loss of other config files besides those.)

1) First, if you may be running CF Enterprise, you could use the CF admin Enterprise Manager>Instance Manager page to create a new instance, which should take only moment. If you're not, you won't be able to install the same CF version again on the same machine, but again you could install the free trial or dev edition on another machine. I have still one more option at the end.

2) But I'm sure your main question is "how did this happen?". It's not common (but it has happened, often just to one file, commonly the neo-cron.xml, which holds scheduled tasks). Folks have found that it seems to happen when CF crashes and was in the process of updating them, so they end up empty (because it never finishes the update).

We could certainly plead for Adobe to fix the process so that it would never happen (there are safer ways, that are more robust if a crash happens mid-way). You'd want to open a feature request (or bug report) at tracker.adobe.com. There's no guarantee anyone from Adobe will see this thread here.

3) One last thing: I mentioned an alternative to installing CF again, on the same machine, even if you're running Standard. Look into Commandbox, which is a new way to get and run CF even on a machine that is already running it, in a way that would not conflict with that other running instance. The following post from early 2016 discusses it better (and though it indicates being in beta, that was then. It no longer is):

http://www.codersrevolution.com/blog/commandbox-multi-server-support-now-in-beta-v310

Let us know how you get going.

[VincentL12]

I found some copies of neo-datasource.xml, neo-secureprofile.xml and other neo*.xml files from about six months ago

and copied these across. The entry screen for CF Admin then came up, but I am being asked for a password (with user: admin).

I changed <var name='admin.security.enabled'><boolean value='true'/></var> to <var name='admin.security.enabled'><boolean value='false'/></var> in neo-security.xml but I am still being asked for a password at log-in (just clicking on the "log-in" button doesn't work).

I turned the CF Server off before re-starting, but it starts automatically on booting up.

I am not sure what I am doing wrong. If I can log in to CF Admin, I think I can sort out any subsequent issues.

[VincentL12]

Charlie. I am in the CF Administrator now.

Thanks a lot for all your communications. They have been a great help.

Vincent

[Charlie Arehart]

Glad to have helped, Vincent.  And I SO heard you say that in a Somali accent. 🙂 (For those who don't get the reference: https://www.youtube.com/watch?v=dvA-mimf2yg)

Thanks for marking my reply as the answer. Hope it may help others in the future.

(And sorry for the delay in responding. Sadly, I can't seem to get the forums to make it be the default that I get an email for any threads I participate in.)

[Paolo Broccardo]

Thanks for this solution, Charlie. I had the same issue and it just saved me a ton of time and possible having to reinstall CF. Luckily for me, the neo-security.xml.backup was still intact. 

[Charlie Arehart]

Glad to have helped, Paolo, and thanks for the kind regards.