ColdFusion11 - can't start. Very few log details

[guilhermeark]

Hi All,

I have a CF 11 Server (sorry I know its pretty outdated - we are scheduling to update it)

Recently our server was restarted and CF couldnt start anymore.

The only CF log file being updated is the 

C:\ColdFusion11\cfusion\logs\coldfusion-error.log

Sorry for the images below but I can't copy and paste or even download the file as its outside my network. 

I have requested them to copy the error log, so as soon as I have it I will update here

But the only thing it states is 

In The windows events - I got this message

Following up some other discussions I have checked the jvm.config but it is pointing to the correct JAVA path.

Its my first post here so I'm sorry if I post in the wrong topic,

Thanks in advance

[Charlie Arehart]

Go to the command line, to the cf11 cfusion/bin folder, and run cfstart. That should give the explanation. If not clear to you, share the results here.

You say the Java path is correct. But if someone had java offer to remove "old jvms", it may have tried to remove the one in THAT folder, but it would have been unable to remove ALL the files in that folder, such that now cf can't start because of that.  Point cf to a supported jvm for cf11 and your cf update level.

I have blog posts to help with more on all this, but I'm typing on mobile and am short on time.  So let's see first what you may find. 

[guilhermeark]

Hi Charlie,

Thank you for the super fast reply, I did indeed see one of your comments to run cf by CMD as admin, I had tried that before and unfortunately got the same answer( I should have said it before, sorry).  I dont think I'm actually getting an error message, is just INFO warnings (the same from the logs).

It has been stuck in this message for more than 10 minutes last time.

If I start it via services it tells me that the service didnt start in time.

We have actually installed our JRE manually - which has been working fine also in other servers, I have compared with other servers that I have already restarted and the file jvm.config has the same settings

This is the folder

Ps: I still havent receive the log file, 

[Priyank Shrivastava.]

Hi, 

Can you please share the content of "context.xml"

Go to \ColdFusion11\cfusion\runtime\conf folder. 

[guilhermeark]

Hi Priyank,

I have checked the context.xml (it hasnt been updated since 2015) and saw that manager pathname = "" was commented out.
When checking with other servers I saw that that line was uncommented so when I uncommented it it worked.

Do you have any idea why that happened or could you explain me what that manager pathname does?

Thank you very much by the way 😃

[Charlie Arehart]

As indicated in the comment in that file, that manager element is about tomcat session persistence (and those are used in cf if you enable j2ee sessions). As it noted, if that element does not exist or is commented out, such sessions ARE persisted, otherwise by providing that empty pathname, the persistence is disabled. More here: 

https://tomcat.apache.org/tomcat-8.5-doc/config/manager.html#Persistence_Across_Restarts

Since you had not changed the value since 2015, it seems it had no negative impact for you unit recently. As for "what changed" then, it seems you are presuming Adobe changed something. I'd propose instead that either a) someone on your end may have changed your cf admin settings to enable that "j2ee sessions" feature. That would have started cf (tomcat under it) persisting those sessions. And they are persisted to a sessions.ser file (in the cfusion or [instance] folder \runtime\work\Catalina\localhost\tmp). That file may have become huge, causing cf to hang in loading it.

How would it become huge? And especially if you may find evidence (in the cf admin audit.log) that it was done months or years ago?

That leads to b) the other thing that may have "changed". Perhaps before your last restart, something had caused cf to create tens or thousands or even millions of sessions. Those would have been persisted, and then now be reloaded on cf's restart.

Your uncommenting of that manager line would PREVENT cf from doing that load (or save) of the sessions to that sessions.ser file.  You may even find that that file still exists, to confirm that it was large. (But I know that cf/Tomcat would delete it on startup if it was uncommented and so persistence being used). 

Fwiw, I talked about this more in a talk in 2012, when cf10 switched us to running on tomcat. You can find the key points of this topic from that talk quoted by someone else in a SO post here:

https://stackoverflow.com/a/15688494/90802

Let us know what you think or may find. 

[Priyank Shrivastava. · Accepted solution]

Hi,

One reason that Manager Pathname could be commented, if you have tried to create the cluster in CF. If you haven't then it will be really difficult to say why it happend. When we create the cluster, Tomcat comment the ManagerPathname and Add an entry for SessionPersistance.

You can uncomment this line and restart CF. It will work.

I hope this helps.

[Charlie Arehart]

But Priyank, since that file had not changed in years, I was focusing on what could have caused things to suddenly be a problem. Can you share your thoughts on what I've proposed? Or might you have another possible explanation for readers to consider?

But sure, what you pose could explain how that file was changed those years ago. Thanks for the observation. 

[Charlie Arehart]

[to clarify, this reply was my response to the OP's first reply to my first comment on Mar 10. Given the ui on mobile, its place in the threading hierarchy is unclear, so I've added this comment to clarify.] 

Ok, so the point of the cmd line was to make sure there was not an error there that you did NOT see perhaps in the cf logs.

So as Priyank notes, the next thing to question is the various xml config files for CF. He notes the context.xml. Others are in the cfusion/lib folder. You probably don't have any version control for those cf xml files, right? To know if they'd changed? The others change often, so just looking at file dates won't help much.

You mention having a working server to compare things. I'd highly recommend you compare the xml files between the working and failing cf instances. You may find some file got mangled on the failing instance. I would strongly recommemd the tooll Beyond Compare, and it's xmlsorted plugin to make quick work of such a comparison.

Finally, if it's important for you to get this resolved, then rather than all this back and forth and hours passing, I can offer direct help via a screenshare. For more, see carehart.org/consulting, for my rates, approach, satisfaction guarantee, and online calendar. Or we can all keep going here. 

[guilhermeark]

Hi Charlie and Priyank,

Thank you so much for the help, indeed you are both corrected.

This server had CF installed in a different way than I'm used seeing.

I dont know if the installer did try to use clusters but I do know we had updated the sessions to use J2ee.

As this didnt require a CF restart and had automatic effect I thought this wasnt the issue (also as we had tested on a dev box before)

The issue was that that line was commented out, as soon as I uncommented it, it worked.

So yes, this issue was caused because of the j2ee change and possibly by the weird install.

I appreciate you both for the time and effort.

Thanks,\

Guil

[Charlie Arehart]

Thanks very much for the update, the clarifying replies, and the kind regards. 

[BKBK]

In The windows events - I got this message

Following up some other discussions I have checked the jvm.config but it is pointing to the correct JAVA path.

By @guilhermeark

Indeed, I think the likeliest cause of the problem is the Java path.

You wrote later in this thread,

"We have actually installed our JRE manually"

Yet the Java directory that you showed has the path C:\ColdFusion11\jre. This is, presumably, not the Java you manually installed, but the one that ships with ColdFusion 11. 

I shall now suggest some steps you can follow to see if they solve the problem. You say you have updated ColdFusion 11, so I shall assume it can run on Java 8. [Charlie Arehart's CF/Java compatibility table shows that any update of ColdFusion 11 newer than Update 3 may use Java 8].

1.  Go to the ColdFusion Downloads page. Scroll all the way down to the section "Additional Third party Installers". Click on the link to select the Java version, and choose Java SE 8u311.
2.  Download jdk-8u311-windows-x64.exe (Size 170 MB). Install it, using the default options (to the same server as CF11). This Java installation's home directory will then be, for example, C:\Program Files\Java\jdk1.8.0_311\.
3.  Open in a text editor the file C:\ColdFusion11\cfusion\bin\jvm.config. Set ColdFusion to point to the new Java home:
   

    java.home="C:/Program Files/Java/jdk1.8.0_311/jre"
   

   
   Save the jvm.config file.

4.  Open Windows Services and restart ColdFusion 11.
   
   Any luck? 
   
   

[Charlie Arehart]

Bkbk, perhaps you skipped over all the other comments, but it became clear after the OP's first reply to my first comment that this problem is not about the jvm.

More than that, the last few comments from Priyank and myself (and the OP, guilhermeark) help explain how the issue seems instead to be about session persistence, due to their uncommenting (in the past)  the manager config element in the context.xml.

We're waiting to hear confirmation from guilhermeark about suggestions and observations we'd made.

Still, what you share may well help someone, so thanks for offering it, and for the reference. 

[BKBK]

Charlie, you have wrongly presumed that I "skipped over all the other comments". I read them all. After that, I said what I think the cause is and how to fix the problem. 

[Charlie Arehart]

Really? After reading all that, and THEN reading me say, "it became clear after the OP's first reply to my first comment that this problem is not about the jvm", you still want to stand on that prospect?  If it was the jvm, it wouldn't have gotten into the log entries in did.

And even when the 3 others of us discussed what seems clearly a session persistence, you want to stand on that jvm issue? And even further now press the heap size?

Why not just wait for the op to respond to the other more specific and likely scenario and solutions offered before sending them and other readers chasing down other rabbit holes?

Again, if you'd really read all the above before making your first comment, we shouldn't we'd to be having this discussion.

We all--you especially--do plenty to help here. It just seems terribly inefficient to have jumped in here with new ideas when the current line of investigation looked promising and was awaiting the OP's response. 

[BKBK]

Charlie, what's "inefficient" about ideas and suggestions? Have you ever known a software developer who cannot handle two or more suggestions at a time? 

On the contrary. What you suggest - the examining of suggestions serially, one after the other - is in fact the less efficient problem-solving strategy. Which is why you won't find any forum that recommends it.

We've had this discussion before. What you propose smacks of  control. To me, at least.

This is a free forum. Anyone may propose any idea at any time. The more ideas, the better. That is in fact a more efficient way of solving complex problems.

[Charlie Arehart]

I do disagree and stand by all I said. 

[guilhermeark]

I do agree with Charlie,

And we had installed that java 8 inside the path Coldfusion/jre

This issue wasnt related to jvm

[BKBK]

A question, @guilhermeark : What are the values of heap size? (That is, the Xmx and Xms values in the jvm.config file.)

[neowire]

@guilhermeark 

I remember having this problem in my environment often back when my team was using CF11. When we were having issues with ColdFusion starting up, it was because the ColdFusion services and W3SVC services were starting at inappropriate times, and it was causing conflicts.

I was able to get around the issue by ensuring the the ColdFusion services started before the W3SVC. Go into your Windows Services window and set W3SVC to Automatic (Delayed) and if you are seeing what I was seeing it should resolve it.

Additionally, if you ever have to restart the ColdFusion services, you must restart the ColdFusion and W3SVC services in appropriate order.

Get-Service -ComputerName $Server -DisplayName 'ColdFusion *' | Stop-Service -Force -Verbose
Get-Service -ComputerName $Server -Name W3SVC | Stop-Service -Force -Verbose
Get-Service -ComputerName $Server -DisplayName 'ColdFusion *' | Start-Service -Verbose
Get-Service -ComputerName $Server -Name W3SVC | Start-Service

Hope this helps...

[guilhermeark]

Hi Neo,

Thanks for your feedback, I did think that this could have been an issue so I did stop w3svc, but even after restarting both services in the right order there was no difference on the issue.

This was indeed caused by the j2ee session variables update.