CF11 Enterprise issue

[ksanford0317]

Had an issue with three of our CF11 Enterprise servers this weekend after a reboot of the VMs. Seven sites were down after the reboot with an error of “Address already in use: JVM_Bind” in the CF exception.log. All servers have multiple instances and each site has their database on separate db server. One server has 11 customer sites on it, each having a database on the same database server. Of these 11, 4 sites were down after the reboot. The error detail is:

"Error","localhost-startStop-1","07/23/16","09:34:19",,"Address already in use: JVM_Bind"

java.net.BindException: Address already in use: JVM_Bind

               at java.net.DualStackPlainSocketImpl.bind0(Native Method)

               at java.net.DualStackPlainSocketImpl.socketBind(DualStackPlainSocketImpl.java:106)

               at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:382)

               at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:190)

               at java.net.ServerSocket.bind(ServerSocket.java:375)

               at java.net.ServerSocket.<init>(ServerSocket.java:237)

               at java.net.ServerSocket.<init>(ServerSocket.java:128)

               at coldfusion.monitor.jetty.server.MonitoringServer.portCheck(MonitoringServer.java:237)

               at coldfusion.monitor.jetty.server.MonitoringServer.checkIfPortFree(MonitoringServer.java:208)

This happened on two other CF11 Enterprise servers, same error, same scenario (database on different db server).

I cannot find any info on what caused this or what we can do to prevent this from happening again. Any help is appreciated.

[ksanford0317]

**EDIT**

When I mention the sites were down, the odd thing is the CF service was running, the site in IIS was running, but customer got a 500 server error when trying to get to the URL. I also could not get into CF Admin on the server. A restart of the CF application server service resolved the issue.

[haxtbh]

This usually means the port number is being used. Most likely the port Tomcat is trying to use. I think its 8500.

I would check to see if port 80, 443 and the tomcat ports are being used by anything else. Were any changes made to the default config to change the ports it runs on?

[ksanford0317]

Sorry, late reply. To answer, nothing has changed on these. The sites run fine, just an issue whenever the VM is rebooted. It is also random, not always the same instance affected after a reboot. Website is not accessible, nor is CF Admin, but the service is running. All fine after a restart of the service.