Copy link to clipboard
Copied
Trying to install and connect to the PDFg Service. Windows Server. Installation of Add-On Services goes just fine and it starts up the Windows Service just fine. In CF Admin, under the PDF Service page, I try to "Verify" the connection and it fails saying "Connection verification failed for PDF Service Manager. Looking at the logs in the Coldfusion Add-On Services folder, the stderrout.log shows the service successfully starting up on port 8993. However the next several lines show the error from when I try to verify the service. I can't find any similar issues like this so turning to the community for help.
Note: neo-document.xml has the same port (8993) that the service is running on.
Any other ideas out there? Log entries are below:
Logs Upon Service Startup:
2022-01-31 22:08:34.374:INFO:oejs.Server:main: jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 11.0.13+10-LTS-370
2022-01-31 22:08:34.405:INFO:oejdp.ScanningAppProvider:main: Deployment monitor [file:///C:/ColdFusionAdd-onServices/webapps/] at interval 1
2022-01-31 22:08:34.827:INFO:oejs.session:main: DefaultSessionIdManager workerName=node0
2022-01-31 22:08:34.827:INFO:oejs.session:main: No SessionScavenger set, using defaults
2022-01-31 22:08:34.827:INFO:oejs.session:main: node0 Scavenging every 600000ms
2022-01-31 22:08:34.874:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@3e62d773{AdminServlet,/AdminServlet,file:///C:/ColdFusionAdd-onServices/work/jetty-0_0_0_0-8993-AdminServlet_war-_AdminServlet-any-/webapp/,AVAILABLE}{C:\ColdFusionAdd-onServices\webapps\AdminServlet.war}
2022-01-31 22:08:34.983:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@22ffa91a{PDFgServlet,/PDFgServlet,file:///C:/ColdFusionAdd-onServices/webapps/PDFgServlet/,AVAILABLE}{C:\ColdFusionAdd-onServices\webapps\PDFgServlet}
2022-01-31 22:08:34.999:INFO:oejs.RequestLogWriter:main: Opened C:\ColdFusionAdd-onServices\logs\2022_02_01.request.log
2022-01-31 22:08:35.030:INFO:oejs.AbstractConnector:main: Started ServerConnector@68eedb9e{HTTP/1.1, (http/1.1)}{0.0.0.0:8993}
2022-01-31 22:08:35.045:INFO:oejs.Server:main: Started @2137ms
Logs Upon Verifying from CF Admin
2022-01-31 22:10:25.233:WARN:oejshC.PDFgServlet:qtp1794717576-24: unavailable
java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
at coldfusion.pdf.service.CFPDFGServicePool.createPDFgServicesPool(CFPDFGServicePool.java:91)
at coldfusion.pdf.service.CFPDFGServicePool.<init>(CFPDFGServicePool.java:53)
at coldfusion.pdf.service.CFPDFGServicePool.getServicePoolInstance(CFPDFGServicePool.java:59)
at coldfusion.pdf.service.CFPDFGServiceManager.initPDFServiceManager(CFPDFGServiceManager.java:77)
at coldfusion.pdf.service.remote.PDFGServlet.init(PDFGServlet.java:228)
at javax.servlet.GenericServlet.init(GenericServlet.java:244)
at org.eclipse.jetty.servlet.ServletHolder$WrapperServlet.init(ServletHolder.java:1289)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:602)
at org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:473)
at org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:728)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:564)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:834)
2022-01-31 22:10:25.237:WARN:oejs.HttpChannel:qtp1794717576-24: /PDFgServlet/verify
javax.servlet.ServletException: javax.servlet.ServletException: PDFgServlet@9ac5bab0==coldfusion.pdf.service.remote.PDFGServlet,jsp=null,order=-1,inst=true,async=false
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:162)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by:
javax.servlet.ServletException: PDFgServlet@9ac5bab0==coldfusion.pdf.service.remote.PDFGServlet,jsp=null,order=-1,inst=true,async=false
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:620)
at org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:473)
at org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:728)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:564)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by:
java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
at coldfusion.pdf.service.CFPDFGServicePool.createPDFgServicesPool(CFPDFGServicePool.java:91)
at coldfusion.pdf.service.CFPDFGServicePool.<init>(CFPDFGServicePool.java:53)
at coldfusion.pdf.service.CFPDFGServicePool.getServicePoolInstance(CFPDFGServicePool.java:59)
at coldfusion.pdf.service.CFPDFGServiceManager.initPDFServiceManager(CFPDFGServiceManager.java:77)
at coldfusion.pdf.service.remote.PDFGServlet.init(PDFGServlet.java:228)
at javax.servlet.GenericServlet.init(GenericServlet.java:244)
at org.eclipse.jetty.servlet.ServletHolder$WrapperServlet.init(ServletHolder.java:1289)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:602)
at org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:473)
at org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:728)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:564)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:834)
Copy link to clipboard
Copied
Which logs are those?
What happens when you launch the following test code as a CFM page?
<cftry>
<cfhttp url="http://127.0.0.1:8993/PDFgServlet/verify" method="get" result="httpResult"></cfhttp>
<cfdump var="#httpResult#" label="Verify PDFg service using CFHTTP - Result">
<cfcatch type="any" >
<cfdump var="#cfcatch#" label="Verify PDFg service using CFHTTP - Error">
</cfcatch>
</cftry>
If you continue to get java.util.ConcurrentModificationException with the above test, it will mean race conditions. That is, two or more threads are simultaneously trying to get access to the same servlet. That will in turn suggest that two instances of the PDFg service are running.
Might the root cause be as follows, You first installed an instance of the PDFg service along with ColdFusion. You then followed that with a second, standalone instance of the service.
Copy link to clipboard
Copied
Thanks BKBK,
Entirely possible that it is installed twice but wouldn't there be a /jetty folder then underneath /cfusion?
The logs I referred to were logs that are in the "ColdFusion2021Add-OnServices\logs" folder.
When I run the code you provide, I just get a generic 500 Server error as follows.
How can I tell if another instance is running (there is only one in the Windows Services).
Copy link to clipboard
Copied
BKBK, another potential possibility is we copied the settings from the CF 2016 installation and did a migrate. On the 2016 Server, it definitely was installed with ColdFusion. Is it possible CF 2021 thinks that is still the case because of migrating the settings and then installing it as an Add-On Service, threw this into a race condition as you mention? Not sure but thought I'd add this info.
Copy link to clipboard
Copied
BKBK, another potential possibility is we copied the settings from the CF 2016 installation and did a migrate. On the 2016 Server, it definitely was installed with ColdFusion. Is it possible CF 2021 thinks that is still the case because of migrating the settings and then installing it as an Add-On Service, threw this into a race condition as you mention?
By @MikePelant
That is worth looking into. Could you share the full list of settings that you copied?
Copy link to clipboard
Copied
I have a change of mind. The root cause of the problem is unlikely to be double PDFg services. I am ruling this out after a test.
When I installed ColdFusion 2021, I enabled the installation of the accompanying PDFg service. On the PDF service page in the ColdFusion Administrator, the PDF service verified OK on port 8993.
Following your comments, I decided to test my own hypothesis. So I downloaded and installed the standalone Add-on Services alongside ColdFusion 2021.
However, everything seems to work just fine. There are no errors - in fact, no log files - in C:\ColdFusionAdd-onServices\logs. Plus, in the ColdFusion Administrator, the PDF service again verifies OK on port 8993. That is why I am ruling out double PDFg services as the root cause.
But that test gave me another idea. What if you install the PDF service that is packaged in ColdFusion? For example, via the ColdFusion Administrator. You could, for test purposes, install all packages available in CF2021 to ensure that you include all possible dependencies.
Copy link to clipboard
Copied
@BKBK , after all of your questions regarding whether it was installed during the intial installation, I did some more digging and I'm embarrassed to admit it DOES appear that it was installed originally. Therefore, no need for the separate Add-On Services installation. That would likely explain the ConcurrentModificationException.
So, with that knowledge, I completely uninstalled the Add-On installation, rebooted the server, and went back into the Admin Console to verify the PDF Service. Still getting the "Connection verification failed" message and the Server.log file indicating:
"Error","http-nio-8500-exec-8","02/02/22","09:58:45","","Connect to 127.0.0.1:8987 [/127.0.0.1] failed: Connection refused: connect http://127.0.0.1:8987/PDFgServlet/verify"
"Error","http-nio-8500-exec-8","02/02/22","09:58:45","","Connection verification failed. The root cause is : Connect to 127.0.0.1:8987 [/127.0.0.1] failed: Connection refused: connect"
Also, there is nothing in the Server.log file when the CF Service is restarted, that indicates the PDFg Service is running on a specific port. All I see that seems to be related to the PDFg service is "Package htmltopdf started...".
I've even tried to remove the htmltopdf package and re-add it (restarting CF inbetween). I still can't get it to connect.
So, I guess I'm back to the beginning here.
Copy link to clipboard
Copied
I can see there has been a change in port number.
Copy link to clipboard
Copied
Thanks again for all of the suggestions. The port number was different in my response as I was trying different ports. Anyway, I'm going to have to come back to this probably early next week as other priorities are getting in my way. One note, however. There is no jetty folder under the cfusion directory leading me back to my original assumption that the PDFg Service was not installed with ColdFusion. In fact, when this server was implemented, we installed just the core ColdFusion and then added packages on top of it (including htmltopdf). At any rate, I'll come back to this next week and provide you some more updates.
Copy link to clipboard
Copied
One note, however. There is no jetty folder under the cfusion directory leading me back to my original assumption that the PDFg Service was not installed with ColdFusion. In fact, when this server was implemented, we installed just the core ColdFusion and then added packages on top of it (including htmltopdf).
By @MikePelant
Did you try installing the PDF package via the Package Manager in the Administrator?