Apache Tomcat (on Linux) mod_jk Connector 403 Forbidden Error

[lynn___]

Hi All,

Hoping someone can provide some pointers to solve this.

I was successfully running on CF2023 update 11 but when I went to update 14 Apache Tomcat mod_jk connector stopped working - i've reverted back to update 11 and it still doesn't work - the browser returns 403 Forbidden

I've checked my server.xml, worker.properties, mod_jk_vhost.conf a million times and they seem to be correct.  I changed the Port being used by the connector in case that was a problem.

I'm seeing in mod_jk.log:

[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0130 63 68 65 2F 32 2E 34 2E 36 32 20 28 55 6E 69 78 - che/2.4.62.(Unix
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0140 29 20 4F 70 65 6E 53 53 4C 2F 33 2E 33 2E 32 20 - ).OpenSSL/3.3.2.
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0150 6D 6F 64 5F 6A 6B 2F 31 2E 32 2E 34 36 00 0A 00 - mod_jk/1.2.46...
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0160 0F 41 4A 50 5F 52 45 4D 4F 54 45 5F 50 4F 52 54 - .AJP_REMOTE_PORT
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0170 00 00 01 30 00 0A 00 0E 41 4A 50 5F 4C 4F 43 41 - ...0....AJP_LOCA
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0180 4C 5F 41 44 44 52 00 00 00 00 0A 00 10 4A 4B 5F - L_ADDR.......JK_
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 0190 4C 42 5F 41 43 54 49 56 41 54 49 4F 4E 00 00 03 - LB_ACTIVATION...
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1368): 01a0 41 43 54 00 FF 00 00 00 00 00 00 00 00 00 00 00 - ACT.............
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_send_request::jk_ajp_common.c (1883): (cfusion) request body to send 0 - request body to resend 0
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1563): received from ajp13 pos=0 len=29 max=65536
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1563): 0000 04 01 F4 00 15 49 6E 74 65 72 6E 61 6C 20 53 65 - .....Internal.Se
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1563): 0010 72 76 65 72 20 45 72 72 6F 72 00 00 00 00 00 00 - rver.Error......
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_unmarshal_response::jk_ajp_common.c (818): (cfusion) status = 500
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_unmarshal_response::jk_ajp_common.c (825): Number of headers is = 0
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1563): received from ajp13 pos=0 len=2 max=65536
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1563): 0000 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [warn] ajp_process_callback::jk_ajp_common.c (2263): (cfusion) AJP13 protocol: Reuse is set to false
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_reset_endpoint::jk_ajp_common.c (930): (cfusion) resetting endpoint with socket 19 (socket shutdown)
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_abort_endpoint::jk_ajp_common.c (900): (cfusion) aborting endpoint with socket 19
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] jk_shutdown_socket::jk_connect.c (931): About to shutdown socket 19 [127.0.0.1:20726 -> 127.0.0.1:8009]
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] jk_is_input_event::jk_connect.c (1410): error event during poll on socket 19 [errno=107] (event=16)
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] jk_shutdown_socket::jk_connect.c (1015): Shutdown socket 19 [127.0.0.1:20726 -> 127.0.0.1:8009] and read 0 lingering bytes in 0 sec.
[Wed Jun 18 08:31:41 2025] [301783:140594684778048] [debug] ajp_done::jk_ajp_common.c (3710): recycling connection pool for worker cfusion and socket

server.xml contains:

<Connector packetSize="65535" protocol="AJP/1.3" port="8009" redirectPort="8455" secret="27837689-fc18-446c-848a-ae113acf00c8" maxThreads="500" connectionTimeout="60000" tomcatAuthentication="false" address="127.0.0.1" allowedRequestAttributesPattern=".*" />

workers.properties contains:

heartbeat_interval=30
heartbeat_limit=90

#Start of workers.properties associated with 'cfusion'
worker.list=cfusion

worker.cfusion.type=ajp13
worker.cfusion.host=localhost
worker.cfusion.port=8009
worker.cfusion.heartbeat_servlet_path=/__cf_connector_heartbeat__
worker.cfusion.connection_pool_timeout=60
worker.cfusion.monitoringsecret=f68691fb-903c-4c39-9166-ddd7fa726cae
worker.cfusion.secret=27837689-fc18-446c-848a-ae113acf00c8
#End of workers.properties associated with 'cfusion'

I'm going crazy trying to solve this - dug through all suggestions I can find online.  I hope someone can help.  Thanks!

[BKBK]

Glad to hear that all this is happening in a pre-production environment. Then I would strongly recommend that you again attempt to install update 14. With one difference this time around.

If you encounter an issue with Update 14, don't revert to Update 11. Instead bring the issue to the forum. Then we can put our heads together and solve it. 

I am sure that, whatever the issues were last time - and there were quite a few (403 status, 500 status, NullPointerException, etc.) - they would have been compounded when you reverted to Update 11. Picture, for example, the installation hanging on to an artefact of Update 12 or Update 13.

[lynn___]

Adding my reply to Charlie to reply to BKBK too...

OMG!!! IT WAS THE GHOSTS!!!!

I upgraded to Update 15 (just came out!) and in general, it worked! I can open a basic .cfm page!!!! HOORAY!

I do have some issues I could use everyone's help on but at least at basic, it works. So relieved!

Here are the 1st 2 things I've encountered:

- The oracle package is not installed. You can install the package through the CLI package manager (/appl/ColdFusion2023/cfusion/bin/cfpm.sh) by running the command : install oracle.
       - I tried going to cfpm and running "cfpm>install oracle:2023.0.11.330706" and I get:
              Downloading the dependent package bcpkix-jdk15on-153.jar
              null
              Downloading the dependent package bcpkix-jdk15on-153.jar
              null
              oracle(2023.0.11.330706) package is already installed.

- Cannot find implementation class coldfusion.tagext.mail.MailTag for the mail tag
       - similarly I tried going to cfpm and running "cfpm>install mail:2023.0.11.330706" and I get:
              Downloading the dependent package bcpkix-jdk15on-153.jar
              null
              mail(2023.0.11.330706) package is already installed.

I've attached the coldfusion-out.log -- I see there is some potentially relevant info there

Any thoughts you all have to resolve these is, as always, very much appreciated. I couldn't have gotten here without your support!

Lynn

[Charlie Arehart]

Wow, so Lynne, first good to hear of the (partial) resolution. (And I really think if you'd redone update 14 it could have gone just as well on getting pages to run. But I sense you were running out of steam at that point.)

As for the new/ongoing problem, it's a larger one. Something in your applying the update IS STILL failing to work completely. It seems in this case (per the one lijg you shared) that it's a failure during the package update process.  That happens AFTER the main cf update, during the first startup of cf after thst

Notice in the coldfusion-out.log you shared, the packages to be updated are first UNINSTALLED, stating here:

Jul 9, 2025 13:52:13 PM Information [main] - Uninstalling the package sharepoint

(and dozens more such lines). Then it wants to INSTALL the package updates, but those fail starting with the first:

Jul 9, 2025 13:52:16 PM Information [main] - zip package will not be deployed as it is not installed.

It should NOT say that. And frankly I suspect you will find the problem (this time AND last time) started during the update, technically at the END of the update. And that's tracked in a different log, the update install log, found in the update 15 folder found under cfusion/hf-updates.

Look at about line 75 of that log, to see table counting successes and fatalerrors. Are there 0 errors?

Then look at the bottom of that large log, about 10 lines from the bottom. It should report trying to download the packages to be updated. Any error there?

I suspect that's where your problem starts. BTW, can this cf instance access the web? Or is there a firewall? It tries to access the url in the cf admin "package manager" page, and in its *settings" tab, which shows the packages url. 

[lynn___]

Not great news...  But i'm still happy to be anywhere other than where I was before.

I'm attaching the install log.  It generally looks like a lot of success.  I do see some errors in that big paragraph toward the bottom.

Sorry, I should have shared before that there is no access to the web - I'm doing an offline installation.

I'll check back again in the morning.

THANKS!!!!

[BKBK]

That is great news, @lynn___ . We're almost there.

Here is a suggestion for resolving the issues of type "XXX package will not be deployed as it is not installed":

1.  Empty the folder /bin/felix-cache (by deleting all its contents).
2.  Restart ColdFusion.
3.  With Administrator/Sudo permissions, launch the ColdFusion Package Manager (cfpm) tool.
        Then run the following cfpm command: install all
4.  Restart ColdFusion.

[lynn___]

Thanks for the suggestion @BKBK.  I was hoping that would be a silver bullet but no such luck.  I'm still getting the same errors.  I'm attached the output from cfpm and the latest coldfusion-out log.

[BKBK]

From the latest error messages and logs, it seems as if your installation is unable to download some dependency Jar files. These in particular, 

bcmail-jdk14-139.jar
bcpkix-jdk15on-153.jar
bcprov-jdk14-139.jar
bcprov-jdk15on-153.jar
bcprov-jdk18on-1.78.1.jar
cf-acrobat-1.1.jar
ib6coreutil-1.3.jar

Many packages depend on them. Their absence means those packages won't wok.

There is a way to correct such packages issues. It consists of downloading all the current ColdFusion 2023 packages yourself. To do so, proceed as follows (I am assuming the absolute path of the ColdFusion installation directory is /appl/ColdFusion2023):

1.  Stop ColdFusion.
2.  As before, empty the folder /bin/felix-cache (by deleting all its contents).
3.  Check to make sure the directory /appl/ColdFusion2023/bundles/ exists. If it does, then back it up by moving a copy of it, with all its contents intact, to a location outside ColdFusion. Then ensure that /appl/ColdFusion2023/bundles/ is empty (by deleting all its contents).
   If the directory doesn't exist to start with, then that is already an issue. In that case, create the directory.
   
4.  Check the firewall rules and ensure that ColdFusion has full access to /appl/ColdFusion2023/bundles/.
5.  With Administrator/Sudo permissions, launch ColdFusion's package manager tool,  cfpm;
6.  Now use the downloadrepo command to download the packages repository. To do so, type 
   downloadrepo /appl/ColdFusion2023/bundles 
   and press ENTER. It may take up to half an hour for the download to complete. So be patient. 
7.  Verify that the directory /appl/ColdFusion2023/bundles does indeed contain the downloaded packages.
8.  Edit /lib/neo-updates.xml and make sure it contains the following:

<packagesurl>file:///appl/ColdFusion2023/bundles/bundlesdependency.json</packagesurl>
<defaultpackagesurl>file:///appl/ColdFusion2023/bundles/bundlesdependency.json</defaultpackagesurl>

9.   Restart ColdFusion 2023.
10.  Check whether ColdFusion 2023 now works as expected.

[lynn___]

Thanks @BKBK 

When I try to run:

cfpm>downloadrepo /appl/ColdFusion2023/bundles

I get:
The packages repository /appl/ColdFusion2023/bundles/bundlesdependency.json is not accessible. You can only load the packages that are available locally in the /appl/ColdFusion2023/bundles directory.

[BKBK]

I am sorry to hear that, @lynn___ .

Here is an alternative solution.
Send an e-mail to cfsup[at]adobe.com and request from them a link to enable you to download the bundles. In the e-mail, state your Operating System, ColdFusion version and Update level.

The link will usually be to a ZIP file of the contents of the bundles directory. If so, then you should proceed as follows:

1.  Stop ColdFusion. 
2.  Empty the bundles folder in your installation.
3.  Extract the ZIP file and copy all its contents to your bundles folder.
4.  Restart ColdFusion.

[lynn___]

Your instruction had be delete all the contents from my bundles folder.  What had been in there was the contents of the "packages zip file" (https://cfdownload.adobe.com/pub/adobe/coldfusion/2023/packages/hotfix-packages-cf2023-015-330825.zi...) I got from the CF2023 Update 15 page (https://helpx.adobe.com/coldfusion/kb/coldfusion-2023-update-15.html), including the bundlesdependency.json file that cfpm is complaining about.

It sounds like what I did originally for the Update 15 is what you're asking me to do now.  Perhaps there's an issue with that Update 15 .zip file?  I'll try reaching out to cfsup and see if they can help.

[lynn___]

@BKBK @Charlie Arehart it turns out that the problem was that I should have left the Update 11 files alone in /appl/Coldfusion2023/bundles/ directory and uzipped coldfusion-2023-update-15.html into a new bundles directory and updated neo_updates.xml to reference the new Update 15 bundles directory.

It was able to be resolved by copying missing file bcpkix-jdk15on-153.jar from /appl/ColdFusion2023/cfusion/hf-updates/hf-2023-00015-330825/bundles/repo/ to /appl/ColdFusion2023/bundles/repo/ and restart CF.

Now to do more testing and see what other issues I might encounter.

PHEW!

[BKBK]

Inspired move, @lynn___ . I was also thinking in that direction. I think that, after your Update 11 installation, something in the environment has been preventing ColdFusion from downloading some of the new packages. So, was there a change in the environment after the installation of Update 11?  For example, a change in network access permissions? 

Because of that, I would not stop at copying just bcpkix-jdk15on-153.jar. I would compare the contents of /appl/ColdFusion2023/cfusion/hf-updates/hf-2023-00015-330825/bundles/repo/ and /appl/ColdFusion2023/bundles/repo/, file by file. (After sorting them aphabetically.) Then, if a file is present in the first directory, but not in the second, I will copy it to the second directory.

Otherwise, something else will break, sooner or later. Take Bouncy Castle as an example. Different versions of its Jar files are needed as dependencies of packages such as mail, pdf and document. 

[lynn___]

@BKBKI credit you for telling me to reach out cfsup... they very quickly suggested the fix for it

[BKBK]

@lynn___ , thanks for the update. I am glad that you finally have a fix.

Could you please share it -- as fully as possible? If someone else arrives here looking for a solution to a similar problem, you will be doing them a big favour.

[lynn___]

@BKBKI believe I have provided all information.  That's why I provided info on the solution i received offline.

[BKBK]

Hi @lynn___ , when you say "they [CFSup] very quickly suggested the fix for it", it's unclear what the fix is. Or am I missing something perhaps?