ColdFusion PMT will not start

Forum|Forum|24 days ago
July 17, 2026
14 replies
100 views

I just discovered that my ColdFusion PMT 2023 server had run out of disk space on the volume where the PMT is installed (separate disk from where Windows is installed). This is a dedicated server running on Windows Server 2022.

I added more space to the volume and tried to restart both the data store service and the PMT service. The datastore service will start, but the PMT service will not. The Windows Event log had no useful information about why it won't start. The PMT exception log says:

[ERROR] 2026-07-17 12:38:46.675 com.adobe.pms.es.client.ElasticSearchClient - Datastore Service not available. Retrying to connect...
[ERROR] 2026-07-17 12:39:19.284 com.adobe.pms.es.client.ElasticSearchClient - Datastore Service not available. Shutting down Performance Management Suite...

This is even though the Datastore Service is running. Here is an excerpt from the elasticsearch log to show this:

[2026-07-17T13:15:19,453][INFO ][o.e.n.Node               ] [REDACTED] version[8.2.0], pid[1188], build[default/zip/b174af62e8dd9f4ac4d25875e9381ffe2b9282c5/2022-04-20T10:35:10.180408517Z], OS[Windows Server 2022/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/17.0.6/17.0.6+9-LTS-190]
[2026-07-17T13:15:19,453][INFO ][o.e.n.Node               ] [REDACTED] JVM home [D:\ColdFusion2023PerformanceMonitoringToolset\jre], using bundled JDK [false]
[2026-07-17T13:15:19,453][INFO ][o.e.n.Node               ] [REDACTED] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\1\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:MaxDirectMemorySize=2147483648, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Delasticsearch, -Des.path.home=D:\ColdFusion2023PerformanceMonitoringToolset\datastore, -Des.path.conf=D:\ColdFusion2023PerformanceMonitoringToolset\datastore\config, -Des.distribution.flavor=default, -Des.distribution.type=zip, -Des.bundled_jdk=true, exit, abort, -Xms4095m, -Xmx4095m, -Xss1024k]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [aggs-matrix-stats]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [analysis-common]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [constant-keyword]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [data-streams]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [frozen-indices]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [ingest-common]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [ingest-geoip]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [ingest-user-agent]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [kibana]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [lang-expression]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [lang-mustache]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [lang-painless]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [legacy-geo]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [mapper-extras]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [mapper-version]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [parent-join]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [percolator]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [rank-eval]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [reindex]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [repositories-metering-api]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [repository-azure]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [repository-encrypted]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [repository-gcs]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [repository-s3]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [repository-url]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [runtime-fields-common]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [search-business-rules]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [searchable-snapshots]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [snapshot-based-recoveries]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [snapshot-repo-test-kit]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [spatial]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [transform]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [transport-netty4]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [unsigned-long]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [vector-tile]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [vectors]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [wildcard]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-aggregate-metric]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-analytics]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-async]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-async-search]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-autoscaling]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-ccr]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-core]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-deprecation]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-enrich]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-eql]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-fleet]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-graph]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-identity-provider]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-ilm]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-logstash]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-ml]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-monitoring]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-ql]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-rollup]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-security]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-shutdown]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-sql]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-stack]
[2026-07-17T13:15:43,719][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-text-structure]
[2026-07-17T13:15:43,734][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-voting-only-node]
[2026-07-17T13:15:43,734][INFO ][o.e.p.PluginsService     ] [REDACTED] loaded module [x-pack-watcher]
[2026-07-17T13:15:43,734][INFO ][o.e.p.PluginsService     ] [REDACTED] no plugins loaded
[2026-07-17T13:15:45,187][INFO ][o.e.e.NodeEnvironment    ] [REDACTED] using [1] data paths, mounts [[CFPMT_App (D:)]], net usable_space [50.6gb], net total_space [99.9gb], types [NTFS]
[2026-07-17T13:15:45,187][INFO ][o.e.e.NodeEnvironment    ] [REDACTED] heap size [4gb], compressed ordinary object pointers [true]
[2026-07-17T13:15:52,328][INFO ][o.e.n.Node               ] [REDACTED] node name [REDACTED], node ID [YP0WvIWLTm-Kyz4SmaeK1Q], cluster name [elasticsearch], roles [ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold]
[2026-07-17T13:15:58,984][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [REDACTED] [controller/5104] [Main.cc@123] controller (64 bit): Version 8.2.0 (Build a8c0a88ede0ff2) Copyright (c) 2022 Elasticsearch BV
[2026-07-17T13:15:59,383][INFO ][o.e.x.s.Security         ] [REDACTED] Security is disabled
[2026-07-17T13:16:00,781][INFO ][o.e.t.n.NettyAllocator   ] [REDACTED] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2026-07-17T13:16:00,828][INFO ][o.e.i.r.RecoverySettings ] [REDACTED] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2026-07-17T13:16:00,891][INFO ][o.e.d.DiscoveryModule    ] [REDACTED] using discovery type [multi-node] and seed hosts providers [settings]
[2026-07-17T13:16:02,219][INFO ][o.e.n.Node               ] [REDACTED] initialized
[2026-07-17T13:16:02,219][INFO ][o.e.n.Node               ] [REDACTED] starting ...
[2026-07-17T13:16:02,484][INFO ][o.e.x.s.c.f.PersistentCache] [REDACTED] persistent cache index loaded
[2026-07-17T13:16:02,484][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [REDACTED] deprecation component started
[2026-07-17T13:16:02,641][INFO ][o.e.t.TransportService   ] [REDACTED] publish_address {REDACTED/REDACTED:9350}, bound_addresses {[::1]:9350}, {REDACTED:9350}
[2026-07-17T13:16:05,656][INFO ][o.e.b.BootstrapChecks    ] [REDACTED] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2026-07-17T13:16:05,656][INFO ][o.e.c.c.Coordinator      ] [REDACTED] cluster UUID [FGyOyIeHQAaiCbe-no8xow]
[2026-07-17T13:16:05,813][INFO ][o.e.c.s.MasterService    ] [REDACTED] elected-as-master ([1] nodes joined)[_FINISH_ELECTION_, {REDACTED}{YP0WvIWLTm-Kyz4SmaeK1Q}{fhLO0IHOSUafnSBCX-IdeQ}{REDACTED}{REDACTED:9350}{cdfhilmrstw} completing election], term: 16, version: 10034, delta: master node changed {previous [], current [{REDACTED}{YP0WvIWLTm-Kyz4SmaeK1Q}{fhLO0IHOSUafnSBCX-IdeQ}{REDACTED}{REDACTED:9350}{cdfhilmrstw}]}
[2026-07-17T13:16:06,109][INFO ][o.e.c.s.ClusterApplierService] [REDACTED] master node changed {previous [], current [{REDACTED}{YP0WvIWLTm-Kyz4SmaeK1Q}{fhLO0IHOSUafnSBCX-IdeQ}{REDACTED}{REDACTED:9350}{cdfhilmrstw}]}, term: 16, version: 10034, reason: Publication{term=16, version=10034}
[2026-07-17T13:16:06,154][INFO ][o.e.h.AbstractHttpServerTransport] [REDACTED] publish_address {REDACTED/REDACTED:9250}, bound_addresses {[::1]:9250}, {REDACTED:9250}
[2026-07-17T13:16:06,154][INFO ][o.e.n.Node               ] [REDACTED] started
[2026-07-17T13:16:06,359][INFO ][o.e.l.LicenseService     ] [REDACTED] license [2d657ab3-52b0-48bd-80c4-5f8866a7a032] mode [basic] - valid
[2026-07-17T13:16:06,359][INFO ][o.e.g.GatewayService     ] [REDACTED] recovered [646] indices into cluster_state
[2026-07-17T13:18:43,081][INFO ][o.e.c.r.a.AllocationService] [REDACTED] current.health="YELLOW" message="Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[generic_settings][0], [archive_settings][0], [monitor_settings][0]]])." previous.health="RED" reason="shards started [[generic_settings][0], [archive_settings][0], [monitor_settings][0]]"

The server name and IP address have been replaced with REDACTED in the log excerpt above.

Am I going to have to reinstall PMT again to get this working?

Correct answer Charlie Arehart

Hi, Carl. I think I can help. Pardon the wall of text, but working “from the outside” with you on this, we have to both assess what you’ve offerred and propose diagnostic steps and then perhaps a solution (if I have guessed right what the problem is).

And this PMT stuff CAN be frustrating, as the available diagnostics aren’t always clear about what’s happening. (And naturally one can wonder “what changed?” since it last worked, but people will generally say “nothing”. That often means they’re not AWARE of some change that led to the problem.)

1) So my first guess would be that the PMT Service is configured to reach out to the PMT Datastore Service (elasticache) using a hostname which does NOT resolve to one of those IP addresses you show it listening on, one of those listed as “redacted” in the elasticsearch.log: the lines talking about “bound_address” and “publish_address”.

1a) Where would one find the hostname it uses? Open the application.properties file in in the top-level pmt “config” file. Per your logs here, that would be D:\ColdFusion2023PerformanceMonitoringToolset\config\application.properties. In that (which is barely 20 lines long typically), find the datastore.host line (you’ll see also the related datastore.port). These are what the PMT uses to connect TO that datastore. It may be a machine name, domain name, or perhaps even an ip address. (If it is an ip, does it match any of those you redacted above?)

1b) Assuming the value is not an IP, go to the command line of the machine running the PMT (don’t do this on any other machine) and do a ping of that hostname value. (It shouldn't matter if the machine you’re trying to ping allows ping. But you could also use tracert instead of ping.) First, does that hostname resolve to anything at all? And if so, what IP address does it report that resolving to?

1c) And again, does it match any of those you had redacted above?

If not, perhaps what “suddenly happened” is something changed so that the hostname used no longer exists, or something changed what that hostname resolves to. It could even be that it used to resolve to an ipv4 address and something in the environment (not CF nor the pmt nor leasticsearch) changed so that it now resolves to an ipv6 address.

2) So what can you do to “solve things”, assuming this is indeed the issue?

Well, you know from the log above what the PMT Datastore (elasticsearch) is willing to listen on (how it’s willing to be talked to from others, like the PMT). You may find that while the bound_address values show ip addresses (perhaps both ipv4 and ipv6), note that the first publish_address starts (typically) with a hostname (yours is redacted).

2a) Assuming you find you CAN ping that hostname successfully, you could try at least just putting that name into the datastore.host value in the application.properties. Keep the file open (and note whatever the value was before changing it), then try starting the pmt service. Be patient: if it can’t reach the host/elastisearch (for whatever reason), the service start will hang for a while as it reattempts things, then will fail (as perhaps you’ve experienced many times).

But if it starts, may there be rejoicing. :-) And then hopefully when you visit the PMT UI itself it will come up (the PMT UI URL is indicated in the last line of the perfmontoolset.log, after it HAS just started successfully). Perhaps you were already familiar with that, if you’d been using the PMT until things “stopped working”.

3) Of course, getting the PMT itself up and seeing that UI is one challenge--another can be datastore issues, like the original disk space problem you reported. And indeed, there’s then the whole matter of configuring the PMT to connect to CF instances (and then the CF instances need to connect to the datastore themselves). It can be a lot to juggle.

And it may be even that what I offer above doesn’t solve things. How else can you dig in more on you rown?

3a) First WRT to your original matter of the datastore running out of disk space, or the memory of the box not being enough for the pmt and its datastore service, note that the CF docs does in fact have a section on configuring/tuning the pmt, though it’s referred to as a section within the PMT docs called the “deployment guide”, which may not be so obvious:

https://guides.adobe.com/coldfusion/en/docs/coldfusion-tools/deployment-guide.html

3b) Perhaps more valuable--especially for kind of diagnosis and discussion of pmt file names and properties as above--much of that thornier stuff is generally NOT covered in any of the PMT docs. But here’s great news.

A lot of it WAS covered in a very helpful preso that Nimit from Adobe did back in 2020 (the PMT came out with CF2018, though many are unaware of it). That video was recorded and eventually put on youtube, and it has a lot of valuable technical detail (which never made it into the docs). It is a level of detail that most may not appreciate (who just “use” the PMT). But for what you’re facing, “it’s gold, Jerry” (Seinfeld reference).

Before I just give you the video link, let me warn you: the audio is quite loud. Lower your volume first. And I warned of this in an extended comment I offered about the video on the page where the preso was first announced--and which used to only offer a reg link that eventually no longer worked). You’ll see that I offered those observations and more, including a TOC for the video in my comment here. And there I also offer the yt link (where I also copied the TOC as a comment).

Hope all that’s helpful. Let us know how things go, Carl. And if my guess above is wrong, and yet none of the other info or resources help you, let us know where things stand and perhaps we can dig in further here.

And of course, this is the kind of troubleshooting I help folks with directly via remote screensharing, where we might solve such a challenge in far less than an hour. It can take longer to write/read here, and of course there’s a time gap between our writing. As a reminder, I have a satisfaction guarantee: no one will pay for time they don’t find valuable. More at carehart.org/consulting.

I share that last point as much for others who may see this. The PMT will likely be a challenge for many folks for years to come, who may find and appreciate this thread.

C

CVonStettenAuthor

Participating Frequently

Just working through step 1 above, and my application.properties file only has three lines (first line is a commented date stamp from this morning, followed by:

perfmontoolset.ip=127.0.0.1
perfmontoolset.pmt_id=<redacted GUID>

I went back to my oldest backups (from May of this year) and the file was quite a bit longer (and included the settings you referred to). I restored that file and now PMT is starting!

Thank you for your insight into the properties file. I would not have guessed that it had been truncated.

Charlie Arehart

Community Expert

Great to hear, and glad to have helped. As for how it happened, I'd not heard of it before (with this file), but some have seen it happen with cf's own neo-*.xml files. My guess is that it can happen when the services are writing to the file(s) right when the service may crash.

Thanks also for marking my comment as an answer. I'm adding that link to give context here, since you've replied at the top level. Also, I will mark your answer here as also helpful to viewers.

/Charlie (troubleshooter, carehart. org)

C

CVonStettenAuthor

Participating Frequently

I meant to reply inline with your comment, but must have clicked the wrong button. Thank you for closing the loop for everyone else!

Charlie Arehart

Correct answer