Clustering Datasources

[haxtbh]

Is there an easy way to maintain datasources across multiple servers without having to log into each one?

We have a collections of servers which are all sat behind a load balancer. Each server runs an instance of CF and requires various datasources added (tenants) on a regular basis.

Currently we have to log into the admin of each one and add the datasource. Although not extremely time consuming, it would be handy to do it once from either a cental administrator or set up in a similar way to how they implements clustered scheduled tasks.

[Charlie Arehart · Accepted solution]

Bad news/good news there. While there is sadly no feature to cluster admin settings, fortunately there are at least two options to automate/synchronized admin settings.

[Update: I have since created a blog post, with the same info as below and more. Some readers may want to read that rather than my writing that follows. I'll welcome and reply to responses in either place.]

First, to clarify: that sched task feature was something inherited by Adobe, as part of the quartz open source package that provides the sched task mechanism. *It* offers the clustered task feature, based on a shared db between the instances.

So how can you synchronize cf admin settings?

The Adobe-provided solution is the cf "admin API", which was introduced in CF7 but has generally failed to be noticed by most. If you google that phrase, you'll find examples. (Do beware just blindly following one's showing setup of dsns. Make sure you are setting the RIGHT settings for your own dsns.)

Alternatively, you could leverage the cfconfig feature offered by the open source CommandBox tool. The cfconfig feature can be used even with servers that are NOT started/controlled by CommandBox.

While to some CF/Lucee devs these are two vital tools they use daily (especially for local development), they are still unknown to or unused by many, or they are not aware that cfconfig can be used with any cf or Lucee instance, by just pointing the tool at the instance's config folder.

Again, just google the phrase cfconfig and various resources can guide you to leveraging the tool for your need. Or again consider the cf adminapi if for some reason cfconfig won't work for you.

Finally, some may advocate/propose "you could just copy the neo xml files between the servers", or at least the neo-datasource.xml where DSN's are stored. (For the sake of other readers, I'll clarify that CF admin settings are nearly all saved in neo*.xml files, in the lib folder under your CF instance.) But copying those files between servers is not wise, for various reasons.

Even between two otherwise identical CF instances (same version, same updates, etc.), there are aspects of the files that may NOT be suited to be copied from one instance to another. For instance, I am pretty sure the password stored for a DSN is encrypted using info specific to the instance (and based on info in files other than those neo xml files). Also, beware that the neo-datasource.xml is also related directly to the neo-drivers.xml, so they should be regarded as a pair.

But I would STRONGLY recommend against copying the neo xml files between instances running DIFFERENT versions of CF, as the layout (elements, attributes, values) of the xml files can change. I realize some "have done it" and had "no problem", but I can tell you I have helped people overcome problems that were due to this practice.

All that said, some folks (out of need) have dug in and confirmed it WAS ok for them to copy some neo xml file, to make it easier to "keep them in sync".

And finally, let me advocate STRONGLY against an idea one may have to somehow point multiple CF instances at a single location of neo xml files (whether for DSNs only or for any of the Admin settings). While I'm not aware of any way in CF to name where it should lok for them, a savvy admin would know that both Linux and Windows (and MacOS) offer a way to create a symlink or alias, so that while an app may look for files in folder x, there will be no such real folder and the OS will find it files REALLY existing in folder y. While that's a great feature for many uses, it's unwise for the concerns I raised above (about copying neo xml files between two otherwise identical servers), let alone for possible race conditions if somehow changes were made to two instances at once (though I realize one could implement protections against that).

Anyway, there are several ideas. I think your best choices are in the first two, but I share the rest for consideration (and mostly recommending against them, for nearly all folks).

Let us know what you think and how it goes for you. And thanks for all your contributions over the years, in various cf online communities. Glad to be able to help when instead you're one doing the asking. 🙂

[haxtbh]

Hi Charlie,

Thanks for the response. Appreciate the long post as well, i'm sure it will go a long way for others too.

Had already been through most of what you suggested but the CFConfig tool is not one I was aware of and it looks like a good starting point!

[Charlie Arehart]

Thanks. And fwiw, I have since created a blog post with the same info and more: https://www.carehart.org/blog/client/index.cfm/2020/4/7/syncing_cfadmin_settings

PS Do you ever identify anywhere who this is, behind this haxtbh alias? 🙂 Or do you prefer to remain anonymous, perhaps?

[haxtbh]

Another one for the carehart.org bookmarks folder!

Forever anonymous I am, but if CF Summit ever comes to the UK,  I will buy you a beer 😉

[Charlie Arehart]

Thanks on all counts. And I really would love to see something like CF Summit held in the UK, like we used to have the CFUnited, SoTR, CF Europe, and other events in the years past. It's been TOO long (15 years since my last preso there). Would be great to connect...and "collect" that beer. 🙂