Per-application datasource not found after service restart.

[bennadel]

I'm running into a rather strange issue and I don't know how to debug it. I'm using a per-application datasource feature in the `Application.cfc`:

this.datasources[ "bennadel" ] = { .... }

And, when I restart the ColdFusion service is sometimes says that the datasource cannot be found. This "state" seems to hold for the entire life of the application, and all requests coming into the application are breaking. But then, if I restart the ColdFusion service again, it works fine (probably). I'm kind of stumped here - not sure how to even go about debugging this.

[Charlie Arehart]

Hey, Ben, and a hearty welcome to the forums. We've missed you. 🙂

First, can you do a dump of getapplicationmetadata()? I'd recommend doing it in a separate template rather than the Application.cfc itself. Do you see the datasource and ALL other this scope things you set?

If you do not, assess whether you may have any code that could affect that surrounding the setting of those vars, like a try/catch or even a cflock with perhaps throwontimeout="no".

Also, are those settings created in a method in the cfc or outside of any (what is termed the "pseudo-constructor", for the sake of other readers)?

It would be awesome if you may stick around and share your wealth of knowledge with folks here, but I realize you've got more irons in the fire than a blacksmith at a rodeo. 🙂 

[Mark Takata]

To add to Charlie's answer, I would also ask where are you setting this? Are you setting in onApplicationStart() (this is where I would set it). I'm convinced there's a race condition of some sort that seems to happen sometimes. I also include a function inside of onRequestStart that lets me restart the application with a long URL variable. Just in case there's some kind of weird issue that happens. But like Charlie said, in this kind of thing I go back to the basics and just start dumping out the app scope and meta data and see what CF thinks it is doing. Is it ALL the app variables not setting, just some? Ping me if you run into more issues!

[bennadel]

Hey all! 👋 so, the datasource that I'm defining is in the `Application.cfc` pseudo-constructor - basically right after I setup the name, mappings, serialization key-casing, etc. It was my understanding that this kind of stuff _had_ to be done in the pseudo-constructor; but, maybe that understanding it outdated? It sounds like @Mark Takata is saying that I can do it inside of `onApplicationStart()`. If that's true, this is news to me.

@Charlie Arehart good idea with the `getApplicationMetadata()` test. It's a bit tricky, this since this _only_ happens in production and I don't want everyone to suddenly get a dump of my app configuration values 🤣 but, I can probalby put it in some separate template and maybe even sanitize the password just in case.

I'll do that stuff and report back. Also, I'll try to hang-out in the forum more. 

[Charlie Arehart]

Great to see the replies from you both. First, yep, no need for others to see the dump, Ben. I'd said, "I'd recommend doing it in a separate template rather than the Application.cfc itself." 

And as for whether to do the set in onappstart or not, I'm ambivalent. I'd just asked whether the set was in a method or not. As for Mark's observation, I think he's saying the race condition was when it WAS in onappstart--though it could definitely happen outside of it. 

But I will press again my question about whether the set is inside anything that can affect its execution...not just an if test but perhaps something less obvious like a try/catch or cflock. 

And Mark, it would be awesome if you'll be in here often going forward. It's been mostly BKBK and myself holding down the fort, with also Dave Watts and Priyank less often (but greatly appreciated), then some others still less often. No slight intended to anyone who does help. 

It would just be great to see if more "reinforcements" might be arriving. It does take effort, but "many hands make light work". 🙂

As you'll see here, not EVERYONE has moved to slack or twitter for help. 🙂  And the knowledge shared here is far more easily found in web searching. 

[bennadel]

There's nothing wrapping the setting of the datasource. It's the darndest thing.

[BKBK]

Hey all! 👋 so, the datasource that I'm defining is in the `Application.cfc` pseudo-constructor - basically right after I setup the name, mappings, serialization key-casing, etc. It was my understanding that this kind of stuff _had_ to be done in the pseudo-constructor; but, maybe that understanding it outdated? It sounds like @Mark Takata is saying that I can do it inside of `onApplicationStart()`. If that's true, this is news to me.

By @bennadel

Your instincts are spot-on, @bennadel . There is no news there. The setting has to be done in the pseudo-constructor.

[Dave Watts]

I don't have a good debugging suggestion either. When I run into a problem like this, I generally just go in and yank on things until they work or break in a different way.

I've never heard of setting those "this." variables in onApplicationStart, onServerStart or onSessionStart. But it seems to work for Mark when he uses onApplicationStart, so try that! In the worst case, it still won't work intermittently and you're back where you started. You might also just see if there's a minor point release to the JVM beyond what you're using. Will upgrading to that work? Who knows? But this kind of random "pushing on buttons" might solve your problem even if you never find out why. Or, you might try using CFTRY/CFCATCH or its CFSCRIPT equivalent around the code you're using here. You shouldn't need it, but again, who knows?

Dave Watts, Eidolon LLC

[BKBK]

Hi @bennadel , I only have questions. But I hope they help.

1.  Is the datasource "bennadel" already configured in the ColdFusion Administrator?
        If so, then the code you've shown will be a repetition. The correct initialization code will probably be
        
        this.datasource="bennadel";
   
   
2.   Have you ensured that all the necessary datasource attributes - and appropriate values - are included in { ... }?
         For example, might the attributes you currently use make ColdFusion to create a connection each time?
         I would provide in { ... } , as a minimum, the attributes required to create a datasource in the ColdFusion Administrator. (Mind the default values)

[BKBK]

I'm running into a rather strange issue and I don't know how to debug it.

By @bennadel

Debug suggestion:

Run the following CFM page in the same directory as the Application.cfc:

<cfscript>
	try {
		writedump(new Application());
	} 
	catch (any e) {
		writedump(var=e, label="Oops!");
	}
</cfscript>

[bennadel]

Ok, so I just restarted the server and the datasource bombed-out again. Here's what I discovered:

• It is never making it past the `onApplicationStart()` method because I have some query-operations being triggered inside that function (preloading some in-memory structures from the DB). Meaning, once it gets in this state, every request is calling the `onApplicationStart()` method.
• If I dumped-out the `getApplicationMetadata()` inside the `onApplicationStart()`, it shows _all the data_ that I expected, including the default `datasource` and the `datasources` configuration.
• If I comment-out the data-pre-loading inside the `onApplicationStart()`, the application bootstraps. But then it errored due to missing pre-loaded data. BUT, if I then put this in the `onRequestStart()` code:

```

applicationStop();

location( url = cgi.script_name );

```

... in order to restart the application and refresh the page, then **everything starts up fine**.

This is just so strange.

[bennadel]

So, to be clear, no error is being thrown in the _pseudo constructor_. The `this.` data structures are all in tact and have the right data. It's getting to the `onApplicationStart()` method correctly. It just can't seem to find the datasource.

[bennadel]

Oh also, I tried copying the `this.datasources` and `this.datasource` configuration to inside the `onApplicationStart()` function - this made no difference. I'm pretty sure you _can't_ define that stuff there.

[bennadel]

I tried creating an isolated test-case for this locally, where I define `this.datasources` inside the `onApplicationStart()` function. Definitely does _not_ work. But, when I move it into the pseudo-constructor, it works. Unless I'm missing something subtle, I don't think you can define any of the per-app settings anywhere other than the pseudo-constructor.

[BKBK]

Oh also, I tried copying the `this.datasources` and `this.datasource` configuration to inside the `onApplicationStart()` function - this made no difference. I'm pretty sure you _can't_ define that stuff there.

By @bennadel

The this variables really have to be defined in the pseudo-constructor. Anyway, you mention this.datasources and this.datasource. Do you define both? And the question I asked earlier: have you made sure you haven't defined the application datasource twice? 

If you define both this.datasources and this.datasource, then this.datasource should be defined before this.datasources. 

[bennadel]

Interesting. I assumed that the `this.datasources` would need to be defined first since it contains the definition for the value that is then in the `this.datasource`:

this.datasources = { "bennadel": { ... } };
this.datasource = "bennadel";

But, I can try to flip them and see if it makes a difference.

[BKBK]

this.datasources = { "bennadel": { ... } };
this.datasource = "bennadel";

By @bennadel

That's the cause of the error right there. The default datasource is being defined twice!

[BKBK]

Correct is:

this.datasource = "bennadel"; // pre-defined in the Administrator
this.datasources = { "bennadel1": { ... }, "bennadel2": { ... } }; // each distict from "bennadel"

Alternatively,

this.datasources = { "bennadel": { ... } }; 

[bennadel]

I thought the point of  `this.datasource` was to define which datasource should be used _by default_ in the queries. Meaning, my `<cfquery>` tags do **not** have a `datasource` attribute. They are like:

<cfquery name="test">
    SELECT 1;
</cfquery>

I _believe_ it's the `this.datasource` that allows this to work. If I were to _just_ use the `this.datasources` **struct**, how would ColdFusion know which datasource to use in the `<cfquery>` tag?

[BKBK]

I thought the point of  `this.datasource` was to define which datasource should be used _by default_ in the queries. Meaning, my `<cfquery>` tags do **not** have a `datasource` attribute. They are like:

<cfquery name="test">
    SELECT 1;
</cfquery>

By @bennadel

Yes, that is so. The way I see it, 

this.datasource='bennadel';

takes for granted that "bennadel" has already been defined and that it is known (at runtime) to ColdFusion. That implies that "bennadel" must previously have been defined, usually in the ColdFusion Administrator.

[BKBK]

Ok, so I just restarted the server and the datasource bombed-out again. Here's what I discovered:

• It is never making it past the `onApplicationStart()` method 

By @bennadel

Just to rule this out: make sure onApplicationStart() has boolean returntype and that it returns True

[bennadel]

It's not even getting as far as the return statement. I am doing some data pre-loading inside the `onApplicationStart()` event, which is breaking because it can't find the datasource. So, the entirety of the `onApplicationStart()` method isn't even executing, let alone returning.

[bennadel]

Ok, so I just tried one more thing. I restarted the ColdFusion service, and the datasource bombed-out. Then, all I did was update a value in the `this.datasources` - I changed `interval: 420` to `interval: 421`, re-uploaded my `Application.cfc` and the site immediately came online.

To be clear, it has nothing to do with `interval`. I tried it a few more times, changing _different_ properties, and everytime I "touch" the `this.datasources` structure, the site immediately bootstraps and comes online. It's like an invalid version of the datasource is getting cached and then ColdFusion re-caches it if any of the keys change.

[Charlie Arehart]

This is all very interesting, and it raises some more questions that may help clarify things, and allow us to better understand whatever is going on:

• You mentioned earlier that "once it gets in this state, every request is calling the `onApplicationStart()` method". Are you doing some logging or other debugging that shows that? 
• Do you mean cf is calling it implicitly, or do you have code thaf calls it on some condition? 
• About your last point here, is the change you make (to interval) being made to the one failing datasource, or perhaps any?
• Are you inferring that Cf is automatically rerunning the onappstart, just because of that change? Or might you be calling it if some error handling code detects something not working? 
• If you dump out the specific this.datasource array element that is "failing", do any of its values differ between when it IS failing and when it is NOT?

In my experience with challenges like this, there will either be some head-slapping realization of something your code was causing, or some eyebrow-raising discovery of some cf behavior none of us is anticipating. 🙂 Looking forward to the "exciting conclusion" to the story. 

[bennadel]

I had started to `dump` some data in the `onApplicationStart()`, and I noticed that all calls to the site were hitting this line of code, which is why I concluded that the app had not been bootstrapped and that ACF as implicitly calling this, attempting to bootstrap the app over and over (it never succesfully completes).

I'll keep experimenting over the weekend. It's hard to test because the problem doesn't happen locally; and, it doesn't happen every time I spin-up the service 😄 Such a crazy problem!