sometimes application variables don't reset

[Guest]

In our application.cfm, many queries are run and set to arrays that are application scope. They are for code tables. To keep all this from happenning each time a template runs, there is a server scope variable that is set to true from another program and in application.cfm, there is something like <cfif the server variable is true> run queries,assign arrays,set server variable to false  </cfif>.

When a new code is added to a code table, we set the server variable to true and the next template that runs resets all the application variables and the new code is in one of them.

Sometimes this doesn't work. Only some of the arrays get reset and the others are empty. The arrays are there but have no elements.

Repeating the process 2 to 4 times seems to make it work.

What can cause it not to work the first time ?

[Adam Cameron.]

It pretty much impossible to say, without seeing the relevant code.

--

Adam

[Guest]

Ok. I hope to make a short version of our application.cfm and the resetting program and put them here tomorrow.

[Guest]

The problem happens even in a test area where I'm the only one using it.

Why does the reset program sometimes have to be run more than once ?

This is the reset program

server.x=0

This is what the application.cfm is like

<cfapplication name="thename">

<cfset ds="dsname">

<cfparam name=server.x type="boolean" default="0">

<cfif server.x eq 0>

   <cfset server.x=1>

   <cfset application.codes=arraynew(2)>

   <cfquery name="qcodes" datasource="#ds#">

      select codes, desc from tblcodes order by codes

   </cfquery>

  <cfloop query="qcodes">

      <cfset application.codes[currentrow][1]=qcodes.code[currentrow]>

      <cfset application.codes[currentrow][2]=qcodes.desc[currentrow]>

      <cfset application.codes[currentrow][3]=qcodes.place[currentrow]>

      <cfset application.codes[currentrow][4]=qcodes.date_eff[currentrow]>

      <cfset application.codes[currentrow][5]=qcodes.date_del[currentrow]>

  </cfloop>

   does this for well over 50 more tables

</cfif>

[mack_]

Is Application.cfm the only place where the application arrays are initialized ?

--

Mack

[Guest]

Mack,

The only place the application variables are assigned is in application.cfm.

[Adam Cameron.]

OK.  Do a pencil and paper exercise.

Start stepping through your code, line by line, starting from a position of server.x=0.  At each step: jot down the current contents of application.codes.  When you get to the third line, start stepping through your code a second time (in parallel to the first time, so you're emulating two different requests running @ the same time).  Continue line by line, and for each line (for each of the two requests), follow the logic through, jotting down the current contents of application.codes.

You should see where you're going wrong fairly quickly.

--

Adam

[mack_]

Hmm, I'm not convinced that's the problems. If the code within cfif is

executed in 2 parallel threads what I expect is that some partial

arrays are discarded but in the end all the arrays have the correct

values (because if a thread manages to set a few rows the second one

executes, re-inits the array and re-sets all the rows, including the

ones that were lost by the re-init).

--

Mack

[Adam Cameron.]

Why would the second request be executing the code in the CFIF?

--

Adam

[Guest]

I don't know. Isn't server.x turned to 1 right after it's entered?

[Adam Cameron.]

Yes.  And what are the ramifications of that?  Whilst the first request is still within the CFIF block, setting these variables, the second request is going to move on past that code, and possibly arrive at some code that expects those variables to already exist before the first request has finished setting them.

Do you see the problem now?

This is very easily demonstrated with this code:

<cfapplication name="raceCondition">

<cfparam name="application.init" default="true">
<cfif application.init>
    <cfset application.init = false>

     <cfset structDelete(application, "msg")>
    <cfset sleep(5000)>
    <cfset application.msg = "Hello World @ #timeFormat(now(), "HH:MM:SS.LLL")#">
</cfif>

<cfoutput>#application.msg#</cfoutput>

Hit that with two different browsers within 5sec of each other, and the second one will error.  This is analogous to your code.

--

Adam

[Guest]

I think so. The declarations happenned but the queries and assignments didn't finish which is why we don't see "error resolving parameter" but instead see some empty arrays.

1. I am reading about cflock, do you think that might be a way to solve this ?

2. What could cause this to happen in the test area where I'm the only user ?

[Adam Cameron.]

My approach would be this:

* take that re-init code out of Application.cfm.  That template is run for every request, for everyone making a request (in that directory subtree, anyhow).  Whereas this code is only ever relevant in very specific circumstances.  You've got the CFIF around it, but from a code structure and maintenance POV, code in a given template should be run most of the time the template is called.  Otherwise it's perhaps in the wrong place.  In this situation you're making a rod for your own back by having that code in there

* I would put the re-init code into a separate template, and then whatever process currently sets that server variable should simply call that template instead.

* why are you creating a nice recordset - a very handy data structure - and converting it into a - rather grim - two dimensional array?  If the data suits being in a DB in a "tabular" format, it does not naturally suit being represented in a 2-D array, which is intended for an entirely different structure and purpose of data.  Why not just leave it in the recordset?  At the very least, an array of structs seems more sensible here, rather than losing the key/value association that the data clearly has, in favour of an index number.

* it make no sense to me that you are setting a SERVER scope variable as a flag for when you want to reset APPLICATION scope variables.

* my re-init template would be like this:

query the data

transform the data if necessary, into a LOCAL variable

set the application-scoped variable to the value of the local variable.

That way you will avoid almost all race-condition scenarios, because the application variable will be (re)set in one fell swoop.  The only consideration you would need to protect against is if you have code in another template, like this:

do something with one of the application-scoped variables

line of code

line of code

[...]

line of code

line of code

do something with that application-scoped variable again, which assumes it's the same value as before

(or similarly with another application-scoped variable in which the logic relates to the earlier value of the first one).

This is because if the re-init code could run after the first line and before the second line, meaning your data integrity is potentially compromised.

If you're likely to have situations like that, then put an exclusive lock around where you SET the application-scoped variable, and a read-only lock around situations like above.  You do not need a read-only lock around other situations where you are using those application-scoped variables.  A lot of people get a bit trigger-happy with CFLOCK, because of instability situations CF used to have with shared-scope variables, in version of CF up to and including CF5.

As for your second question, why this also happens on dev, if you have only single requests firing at once... not sure.  Do you have a CFERROR tag or a global error handler that could perhaps be hiding errors from you?  Are you making incorrwct assumptions about what's coming back from the DB?  EG: your code is assuming application.codes[6] is ALWAYS going to exist, but in some situations only five rows are coming back from the DB?  Are you using frames or iframes, or AJAX requests which might be causing simultaneous requests being fired off despite you yourelf only firing off the one request by hand?

To  be honest, I'd not worry about this.  You code just won't work in production, so you need to fix it.  Fix it and see if you still get the problem in dev.

--

Adam

[Guest]

Thank you very much.

Moving all the queries and assignments to another template that we run ourselves seems like a good idea.

There are many users and frequent use of long code lists, maybe arrays are best here.

An application rather than a server scope would work for reset too but back then, I wanted to try the new scope. If all this is done in another template, it won't be needed.

We are using frames but I do not see how it makes multiple requests at once.

Have found no errors are in the logs about this, think the declarations happen fast enough so the arrays are defined.

[Adam Cameron.]

There are many users and frequent use of long code lists, maybe arrays are best here.

I doubt it.  A multi-dimensional array is almost never the best data structure for business data.

If it makes sense to have the columns in the DB have descriptive names, ie: code,desc,place,date_eff,date_del, then it makes the same amount of sense to maintain those column names in your CF-based data struct too.  Why are you calling the code value "1" and the desc value "2"?  It's not because code is "lower" than desc is it?  The question itself makes no sense, so clearly not.  There is no numeric relationship between those elements.  One should only use an array when there's a sense of numeric sequence in the elements. However clearly here there isn't a numeric sequence.  There is a numeric sequence from row to row, but not column to column.

What would make sense for you to do is either:

* leave it in the query.  I mean... why wouldn't you?  The data is already structured in a sensible, easily accessible fashion.

* construct it as an array of structs, thus:

<cfset aTemp = arrayNew(1)>
<cfquery name="qcodes" datasource="#ds#">
    stuff
</cfquery>

<cfloop query="qcodes">
    <cfset aTemp[currentRow] = {
        code    =code,
        desc    =desc,
        place    =place,
        date_eff=date_eff,
        date_del=date_del
    }>
</cfloop>
<cfset application.codes=aTemp>

We are using frames but I do not see how it makes multiple requests at once.

Well the frameset request and the contents of each frame are separate requests, right?  So if you have a frameset with two frames in it, the initial load of the pages is three requests right off the bat.

--

Adam

[mack_]

OP's statement that "The arrays are there but have no elements." made

me suspect some kind of re-init by parallel threads. Your scenario

sounds much more likely if the code is as presented but I would expect

"undefined variable" errors to occurs for some of the variables that

are not initialized yet.

--

Mack

[Adam Cameron.]

Any request that comes through after the array is recreated, but before the end of the CFLOOP will see the array, but not (all ~) the contents of it.

--

Adam