CFLOCK an image during parse

LEGEND ,
Oct 22, 2018 Oct 22, 2018

Copy link to clipboard

Copied

Hello, all,

About two years ago, we started having serious latency issues with our restricted portal.  Many of us spent weeks trying to track down what the cause was. 

The rest of us were scrambling trying to optimize the portal as much as we could, in the hopes that we could speed up the site load times.  Part of this band-aid approach involved a custom-made News slideshow on the landing page.  Instead of grabbing four UUIDs, Headlines, and images on every page load, we set up the application.cfc so that every two minutes the query would run and store the UUIDs, Headlines, and images into application variables, and pull from those.

Naturally, I placed an exclusive CFLOCK around the process in the application.cfc.  But we are using a separate include for retrieving the images on page load, and part of that is using CFHEADER/CFCONTENT to serve up the images, passing an ID in the URL scope.  This reduced the page load time by only a few seconds, but it helped.

Three weeks in, we finally came to the conclusion that a new STIG that had been applied was the culprit.  We undid the portion of the STIG that was causing the issue, and voila!, the pages were loading much faster.

At the time we had other project to work on, so I left the above described process in place.  And I don't currently have time to put it back to the way it originally was.  However, we are now experiencing (and have been for quite some time) issues involving a missing array element related to the images.  What I suspect is happening is that when the page loads, if it happens when the images are being refreshed, it's not finding the image in question because it's being overwritten by the application.cfc.

Is there a way to place a CFLOCK around the images during read so that the application.cfc can't overwrite images if they are being accessed for page load?  Is it as simple as placing an exclusive CFLOCK around the CFHEADER/CFCONTENT tags?

V/r,

^ _ ^

Views

327

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Correct Answer

Advocate , Oct 23, 2018 Oct 23, 2018
Your lock is being applied too late, and worse, you are locking and unlocking in a loop.Your first block of code should protect all application variables that are being changed:<cflock scope="application" timeout="40" type="exclusive">    <cfset application.init_datetime = now() />    <cfset application.NEWSlist = new components.root().NEWSlist() />            <cfset application.newsArray = ArrayNew(2) />          <cfset variables.count = 1>          <cfloop query="application.NEWSlist">        ...

Likes

Translate

Translate
Adobe Community Professional ,
Oct 22, 2018 Oct 22, 2018

Copy link to clipboard

Copied

You could certainly use CFLOCK to prevent two blocks of code from running simultaneously - in fact, that's all CFLOCK really does. You could use something fairly broad with the SCOPE attribute, but you'd probably be better served with a more narrow NAME attribute.

Is your potential concurrency problem happening during onApplicationStart, or onRequestStart? If it's happening in onApplicationStart, I don't think you should be having a locking conflict in the first place. If it's in onRequestStart, there could certainly be a concurrency issue there with other running requests. I guess from reading your message that it's probably happening in onRequestStart but I just wanted to make sure.

You might be able to solve this problem more easily by just using the caching attributes of the CFQUERY tag, actually: CACHEDWITHIN or CACHEDAFTER.

Dave Watts, Fig Leaf Software

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 22, 2018 Oct 22, 2018

Copy link to clipboard

Copied

Hi, Dave,

I've not studied CFLOCK in any great detail.  What is the difference between a SCOPE lock and a NAME lock, and why is NAME more narrow?

The concurrency problem is within the include, so I'm assuming that the issue is happening in onRequest, not onRequestStart (or my assessment could be completely incorrect.  )

To wit:

Diagnostics: The element at position 3, of dimension 2, of an array object used as part of an expression, cannot be found.

The error occurred on line 7.

Line 7 is the CFHEADER/CFCONTENT.

You might be able to solve this problem more easily by just using the caching attributes of the CFQUERY tag, actually: CACHEDWITHIN or CACHEDAFTER.

We were all panicked and trying to speed things up with a quickness, so caching the query didn't even occur to me.    What I _should_ do is put it back to pulling from the database on every page load (with a cached query), but don't have time for that, right now.  But are you saying that just placing a named CFLOCK around the CFHEADER/CFCONTENT should prevent these error emails from being sent?  Granted, we only see about ten of them in a day, but they are filling up the org inbox over time.

V/r,

^ _ ^

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Oct 22, 2018 Oct 22, 2018

Copy link to clipboard

Copied

I would start with the query caching attributes, actually. It should be a lot simpler than locking. CF takes care of everything itself when you use query caching attributes.

The difference between SCOPE and NAME is that when you use the SCOPE attribute, any other conflicting lock also using the same SCOPE attribute will be blocked. In a large application using CFLOCK for a lot of things, you could have unrelated things that are blocked just because they have SCOPE="application". With the NAME attribute you can make up your own names, so you're less likely to run into lock conflicts. You could create a name just for this one problem.

Dave Watts, Fig Leaf Software

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 22, 2018 Oct 22, 2018

Copy link to clipboard

Copied

Thanks, Dave.  I'll give that a shot and report back.

V/r,

^ _ ^

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Oct 22, 2018 Oct 22, 2018

Copy link to clipboard

Copied

Sounds like a challenging issue. If you were to share the code, or even pseudo-code, then we could put our heads together and solve it the way ants and bees do.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

Hi, BKBK,

Adding the CFLOCK to the CFHEADER/CFCONTENT didn't work.  I have one (thankfully only one) error email in the org inbox related to this issue.

I'll post the relevant parts.  Hopefully someone can see what I cannot.

A part of onApplicationStart():

     <cfset application.init_datetime = now() />

     <cfset application.NEWSlist = new components.root().NEWSlist() />

        <cfset application.newsArray = ArrayNew(2) />
        <cfset variables.count = 1>
        <cfloop query="application.NEWSlist">
            <cflock scope="application" timeout="40" type="exclusive">
                <cffile action="write" file="#this.webrootmapping#images\z#variables.count#.jpg" output="#application.NEWSlist.media_file#" nameconflict="overwrite" />
                <cfset application.newsArray[variables.count][1] = application.NEWSlist.uuid>
                <cfset application.newsArray[variables.count][2] = application.NEWSlist.TITLE>
                <cfset application.newsArray[variables.count][3] = application.NEWSlist.media_file>
                <cfset variables.count++>
            </cflock>
        </cfloop>

Part of onRequestStart():

            <cfif DateDiff('n',application.init_datetime,now()) gt 1><!--- Once a minute, refresh the images --->
                <cfset onApplicationStart()>
            </cfif>

Part of include that serves images to main page:

    <cflock name="panImage" timeout="10" throwontimeout="yes">

          <cfheader name="content-disposition" value="inline; filename=z#val(url.id)#.jpg" />

          <cfcontent type="image/*" variable="#application.newsarray[val(url.id)][3]#" reset="yes" />

    </cflock>

V/r,

^ _ ^

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

When you use locks, they have to share a name or scope to have any effect. If you have one lock with SCOPE="application", it won't prevent code within another lock with NAME="panImage" from running. Basically, locks are just signals saying "if both of these locks have the same name, and one is exclusive, don't run the code in one until the other has finished".

That said, again, I think you'd be better off just adding query caching to your query. That would be the simplest approach. I like simple! I don't like rerunning onApplicationStart every minute, to be honest.

Dave Watts, Fig Leaf Software

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Advocate ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

Your lock is being applied too late, and worse, you are locking and unlocking in a loop.

Your first block of code should protect all application variables that are being changed:

<cflock scope="application" timeout="40" type="exclusive"> 

  <cfset application.init_datetime = now() /> 

  <cfset application.NEWSlist = new components.root().NEWSlist() /> 

 

        <cfset application.newsArray = ArrayNew(2) /> 

        <cfset variables.count = 1> 

        <cfloop query="application.NEWSlist"> 

                <cffile action="write" file="#this.webrootmapping#images\z#variables.count#.jpg" output="#application.NEWSlist.media_file#" nameconflict="overwrite" /> 

                <cfset application.newsArray[variables.count][1] = application.NEWSlist.uuid> 

                <cfset application.newsArray[variables.count][2] = application.NEWSlist.TITLE> 

                <cfset application.newsArray[variables.count][3] = application.NEWSlist.media_file> 

                <cfset variables.count++> 

        </cfloop>

</cflock>

Your second lock should use the same scope (application) but can use the ReadOnly type.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

LATEST

Eddie, that was it.  I placed the lock outside the loop, and placed an application readonly lock around the CFCONTENT / CFHEADER, and we have not seen a single error message related to this, since.  Thank you.

V/r,

^ _ ^

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

My suggestion is similar to EddieLotter​'s.

1) From what we can see, there is no need for the image-file code to be in onApplicationStart. In fact, onApplicationStart should be free to undertake other application business. We should therefore decouple the image-file use-case from the main application. For example, by assigning the responsibility to a separate CFC. That would in fact make for more flexible, object-oriented code.

Img.cfc

<cfcomponent >

<cffunction name="writeImageFile" access="public" returntype="void" output="false">

    <cfargument name="path" required="true" type="string">

    <cflock scope="application" timeout="40" type="exclusive">  

        <cfset application.init_datetime = now() />  

        <cfset application.NEWSlist = new components.root().NEWSlist() />  

        <cfset application.newsArray = arrayNew(2) />    

        <cfloop query="application.NEWSlist">  

            <cffile action="write" file="#path#images\z#currentRow#.jpg" output="#application.NEWSlist.media_file#" nameconflict="overwrite" />  

            <cfset application.newsArray[currentRow][1] = application.NEWSlist.uuid>  

            <cfset application.newsArray[currentRow][2] = application.NEWSlist.TITLE>  

            <cfset application.newsArray[currentRow][3] = application.NEWSlist.media_file>    

        </cfloop>

    </cflock>

</cffunction>

</cfcomponent>

2) In onApplicationStart():

<cfset new Img().writeImageFile(this.webrootmapping) >

3) In onRequestStart():

<cfif dateDiff('n',application.init_datetime,now()) gt 1><!--- Once a minute, refresh the images --->

    <cfset new Img().writeImageFile(this.webrootmapping)>

</cfif>

<cflock scope="application" timeout="10" throwontimeout="yes" type="readOnly">

      <cfheader name="Content-Disposition" value="inline; filename=z#val(url.id)#.jpg" />

      <cfcontent type="image/*" variable="#application.newsarray[val(url.id)][3]#" reset="yes" />

</cflock>

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines