CF2016 - <cfinclude> dynamic template name - java.IO.IOException *sometimes*

[bprop]

Hello,

I have a problem with CF2016 after upgrading two of our three servers; all are running behind a load balancer.  The CF2016 servers have had this problem multiple times since upgrading to CF2016, while the CF11 server has never had this problem.  So I'm pretty confident it's CF2016.

Based on a URL parameter appl, we check to see if the index.cfm file exists in that folder:

<cfset filename="#appl#\index.cfm">

<cfif fileExists("d:\docs\#filename#")>

     <cfinclude template="#filename#">
<cfelse>

     <cfthrow type="applDoesNotExist">
</cfif>

This code gets run hundreds of thousands of times per day, most with no trouble at all.

Occasionally (maybe 3x a day), the bolded <cfinclude> line above errors with:

java.io.IOException

The process cannot access the file because it is being used by another process

Key points:

• This ONLY happens when cfincluding dynamically-created template names.  Any instances where the template name is hard coded, e.g. <cfinclude template="testapp\index.cfm"> are not affected
• It can happen with any variation of #appl# - i.e. it's not just one #appl# that gets this error
• The .cfm files which it is trying to access EXIST
• The .cfm files which it is trying to access are UNTOUCHED and have been untouched, in most cases, for years.
• This is only happening in CF2016, not in CF11 with 100% matching settings (including things like template caching)
• Again, 99.99% of the time, the exact same URL gets hit and it works fine. In other words, it is hard to replicate this issue

I don't understand the error.  We have ensured that no processes are touching .cfm files (and besides, the fact that this is only happening to files with *dynamically generated* (variable) names seems to contraindicate such an explanation)

Any ideas what behavior behind the scenes in CF2016 is causing this?  Thanks for insight.

[James Moberg]

I was encountering the same issue after upgrading some legacy apps from CF8/9 to CF10/2016.  We have a special template sub-directory where we maintain themes.  Occasionally CF10/2016 would insist that it didn't exist and throw an error.

The only thing we've identified that works is to add an application.cfc file to these projects and specifically define the "rootDir" & generate mappings for the sub-directories that were getting lost.

    <cfset this.rootDir = getDirectoryFromPath(getCurrentTemplatePath())>

    <cfset this.mappings = {}>

    <cfset this.mappings[ "/templates" ] = "#this.rootDir#templates/">

[EddieLotter]

Surely this:

<cfif fileExists("d:\docs\#filename#")>
  <cfinclude template="#filename#">

should be:

<cfif fileExists("d:\docs\#filename#")>
  <cfinclude template="d:\docs\#filename#">

Otherwise you are checking that a file exists in one location but then trying to include it in the folder of the caller.

Cheers

Eddie

[bprop]

I may have taken a couple liberties (or missed a character) when consolidating the code for easy view, but the second line is relative from the index.cfm file where the <cfinclude> is. We generally don't cfinclude absolute file paths with drive letters, though that's what we have to use for fileExists.

So for example:

D:\

     \docs

          index.cfm <- this is where the <cfinclude> is

          \appl1

index.cfm  <- this is the file we're <cfinclude>ing

othertemplate.cfm

          \appl2

index.cfm

othertemplate.cfm

          \appl3

index.cfm

othertemplate.cfm

So if we have <cfinclude template="appl1\index.cfm">, it is relative to the index.cfm file under D:\docs

But for purposes of fileExists(), we use the full template path, D:\docs\appl1\index.cfm

[bprop]

Jamo, thank you for the reply.  Can you further show what you did once you creating the mapping in application.cfc? All of our <cfincludes> are from a path relative to webroot.

So let's say we had a directory that looks like this

/  <-- (CF webroot)

     index.cfm <-- this is where the <cfinclude> is

     appl1/

     appl2/

     appl3/

and in index.cfm, we had:

<cfinclude template="#appl#/index.cfm">

...that is equivalent to saying:

<cfinclude template="appl1/index.cfm">

right?

So, if we put a mapping in application.cfc as you did, do we then use that mapping variable in the <cfinclude> statement?  Or does it just help ColdFusion find templates that it occasionally misses?  Trying to understand the mechanics. Thanks!

[James Moberg]

I believe adding mapping serves as a fallback when ColdFusion loses its way and can't remember where home is.  We have a file in the root directory that CFIncludes templates in a sub-directory directly off the root using a relative path and ColdFusion will sometimes throw an error stating it "could not find it".

In an older app that doesn't currently have an application.cfc file, I received an inconsistently occurring ColdFusion error that looks like this:

Could not find the included template /portal/templates/myFooter.cfm. Note: If you wish to use an absolute template path (for example, template="/mypath/index.cfm") with CFINCLUDE, you must create a mapping for the path using the ColdFusion Administrator. Or, you can use per-application settings to specify mappings specific to this application by specifying a mappings struct to THIS.mappings in Application.cfc.

Using relative paths (for example, template="index.cfm" or template="../index.cfm") does not require the creation of any special mappings. It is therefore recommended that you use relative paths with CFINCLUDE whenever possible.

The frustrating thing is these templates are successfully executed 99.999% of the time without throwing any error.

NOTE: When using a user-defined dynamic path or scriptname, we use FileExists() to determine if the template exists and further restrict traversing unauthorized directories & scripts. (We don't want script kiddies passing bogus parameters and executing unauthorized paths/scripts.)

The solution to get it to work without "occasionally crashing without reason" in CF10+ is to add redundant mapping.  Depending on the version of ColdFusion/Lucee you are using, not all application.cfc features are available (or work as advertised), so consult the documentation at CFDocs.org.

CFDocs.org: Application.cfc Code Examples and CFML Documentation

Here's a very basic application.cfc configuration that defines mapping.

<cfcomponent displayname="MyWebApplicationName">

<!--- This is older tag-based syntax. I'm aware of that and no need to point it out. Some script-based shortcuts don't work well (or consistently) with older versions of ColdFusion --->

<cfset ScriptProtect = "All">

<cfif CGI.Remote_Addr IS "127.0.0.1">

<cfset ScriptProtect = "none">

</cfif>

<cfset THIS.name = "MyWebApplicationName">

<cfset THIS.customtagpaths = "C:\ClientSpecificCustomTagsDir">

<cfset THIS.scriptProtect = ScriptProtect>

<cfset this.rootDir = getDirectoryFromPath(getCurrentTemplatePath())>

<cfset this.mappings = {}>

<cfset this.mappings[ "/appl1" ] = "#this.rootDir#appl1/">

<cfset this.mappings[ "/appl2" ] = "#this.rootDir#appl2/">

<cfset this.mappings[ "/appl3" ] = "#this.rootDir#appl3/">

<!--- etc --->

<cfset this.timeout = 65000>

<cffunction name="onRequest">

<cfargument name="targetPage" type="String" required="true">

<cfinclude template="./appInit.cfm"><!--- or whatever your regular global/security script is --->

<cfinclude template="#Arguments.targetPage#">

</cffunction>

</cfcomponent>

[bprop]

Thank you for the detailed reply.

As I go over it now, I think our problems may be similar (and similarly frustrating), yet unrelated.  In your case, CF is getting lost and is unable to find a template file. In our case, CF is finding the file; but it errors because it claims the file is in use by another process. 

In fact, I do recall having had the same issue as you previously; in fact I believe I asked about it on an Adobe Forum.  I imagine your solution would have fixed it at the time.  But over time, perhaps with patching or upgrades, we stopped encountering it.

We may add mappings as you describe simply because it can't hurt. But I'm skeptical.  Thanks again for the reply.

[BKBK]

Occasionally (maybe 3x a day), the bolded <cfinclude> line above errors with:

java.io.IOException

The process cannot access the file because it is being used by another process

Surpising behaviour, perhaps due to the dynamic variables. If so, a named lock might help

<cflock name="cfincludeFileLock" type="exclusive">

    <cfset filename="#appl#\index.cfm">

    <cfif fileExists("d:\docs\#filename#")>

         <cfinclude template="#filename#">
    <cfelse>

         <cfthrow type="applDoesNotExist">
    </cfif>

</cflock>