CGI Scope is empty with ColdFusion 2021 on windows 2019

[Blair22505870i4ma]

I have found that on two of my coldfusion 2021 servers, at random intervals, the CGI scope starts returning an empty struct.   When this happens, the only solution seems to be a restart of the ColdFusion service.

there dont seem to be any relevant entries in any of the ColdFusion logs or the Windows Event Viewer.

Has anybody ever experienced this?

[BKBK]

structKeyList doesn't work... I can't remember if it throws an error or produces an empty string/array but no way to see the keys (my first thought too was to loop over structKeyList).  The dumps are just diagnostic ... the missing cgi code causes errors bc the code expects them to be populated.

Gabe

I would then suggest the following test code

<cftry>
	<cfoutput>#structKeyList(cgi)#</cfoutput>
<cfcatch type="any">
	<cfdump var="#cfcatch#" label="CGI structkeylist error" format="html" output="#getDirectoryFromPath(getCurrentTemplatePath())#CGI_structkeylist_error.html">
</cfcatch>
</cftry>

[BKBK]

I should add that, in the last CGI dump test that I proposed, you can bypass the browser by doing a cfhttp GET to the test cfm page.

[EddieLotter]

I don't know how they have things set up at TryCF.com but dumping the CGI scope shows content on all CF versions except 2021. Coincidence?

[BKBK]

Nice find, @EddieLotter 

What happens when you wrap the cfdump in a try/ catch and dump errors? I cannot test that myself on this teeny weeny phone screen.

[EddieLotter]

@BKBK it doesn't throw an exception, it just shows the "struct [empty]" message.

[BKBK]

I don't know how they have things set up at TryCF.com but dumping the CGI scope shows content on all CF versions except 2021. Coincidence?

By @EddieLotter

Hi @EddieLotter ,

Things are even worse at cffiddle.org. There the code

<cftry>
	<cfdump var="#cgi#">
<cfcatch type="any">
	<cfdump var="#cfcatch#">
</cfcatch>
</cftry>

fails on CF2016 Update 17, CF2018 Update 13 and CF2021 Update 3. In each case the result is a screen containing the text "CGI variables cannot be accessed" and a spinning hourglass. No errors, nothing. 

[BKBK]

My conclusion is, what we've been seeing with CGI is caused by 

• a bug, or
• a setting we're yet unaware of.

I found some more odd behaviour, including a happy surprise.

On TryCF.com's CF2021 engine:

    dump of CGI = empty struct;

    structIsEmpty(cgi) = Yes;

    structCount(cgi) = 46;

    structKeyList(cgi) = [empty string].

Yes, buggy. Nevertheless, I was able to find a workaround of sorts.

The following code does produce CGI variables on TryCF.com's CF2021 engine:

<cfset cgiKeyList="HTTPS_SECRETKEYSIZE,REMOTE_HOST,SERVER_PROTOCOL,
					CERT_SERVER_SUBJECT,REMOTE_ADDR,CERT_SERVER_ISSUER,
					SERVER_SOFTWARE,PATH_TRANSLATED,HTTPS_SERVER_SUBJECT,
					CERT_KEYSIZE,CF_TEMPLATE_PATH,HTTP_URL,CERT_SERIALNUMBER,
					CERT_SUBJECT,HTTP_REFERER,AUTH_PASSWORD,HTTPS,
					CONTENT_TYPE,REQUEST_METHOD,SCRIPT_NAME,CERT_ISSUER,
					SERVER_NAME,PATH_INFO,AUTH_TYPE,GATEWAY_INTERFACE,
					SERVER_PORT,HTTPS_SERVER_ISSUER,HTTP_ACCEPT_LANGUAGE,
					CONTEXT_PATH,SERVER_PORT_SECURE,CERT_COOKIE,WEB_SERVER_API,
					HTTPS_KEYSIZE,AUTH_USER,REMOTE_USER,HTTP_HOST,
					CONTENT_LENGTH,QUERY_STRING,HTTP_ACCEPT,CERT_SECRETKEYSIZE,
					HTTP_USER_AGENT,HTTP_ACCEPT_ENCODING,HTTP_COOKIE,
					CERT_FLAGS,LOCAL_ADDR,HTTP_CONNECTION">
<cftry>
	<cfoutput>   	
		<cfloop list="#cgiKeyList#" index="key">
			Key: #StructFindKey(cgi,key)[1].path# | Value:#StructFindKey(cgi,key)[1].value# <br>
		</cfloop>		
    </cfoutput>
<cfcatch type="any">
	<cfdump var="#cfcatch#">
</cfcatch>
</cftry>

[gabrieldavis321]

HI BKBK,

I added the code above to our diagnostic page and will let you know if the hard coded lists works.

Thanks,

Gabe

[Charlie Arehart]

I'll add that I'd dug into this also when I'd seen over the weekend the mention that the dump of cgi is empty on trycf. In my testing, it's ALWAYS empty, unlike Gabe's situation where it's on and off. (And FWIW, when I run against my own cf2021, I never see it empty.)

Like bkbk says, this variation could be due to a bug or a setting. I'm not aware of any admin setting, and I compared dumps of getapplicationmetadata() and the server scope and found no differences that seemed related.

Here's another possibility: maybe it's a configuration difference, like how cf was installed. I use the full installer typically, while some deploy using Commandbox (like trycf has). Others might use the new cf2021 zip install option, and still others use war files or docker images.

Gabe, since your situation is so off and on, I'm less inclined to think yours is about how you implemented cf, but for the sake of completeness, how did you? 🙂 

[gabrieldavis321]

Hi BKBK,

The hardcoded list loop seems to have worked at least when accessing under cfusion/wwwroot... I have to wait until it happens again and test under the IIS root, but it definitely has potential... thanks!

below is the results of your code 

-Gabe

Key: .HTTPS_SECRETKEYSIZE | Value:

Key: .REMOTE_HOST | Value:0:0:0:0:0:0:0:1

Key: .SERVER_PROTOCOL | Value:HTTP/1.1

Key: .CERT_SERVER_SUBJECT | Value:

Key: .REMOTE_ADDR | Value:0:0:0:0:0:0:0:1

Key: .CERT_SERVER_ISSUER | Value:

Key: .SERVER_SOFTWARE | Value:

Key: .PATH_TRANSLATED | Value:D:\ColdFusion2021\cfusion\wwwroot\cgiTest\index.cfm

Key: .HTTPS_SERVER_SUBJECT | Value:

Key: .CERT_KEYSIZE | Value:

Key: .CF_TEMPLATE_PATH | Value:D:\ColdFusion2021\cfusion\wwwroot\cgiTest\index.cfm

Key: .HTTP_URL | Value:

Key: .CERT_SERIALNUMBER | Value:

Key: .CERT_SUBJECT | Value:

Key: .HTTP_REFERER | Value:

Key: .AUTH_PASSWORD | Value:

Key: .HTTPS | Value:on

Key: .CONTENT_TYPE | Value:

Key: .REQUEST_METHOD | Value:GET

Key: .SCRIPT_NAME | Value:/cgiTest/index.cfm

Key: .CERT_ISSUER | Value:

Key: .SERVER_NAME | Value:localhost

Key: .PATH_INFO | Value:

Key: .AUTH_TYPE | Value:

Key: .GATEWAY_INTERFACE | Value:

Key: .SERVER_PORT | Value:26268

Key: .HTTPS_SERVER_ISSUER | Value:

Key: .HTTP_ACCEPT_LANGUAGE | Value:en-US,en;q=0.9

Key: .CONTEXT_PATH | Value:

Key: .SERVER_PORT_SECURE | Value:1

Key: .CERT_COOKIE | Value:

Key: .WEB_SERVER_API | Value:

Key: .HTTPS_KEYSIZE | Value:

Key: .AUTH_USER | Value:

Key: .REMOTE_USER | Value:

Key: .HTTP_HOST | Value:localhost:26268

Key: .CONTENT_LENGTH | Value:

Key: .QUERY_STRING | Value:

Key: .HTTP_ACCEPT | Value:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9

Key: .CERT_SECRETKEYSIZE | Value:

Key: .HTTP_USER_AGENT | Value:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36

Key: .HTTP_ACCEPT_ENCODING | Value:gzip, deflate, br

Key: .HTTP_COOKIE | Value:CFID20211499976351=607381; CFTOKEN20211499976351=5b81fdc0a4e1bd98-E437C1CE-B6A7-6910-CB2E18C6708B46AB; JSESSIONID=544A376E65DD221895CAAF99666E5755.cfusion

Key: .CERT_FLAGS | Value:

Key: .LOCAL_ADDR | Value:0:0:0:0:0:0:0:1

Key: .HTTP_CONNECTION | Value:keep-alive

[gabrieldavis321]

Hi Charlie,

We used the regular installer (ColdFusion_2021_GUI_WWEJ_win64.exe)  to install CF and at that time it shipped with update 2.  Then we used the lockdown tool (ColdFusion_2021_Lockdown_WWEJ_win64.exe)  to connect to IIS.  Then we had to unistall/re-install ASP so we used the connector tool to hook CF back to IIS.

-Gabe

[gabrieldavis321]

Hi BKBK,

Your code worked at least when under cfusion/wwwroot.  I have to wait for it to fail again to test if it works agaisnt the same code under IIS root, but it is looking promising!

Gabe

[BKBK]

Hi @gabrieldavis321 ,

Thanks for the update.

[gabrieldavis321 · Accepted solution]

Hi BKBK and Charlie,

First I would like to thank you both for your help.  I believe we found the root cause and it WAS code.  There was  a function to scrub passwords from structs so that we could safely create error handling emails with dumps of form, request and CGI and not have the password displayed.  Below is from the developer: 

I think this was a subtle difference between CF2016 and CF2021 in the way it handles copying structs inside a function.  Let me explain a little further…

Looking at the sanitizeStruct function, the first line says:

<cfset var retVal = arguments.sourceStruct/>

In CF2016, the expectation was that this was creating a local copy of the passed struct (in our case the CGI scope).  So later in the function when we clear the struct:

<cfset structClear(local.retVal)/>

It was safe because we were clearing the LOCAL copy and not the original struct.

It seems that in CF2021, when we create out local variable, it must not be creating a pointer instead of a local COPY.  So later when we clear the struct, in CF2021, it seems to be clearing the original CGI scope.

I think this could possible be solved by changing this line:

<cfset var retVal = arguments.sourceStruct/>

To be this:

<cfset var retVal = Duplicate(arguments.sourceStruct)/>

This would ensure that the local variable is a duplicate of the passed struct instead of a reference to it."

[BKBK]

Hi @gabrieldavis321 ,

Karamba! It all makes sense now. Great explanation. Thanks for sharing.

( In any case, I expected CGI to be treated as a constant of the environment. But that's another story. )

[James Moberg]

I thought the CGI scope was read-only? I performed some tests... Lucee treats CGI as a read-only scope.

https://dev.to/gamesover/coldfusion-cgi-scope-is-not-read-only-1c8h

In my test, CF2016 and CF2021 both cleared the CGI scope that was passed to a UDF and then deleted by reference.  (I had to test locally since TryCF & CFFiddle both block access to the CGI scope.)

I wonder how many preexisting apps this pointer reference change (versus copy) is going to negatively impact.  This same nuance exists when porting any Adobe ColdFusion application to Lucee CFML. It's difficult to test and is one of the reasons we haven't been able to upgrade from CF2016 as of yet,  (We're slowly unit testing our framework and libraries using TestBox).

[BKBK]

CGI should be read-only, according to Adobe's own documentation. 

A bug report is in order: https://tracker.adobe.com/#/view/CF-4212734 

[w49369461]

What kind of load balancers are being used infront of these servers? nginx?