Coldfusion 2016 vs Coldfusion 9 compile time

[Nebukad]

Hi,

We are in the process of upgrading from coldfusion 9 to 2016 and we see an overall decrease of performance for coldfusion 2016. As far as we read coldfusion 2016 should be faster then coldfusion 9but this is not the case. Below a test script that can be run to see the difference in performance:

<!--- Machine info --->

<cfset runtime = createObject("java", "java.lang.System")>

<cfset props = runtime.getProperties()>

<cfset env = runtime.getenv()>

<Cfoutput>

java.version: #props["java.version"]#<br>

java.vm.name: #props["java.vm.name"]#<br>

os.name: #props["os.name"]#<br>

PROCESSOR_IDENTIFIER: #env["PROCESSOR_IDENTIFIER"]#<br>

PROCESSOR_ARCHITECTURE: #env["PROCESSOR_ARCHITECTURE"]#<br>

NUMBER_OF_PROCESSORS: #env["NUMBER_OF_PROCESSORS"]#<br><Br>

</Cfoutput>

<!--- Create a query --->

<cfset myQuery = QueryNew("Name, Time, Advanced", "VarChar, Time, Bit")>

<cfset testQuery = QueryNew("ColumnA,ColumnB,ColumnC,ColumnD,ColumnE,ColumnF,ColumnG,ColumnH,ColumnI,ColumnJ,ColumnK,ColumnL,ColumnM,ColumnN","VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar,VarChar")>

<!--- Populate the query --->

<Cfloop from=1 to=10 index="x">

    <cfset QueryAddRow(testQuery, 1)>

    <cfloop index="intLetter" from="#Asc('A')#" to="#Asc('N')#" step="1">

        <cfset temp = QuerySetCell(testQuery, "Column#chr(intLetter)#", "Row #x# column #intLetter#", x)>

    </cfloop>

</cfloop>

<Cfset init = GetTickCount()>

<cfloop from=1 to=10 index="x">

    <Cfset init2 = GetTickCount()>

    <!--- Query to structure --->

    <Cfset queryToStruct = structNEw()>

    <cfloop query="testQuery">

        <cfset queryToStruct[testQuery['ColumnA'][testQuery.currentrow]] = structNew()>

        <cfloop list="#testQuery.columnList#" index="key">

            <cfset queryToStruct[testQuery['ColumnA'][testQuery.currentrow]][testQuery[key][testQuery.currentrow]] = evaluate(key)>

        </cfloop>

    </cfloop>

    <cfoutput>#x#:#GetTickCount()-init2#<br></cfoutput>

</cfloop>

<cfoutput>-----------<br><b>#GetTickCount()-init#</b><br><br><Br></cfoutput>

<Cfdump var=#queryToStruct#>

I ran this code on two machines with the exact same hardware specifications (same cpu / same ram). Both versions have the same Minimum JVM Heap Size (5024 MB) and Maximum JVM Heap Size (5048 MB). The performance on the coldfusion 9 server is by average 4x faster. My question is why? And is there anything i can do to improve the performance on the coldfusion 2016 machine. If necessary live exampes are available.

Update:

I wonderded if the windows server / IIS slowed things down on de coldfusion 2016 machine so i did some further testing. I installed the coldfusion 9 developer edition, coldfusion 11 developer edition and the coldfusion 2016 developer edition all with the build in webserver on the same VPS. Running the script above.the coldfusion 9 is fastest, coldfusion 11 seems slightly slower. Coldfusion 2016 is much slower then the older versions.

Message was edited by: Thijs

[Dave Watts]

I think you've got a whole bunch of things going on here. First, it looks like you're not measuring compile time, but run time. Run time is more important anyway, but you should understand the difference. Your pages are compiled the first time they're run, and potentially may be recompiled in the future if the compiled code is no longer stored. Second, the optimal settings for each version of CF are quite likely to be different, as they use different JVMs and servlet containers, etc. Pretty much everything is up in the air, performance-wise, unless you've tuned your environment for optimal performance. Third, unless you plan to serve results through the built-in web server, testing performance through it is kind of questionable. You want to include the performance of the web server and the connector etc. Fourth, the test you have focuses on creating and manipulating a query object. It is quite possible that this is slower in 2016 than in older versions, but it's not representative of the vast majority of CFML code in the world. Finally, this test is not going to be all that useful even after you've tuned your environment, because it is a serial test of one CFML file. If you really want to know how well CF works, you really have to test it in parallel, with a load test tool. Your users will interact with your site in parallel, and your test regimen should do the same.

Dave Watts, Fig Leaf Software

[Nebukad]

Thanks for your reply. You are absolutely right about compile time / run time.

Your second point about the optimal settings can be correct however the difference in performance is to big and i made a clean install no changes to the default settings.

I agree with your third point, however it is very difficult to test something like this. I tried to write a simple reproducible code which can be tested on different versions of coldfusion.

Regarding your fourth point, we noticed a decrease in overall performance when trying to migrate from coldfusion 9 to 2016. This is just a very simple example, my question in not about this code.

Regarding your final comment, so you can have an idea what kind of results we are getting take a look at the following page:

http://136.144.177.152/test2.asp

http://136.144.177.152/test-toma.asp

http://136.144.177.152/test-ag.asp ​

The code is actually processed by the different coldfusion versions. So you can reload the page to update the results.

[Nebukad]

There seems to be a problem with indexed access notation for queries and arrays. A live example can be viewed here:

http://136.144.177.152/bug-adobe.asp