CFChart issues

Report · Apr 17, 2017

I've been working on tracking down some funkiness with cfcharts. specifically their memory use. This is what I've noticed about it in CF2016 Update 3.

Adobe uses EHcache to cache cfcharts in a Cache Instance called "CF_Chart_CacheManager".
The "CF_Chart_CacheManager" doesn't get created until you use the cfchart tag.
Previous versions of CF used to have the ability to set Threads in the CFAdmin, CF2016 does not, although this is still available in the adminapi. Not sure if it works. The docs says you can set this from 1 to 5.
You can't set a custom Disk cache location. I can't set it in CFAdmin, the neo-graphing.xml, or the adminapi. It always resets back to the default.
Changing the settings CFAdmin-> Server Settings -> Charting requires a restart to take affect.
The Time-To-Live only appears to work when cache type is set to Disk Cache. This setting appears to be ignored when set to Memory.
It doesn't seem to matter if you set it to Disk Cache or Memory Cache, the cfchart images seem to generate in both places. This means disk activity and memory.
Although you can create the exact same chart over and over, it never really uses the previous chart its suppose to, it keeps creating new ones.
When you set to Disk Cache, EHCache Max Elements In Memory are set to 1 and Disk Persistence: true. The timeToLiveSecods is 0, which means forever and timeToIdleSeconds is 0, also forever. Although "Maximum number of cached images" don't seem to apply to the Disk Cache, but it does seem to apply to the Memory Cache.
Did I mention I hate ZingCharts? If they are going to break it, at lease break it with prettier charts. So many other options they could have chosen. These are ugly.
Even if you set timeToIdleSeconds or timeToLiveSeconds in EHcache, there is no thread that will come along and expire those unless they get accessed with a get(), then they return null and get expired. This isn't happening with the charts, so they just sit in the CF_Chart_CacheManager. So then you have to wait to hit your maxElementsInMemory for EHcache to expire them. This isn't an Adobe issue, it’s just how EHcache works.
Ultimately what's happening is that all these cfcharts are getting added to the EHcache Memory and not being removed until you hit a max. This is causing memory issues on my server.
I also suspect that there may be a memory leak in the jvm heap related to all of this. I haven't gotten proof yet, but I did a heap dump with JProfiler and I had about 5G's of charts in memory. Even if I maxed out my Max Elements In Memory, it shouldn’t have used all that memory. I'm doing some debugging and will get back to this thread with my findings.

I’ve created some code to test some stuff. Feel free to play around with it.

<cfscript>
/* what am I looking for */
chartCacheName='CF_Chart_CacheManager';
/* gets all the Cache Managers in EHCache */
cacheManagers = createObject('java', 'net.sf.ehcache.CacheManager').ALL_CACHE_MANAGERS;
for (item in cacheManagers) {
writeOutput('EHCache Manager Instance Name:' & #item.getName()# & '<br />');
/* pluck our Chart Cache */
if ( item.getName() == chartCacheName ) {
cm = item;
thisCache = cm.getCache(chartCacheName);
}
}
if(!isdefined('thisCache')) {
WriteOutput("CF_Chart_CacheManager doesn't exist yet.");
abort;
}
/* override configuration settings */
//thisCache.getCacheConfiguration().setTimeToLiveSeconds(30);
//thisCache.getCacheConfiguration().setTimeToIdleSeconds(30);
//thisCache.getCacheConfiguration().maxElementsInMemory(5);
</cfscript>

<cfoutput>
<ul>
<li>Count: #NumberFormat(thisCache.getStatistics().getSize())#</li>
<li>Max Elements In Memory: #numberformat(thisCache.getCacheConfiguration().getMaxelementsInMemory())#</li>
<li>Disk Persistance: #YesNoFormat(thisCache.getCacheConfiguration().isDiskPersistent())#</li>
<li>Eternal: #YesNoFormat(thisCache.getCacheConfiguration().isEternal())#</li>
<li>Overflow To Disk: #YesNoFormat(thisCache.getCacheConfiguration().isOverflowToDisk())#</li>
<li>Time To Live: #NumberFormat(thisCache.getCacheConfiguration().getTimeToLiveSeconds())#</li>
<li>Time To Idle: #NumberFormat(thisCache.getCacheConfiguration().getTimeToIdleSeconds())#</li>
<li>Exists: #YesNoFormat(cm.cacheExists(chartCacheName))#</li>
</ul>
</cfoutput>

<cfchart format="png" chartwidth="600">
<cfchartseries type="bar" colorlist="##00FF00,##CC0000,##CCFF00,##FF0000,##CC0099,##0000FF,##FFFF66,##FFFFFF">
<cfchartdata item="Count" value="#thisCache.getStatistics().getSize()#" />
<cfchartdata item="Hits" value="#thisCache.getStatistics().cacheHitCount()#">
<cfchartdata item="Disk Hits" value="#thisCache.getStatistics().localDiskHitCount()#" />
<cfchartdata item="Memory Hits" value="#thisCache.getStatistics().localHeapHitCount()#" />
<cfchartdata item="Memory Misses" value="#thisCache.getStatistics().localHeapMissCount()#" />
        <cfchartdata item="On Disk" value="#thisCache.getStatistics().getLocalDiskSize()#" />
        <cfchartdata item="On Heap" value="#thisCache.getStatistics().getLocalHeapSize()#" />
        <cfchartdata item="Off Heap" value="#thisCache.getStatistics().getLocalOffHeapSize()#" />
</cfchartseries>
</cfchart>

<cfset WriteDump(thisCache.getStatistics())>

<cfset WriteDump(thisCache.getKeys())>

Report · Apr 17, 2017

I put in a request on the ColdFusion Slack team Adobe channel for someone at Adobe to take a look.

Report · Apr 17, 2017

Can you submit a bug report for the various caching issues you've identified: ColdFusion Issue Tracker?

-Carl V.

Report · Apr 17, 2017

I will, I'm just trying to fully understand what's happening under the hood and put together some reproducible examples. I probably need another day to put something together.

Report · Apr 17, 2017

Also the NAME attribute doesn't work at all when chart type is flash or html. I've created a bug report: CF-4198527

The memory leak issue with cfcharts eats 12MB of server RAM every time I refresh a page with 8 charts in it. The CF process went from 1GB (physical RAM) to over 2GB after a couple hours of just me refreshing the page - after rebooting it from when it crashed earlier for the same reason! (Java out of memory error). I cannot use CF2016 in production in this state.

I also don't like Zingcharts, although I'm about 4 years late to this party! They look awful compared to Webcharts3D, which is also smarter at formatting pies, bars, legends and tooltips.

So basically everything in my application to do with charts is broken functionally and aesthetically when upgrading to CF2016.

Report · Apr 25, 2017

It appears that each Ehcache ZingCharts object uses a lot of memory. If you have the "Maximum number of cached images" set to a high number (500+), this can eat up lots of memory and never let it go. Ehcache has a setting called "timeToLiveSeconds", but it doesn't do what you think it does. It simply invalidates the object, it does not remove it from memory. So if you have 1000 objects cached in Ehache, it will use all the memory for those 1000 objects, even if you have a short "timeToLiveSeconds". The only way Ehcache starts to evict objects from memory in CF is when you hit the "Maximum number of cached images". Ehcache does have a method called evictExpiredElements() that could evict them, but there is no process to call this. This is by design for performance.

I was able to create a workaround for the memory issue, at least in terms of it not using all the memory. This doesn't resolve the other issues, but keeps the server from crashing. You should set the "Cache type" to Memory and set a low "Maximum number of cached images". I set mine to 100.

CFAdmin doesn't set the Ehcache timeToLiveSeconds, but you think it would with the setting "Time-to-Live of each chart in seconds", so you can ignore that. That setting only relates to "Cache type": Disk. If you have a "Cache type": Disk, Adobe sets the initial Ehcache "Max Elements In Memory" to 1 and "Overflow To Disk" to true, but I noticed that "Max Elements In Memory" number can be exceeded. When this happens, I'm not sure there is a limit, eventually you'll run out of memory.

Report · Apr 25, 2017

Interesting, Neo. As an experiment I set cache type to Memory (previously disk) and used the default of 50 for the cache. I restarted the CF service for good measure and ran my cfchart script. CF generated temp files on disk for each chart! That's not what I call a memory cache.

Reloading the page constantly ate away at the server's memory. An additional 700MB was consumed by coldfusion.exe in 10 minutes. 😞

Report · Apr 25, 2017

Yeah, I think it has to write to disk to server them up through the CFFileServlet? The time-to-idle will still delete the ones on disk at that amount of time, even if Cache Type is Memory. I've been running some production traffic over my CF2016 install and have been monitoring it with the code below. My 100 "cached" charts seem stable around "On Heap Bytes: 669,560". I wonder if the type of charts matters? Also, if you're running a big loop, it might not have time to do a garbage collection. Might want to force one.

Update 4 just came out today, I don't think it fixes some of my issues, but maybe its better since they did address some Chart issues. I'm going to apply it and see what happens.

/* get an instance of Ehcache */

Ehcache = CreateObject('java', 'net.sf.ehcache.CacheManager');

/* get cacheManager instance */

cm = Ehcache.getCacheManager('CF_Chart_CacheManager');

/* this call will initialize */

cmStatus = cm.getStatus();

/* get cache */

thisCache = cm.getCache('CF_Chart_CacheManager');

</cfscript>

<ul>

<li>Count: #NumberFormat(thisCache.getStatistics().getSize())#</li>

<li>Max Elements In Memory: #numberformat(thisCache.getCacheConfiguration().getMaxelementsInMemory())#</li>

<li>Disk Persistance: #YesNoFormat(thisCache.getCacheConfiguration().isDiskPersistent())#</li>

<li>Eternal: #YesNoFormat(thisCache.getCacheConfiguration().isEternal())#</li>

<li>Overflow To Disk: #YesNoFormat(thisCache.getCacheConfiguration().isOverflowToDisk())#</li>

<li>Time To Live: #NumberFormat(thisCache.getCacheConfiguration().getTimeToLiveSeconds())#</li>

<li>Time To Idle: #NumberFormat(thisCache.getCacheConfiguration().getTimeToIdleSeconds())#</li>

<li>On Heap: #NumberFormat(thisCache.getStatistics().getLocalHeapSize())#</li>

<li>Off Heap: #NumberFormat(thisCache.getStatistics().getLocalOffHeapSize())#</li>

<li>On Heap Bytes: #NumberFormat(thisCache.getStatistics().getLocalHeapSizeInBytes())#</li>

<li>Off Heap Bytes: #NumberFormat(thisCache.getStatistics().getLocalOffHeapSizeInBytes())#</li>

<li>On Heap Hit Count: #NumberFormat(thisCache.getStatistics().localHeapHitCount())#</li>

<li>Cache Evict Count: #NumberFormat(thisCache.getStatistics().cacheEvictedCount())#</li>

<li>Cache Expired Count: #NumberFormat(thisCache.getStatistics().cacheExpiredCount())#</li>

<li>Max Bytes Local Heap: #NumberFormat(thisCache.getCacheConfiguration().getMaxBytesLocalHeap())#</li>

<li>Max Bytes Local Heap: #NumberFormat(thisCache.getCacheConfiguration().getMaxBytesLocalOffHeap())#</li>

<li>Max Bytes Off Heap: #NumberFormat(thisCache.getCacheConfiguration().getMaxMemoryOffHeap())#</li>

</ul>

</cfoutput>

Report · Apr 25, 2017

Update 4 doesn't fix the memory leak issue unfortunately. Adobe are still working on it and have asked me to send them some dumps.

My cfchart loop is just javascript reloading the page, so there's never a build up of requests, and no one else is using the dev server. I don't think garbage collection is supposed to wait until there are no requests, otherwise GC would never happen on a busy production server.

Report · Apr 18, 2017

This won't help your performance issue, but I highly recommend looking at the zingchart documentation to get the look and feel you want. Relying on cfchart is probably a mistake, but understanding that recoding may not be an option, using the styling capabilities will help a lot.

Adobe Community

CFChart issues