Client Storage Purge

Hi David,

I hope this information is helpful to you.

Symptom: CPU spins at 100% - and high memory usage - about every hour after startup (large memory use too).

Here's a common one that some fall into going from development to production. The default for storing client variables is "REGISTRY". Once the number of records in the registry is large, the query to get all records and delete expired client records can take 100% CPU for minutes at a time.

Registry client storage should never be used for production systems but often developers fall into this by accident by not explicitly specifying a data source (and the ColdFusion admin defaulting to "REGISTRY"). Since the registry isn't a real database, ColdFusion has to retrieve the entire registry client tree (high memory usage) and compare the date/time one at a time to decide whether to purge a record. This is a CPU-intensive operation with an hourly purge that may only delete a few records.

Regards,

Nimit

David, there’s no need for 21 questions. But conversely there is also not as simple an answer as some may purport.

Bottom line, yes, the CPU use is likely due to the purge that’s running (by default every 67 minutes). And because your default client storage is “registry”, it means that it’s purging the registry, and that can take a lot of CPU. But the answer is NOT to just change the default location to cookie, as some will assert. If you are really using client variables, and you change the location like that, then all the client vars will be “lost” for your users, which you may not want. But I appreciate the concern over performance, and the temptation to “do whatever it takes” to stop the madness of high CPU use.

But let me propose something else: it may not be that you need to be having CF write the client variables at all. If you don’t really use client variables in your code (setting values for variables like client.somename), note that you will STILL be writing to that client store on EVERY page request where you have enabled clientmanagement (in a cfapplication tag, typically in application.cfm, or with this.clientmanagement in application.cfc).

If you click on that “registry” link (or any database listed) on that client storage Admin page, you will see a setting for “disable global client variable updates”. That is NOT checked by default, which means that CF then tracks its own client variable (hitcount and lastvisit) for EVERY page that enables clientmanagement, even if it never “uses” client variables.

Now, should you turn that off? Likely so. The only caveat is that if you have any code in your app that USES these special variables (client.lastvisit and client.hitcount). If you do not, then you don’t need that feature enabled. Disabling it will stop CF writing those to that location (the registry, or a database selected to be a client storage mechanism), at least for requests to pages that don’t otherwise really use client variables. Many have found that doing that alone has helped a lot, in that each page request no longer writes to the client store. (And it’s worse for visits from spiders, bots and other automated requests, as they would CREATE A NEW client repository entry on EVERY page.

But doing that change alone WILL NOT STOP the hourly purge. That data is still there, and CF will still try to purge it. Now, some are tempted to say “well just change the purge interval to 24 hours or something”. And that will indeed stop it happening hourly, but it means that it will now happen at least daily, and the clock starts when CF comes up, so it may happen in the middle of the day. Not good.

The real solution is to remove the client variables yourself. But even that is not trivial.

First, if it’s a database, you may be tempted to just delete all the records in the cglobal table, but beware: if there are records in the cdata table (real client variables you’ve written from your code), then you don’t want to delete the records in cglobal that are related to those.

Second, it’s even harder when they’re in the registry. First: DO NOT go into regedit and click to expand the keys in the location where they are written (HKEY_LOCAL_MACHINE\SOFTWARE\Macromedia\ColdFusion\CurrentVersion\Clients). Doing that could bring your server down itself, if there are millions of keys (client variable entries) that have been written (which again is NOT that unusual).

Instead, you would want to delete them from a command line tool (like REG). But even that is not as straightforward as it seems. You would want to delete all those that are not really holding “real” client variables (if you use them at all). That’s not easy to figure out. But if you’re scrambling and willing to sacrifice any “real” client variables in purging the evil demons, you can do it from the command line with:

REG DELETE HKLM\SOFTWARE\Macromedia\ColdFusion\CurrentVersion\Clients

Just beware that may take a long time to run. You may want to do it during times when users would not be as affected. Then again, most servers are hit more by spiders and bots than by real people, so there’s never a “good time” when they may not be affected, but you may not care as much about them. That said, don’t ignore the significance of them (spiders, bots, and other automated requests) to be the real root cause of all this client variable bloat. In my day-to-day CF troubleshooting consulting practice, I find this to be a problem for someone pretty much every week.

Indeed, I have written about it and you can find more in the following entry(on the impact of both client variables and session variables, and by association memory, which makes people think CF has a “leak”):

http://www.carehart.org/blog/client/index.cfm/2006/10/4/bots_and_spiders_and_poor_CF_performance

You’ll see there also a link to a recorded presentation I also did on the topic.

So, no, not 21 questions. But perhaps 21 points to ponder. If this problem was easy (to understand and solve) it wouldn’t be such a burden. I do what I can, with the blog and presentation, but obviously many (most) will never see it. I didn’t just point you to it since your question was more specifically just about the client vars in the registry. I’ve drawn out the most salient points here.

Let us know if that helps.

/charlie arehart

charlie@carehart.org

Providing fast, remote, on-demand troubleshooting services for CF (and CFBuilder)

More at http://www.carehart.org/consulting

See also http://www.cf911.com for more on CF troubleshooting resources