Skip to main content
ilssac
ilssac
Inspiring
February 16, 2010
Question

CPU spinning at 50%

  • February 16, 2010
  • 4 replies
  • 4063 views

In my continuing saga to figure out what is going on with my ColdFusion server.  The CPU started spinning at 50% last Thursday midday and it has not stopped in the four days since.

For perspective this if for our internal web server that historically, normally operates under 5% CPU usage.   Plus this is our department internal web application server for the California Department of Pesticide Regulation, thus with the Furlough last Friday, the weekend and the holiday yesterday, there was nobody here using it the last four days.  But the server never wavered and sat at a steady 50% the entire time.

I have taken a snapshot of the threads currently running on this server.  It can be seen here along with several I took last Thursday.
http://www.ilsweb.com/JRun4-stack-traces/

The only thing that I see that *might* be odd, is there is this one thread that had never changed in the last four days.  Can anybody tell me if it is unusual in any way?

Name: Thread-110
State: RUNNABLE
Total blocked: 0  Total waited: 0

Stack trace:
sun.print.Win32PrintServiceLookup.notifyPrinterChange(Native Method)
sun.print.Win32PrintServiceLookup.access$100(Win32PrintServiceLookup.java:32)
sun.print.Win32PrintServiceLookup$PrinterChangeListener.run(Win32PrintServiceLookup.java:302)

I did some searching on this and got some hints that it might be an issue, but nothing definitive nor any concise idea what this might really mean.

By the way, is there some way to tie a specific thread to a piece of code?

TIA

Ian

    This topic has been closed for replies.

    4 replies

    ilssac
    ilssacAuthor
    Inspiring
    March 3, 2010

    NEW INFORMATION

    Using process explore from Microsoft's sysinternals website.  We narrowed down the CPU spinning to a thread running Msvcr71.dll inside of the Jrun process inside the Jrunsvc process.  When we suspended this thread that was using 50% of the servers CPU, the problem resolved itself.  As best as I can tell this has not introduced any instability on the server.  But it is scheduled for a reboot soon anyway, so it should get a clean start at that time.


    Searching for information on this dll I have never been aware of before, I found Jochem's web blog here:

    http://jochem.vandieten.net/tag/msvcr71dll/


    Does it seem resonable to try this fix for our problem, just grabbing the latest version of the Msvc71.dll file and putting it into the Jrun folder?  I presume this is the root Jrun folder or should it be somewhere deeper?  Does having a multi-server configuration matter?  Should we have the same Msvcr71.dll file in each Jrun server folder?


    I'm uncertain this is a related problem as only one of our four JRun, aka ColdFusion, servers in our multi-home configuration has experienced this spinning problem.  But I am grasping at straws here.

    ilssac
    ilssacAuthor
    Inspiring
    March 1, 2010

    I have no idea if this is a related problem or not,

    But I have notices that our {serverName}-out logs are filling up with messages like this.

    02/28 18:31:27 Information [cfthread-112] - BrowserFontProvider: Can't find font file for @Arial Unicode MS
    02/28 18:31:27 Information [cfthread-112] - BrowserFontProvider: Can't find font file for @Arial Unicode MS
    02/28 18:31:27 Information [cfthread-112] - BrowserFontProvider: Can't find font file for @Arial Unicode MS

    Thousands of them, filling up dozens of log files every day.

    Even if this is not related to the 50% cpu problem, this is not a good thing is it?

    A
    Adam Cameron.
    Inspiring
    March 2, 2010

    Well it's less than ideal ;-)

    Is the font actually listed in CFAdmin?

    You had some slow threads going on about printing the other day, didn't you?  It might be some sort of weirdness with <cfprint> trying to do something with font selections, or checking what fonts are available or something?  I've seen printers get arsey about fonts being missing in the past... although I have reason to think <cfprint> would... but, you know: worth looking at.

    Or it's so common a font that perhaps CF just ass-u-me`s that it's there, and some inner workings of its PDF management gets screwed up with it missing?

    All wild wild guesses, of course.

    --

    Adam

    ilssac
    ilssacAuthor
    Inspiring
    March 2, 2010

    A Cameron wrote:

    Well it's less than ideal ;-)

    Is the font actually listed in CFAdmin?

    I don't know if it is in CFAdmin or not, I guess I should look.

    But I have already determined that it does not exist on the server.  Apparently this is a "special" MS version of Arial that would be installed if you have installed any of the common office productivity tools from the MS Office suite.   Something, of course, we have not done on our server.  If you don't, you are supposed to buy an Arial Unicode MS license to install it on a system without a licensed version of one of these Office tools.

    I think I will just recomend that the application be reworked to use one of the other six flavors of Arial that are already on the server.  I am 99.99999% sure that this PDF generation code is not using anything that is only available in the Arial Unicode MS version of the font.

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    February 25, 2010

    Sticks out like the proverbial sore thumb: "Flash Remoting Running requests"

    ilssac
    ilssacAuthor
    Inspiring
    February 25, 2010

    BKBK wrote:

    Sticks out like the proverbial sore thumb: "Flash Remoting Running requests"

    Except that the only Flash Remoting running is the ColdFusion Monitor used to get the thread dump, and it was only turned on after the CPU started spinning.  So I have not put much weight into that being part of the problem.

    A
    Anonymous
    February 25, 2010

    Im wondering if it could be the GC going nuts?

    A
    Adam Cameron.
    Inspiring
    February 21, 2010

    You've not just restarted the CF service in the hope it was just a temporary aberation?

    Have you run FusionReactor (or CF's own server monitoring, I guess... never used it...) on it to see what might be causing it?  You should just be able to kill that thread.

    FusionReactor, from memory, will let you cross reference threads to actual code.

    --

    Adam

    ilssac
    ilssacAuthor
    Inspiring
    February 23, 2010

    A Cameron wrote:

    You've not just restarted the CF service in the hope it was just a temporary aberation?

    Yes many times, and sooner or later the spinning has returned.

    Have you run FusionReactor (or CF's own server monitoring, I guess... never used it...) on it to see what might be causing it?  You should just be able to kill that thread.

    FusionReactor, from memory, will let you cross reference threads to actual code.

    I'll have to check out FusionReactor to see if it truely does cross reference threads to actual code.  That is something I have not been able to do for the threads that hang around.  With the jrpp threads that have the cfml reference as part of the thread dump it is pretty simple, but none of those threads seem to be the problem as far as I can tell from the thread dumps I have gotten from the ColdFusion built in monitoring tools.

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices