Skip to main content
J
jj2.
Participant
March 16, 2023
Question

ColdFusion using 2-3x more memory since service restart

  • March 16, 2023
  • 2 replies
  • 667 views

Hi,

 

Approximately one week ago we had an issue with one of our ColdFusion 2018 servers (something along the lines of "could not process Application.cfc"), which required a restart of both instances running on the server. I am not sure if this part is relevant but when restarted the first instance came back fine, while the second would not start for approximately 10 minutes (it took I think three separate service "start" attempts before it eventually came back online).

 

Prior to the restart we had consistent memory ussage on both instances of ~10-12% (average daily), while it has now been in the 25-40% daily average range for the last week according to FusionReactor. While there were minor changes to three cfm/cfc files on the same day, I don't believe they were anything that would explain this increase. Other metrics like CPU usage appear to be consistent with what they were previously.

 

Both instances have a minimum JVM heap size of 256MB, and a maximum of 5120MB. The server itself has 24GB total available, with approximately 10GB currently available.

 

Does anyone know what might explain this increase in memory usage? I am not even sure if the increase is actually an issue or whether the JVM memory management is now just working a bit differently for some reason? I guess my main concern is if there has now been a memory leak introduced, although the daily usage seems to be a bit up and down so I am not sure about that either.

 

I have attached a screenshot of the Heap Memory Graph for one of the instances from FusionReactor for reference. 

 

Regards,

 

Jeremy.

 

    This topic has been closed for replies.

    2 replies

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    March 16, 2023

    The Heap Memory Graph you've shared looks fine to me. However, you should rule out any errors. To do so, have a look at the ColdFusion logs and FusionReactor logs. Also have a look at the Longest Transactions in FusionReactor.

     

    You say there were code changes in cfm/cfc files. That could have led to the increase in memory use. For example, if the changes introduced new objects, memory use would increase.

     

    On to the problem of restarting. The onApplicationStart eventhandler may be using a lot more heap memory in the creation of objects. That might be compounded by the fact that the initial heap size of 256 MB is too small. To verify this idea, set Xms equal to Xmx:

     

    -Xms5120m -Xmx5120m

     

    Then restart the ColdFusion instance.

      J
      jj2.Author
      Participant
      March 16, 2023

      Thank you, BKBK. I have checked the error log and the only issue appears to be the following, but they are also recorded prior to when the restart occurred, so may not be having an impact in regards to this:

       

      org.apache.catalina.ha.session.DeltaManager messageReceived
      SEVERE: Manager [localhost#]: Unable to receive message through TCP channel

       

      I understand this is an issue with session messages being received out of order? Probably a topic for a separate discussion.

       

      In regards to new code changes, there was nothing that I would expect to have a noticable increase in memory - e.g. new obects etc. I will review again to confirm, however.

       

      Finally, thank you for the advice regarding the lower heap size value in regards to restarting. I will take a look at this and see if it makes a difference.

       

      Regards,

       

      Jeremy.

       

      Charlie Arehart
      Community Expert
      Charlie ArehartCommunity Expert
      Community Expert
      March 16, 2023

      Is your FusionReactor running as either its version 9.2.0 or 9.2.1? If so, there was a problem (with its default choice to enable some cf monitoring) which caused cf heap use to rise. Your graph doesn't look that bad, but since your concerned, if you update to 9.2.2, let us know if that solves things. 

      /Charlie (troubleshooter, carehart. org)
        J
        jj2.Author
        Participant
        March 16, 2023

        Thanks Charlie, FusionReactor is currently version 8.7.0, so it seems like we are a bit behind. Would what you described still be applicable? 

         

        Regards,

         

        Jeremy.

        Charlie Arehart
        Community Expert
        Charlie ArehartCommunity Expert
        Community Expert
        March 16, 2023

        Hi Charlie,

         

        Manually running the garbage collection dropped the used heap to ~250MB on both instances.

         

        Instance 1:

         

        Heap max: 4551 MB

        Heap allocated: 3438 MB

        Heap used: 253 MB

         

        Instance 2

         

        Heap max: 4551 MB

        Heap allocated: 3427 MB

        Heap used: 251 MB

         

        The session totals seems to be consistent and there are not any new automated processes or similar that would be creating new sessions etc.

         

        Like you have mentioned, there is probably nothing to worry about here, I am just being a bit pathological in that the reported numbers are now different.

         

        Regards,

         

        Jeremy.


        Yep, some people would kill to have CF run at only 250mb used heap. Count your blessings and call it a day. 🙂 Hope that the info I shared may help you in the future, or others as they may find it.

        /Charlie (troubleshooter, carehart. org)
          Terms & ConditionsAccessibility statement
          Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices