Highlighted

CFThread GC Overhead Limit Exceeded

New Here ,
Aug 18, 2014

Copy link to clipboard

Copied

I have an application that trades virtual items and have a single page which gets all my accounts and for each one creates a thread that firstly logs the account in and then searches and buys for items for as long as the session is active.  I should point out at this point that this is my first experience of using cfthread.

I'm having issues with it. Every 30 minutes (if not less) my ColdFusion server comes to a standstill and I have to restart the service. Upon restarting the service I check the logs and there are errors that say "GC Overhead Limit Exceeded".

I have looked extensively online but as much as cfthread is new to me, so is the JVM and how it operates. I'm running on CF10 Enterprise Edition and have loaded up the server monitor and surely enough I can see the JVM memory usage grow and grow until the limit has been reached (just now I have it set as 2gb as when I had it set higher the memory seemed to fill up quicker). Even when I select the Run GC option in the monitor it does not reduce the memory usage very much, if at all.

Is this more than likely something to do with my code? At the moment I have just under 50 threads being created but as I add more accounts to the application then the more threads that will be required.

Here is the code from the page...

<script>

  /* RELOAD PAGE EVERY 65 MINUTES */

  setTimeout(function(){

    window.location.reload(1);

  }, 3900000);

</script>

<!--- GET ACTIVE ACCOUNTS --->

<cfquery name="getLogins" datasource="myDB">

SELECT * FROM Logins WHERE active = 1

</cfquery>

<!--- LOOP THROUGH ACCOUNT --->

<cfloop query="getLogins">

  <!--- HAVE A SLEEP SO IP DOESN'T GET FLAGGED FOR SENDING TOO MANY REQUESTS AT ONCE --->

  <cfset Sleep(30000) />

  <!--- CREATE THREAD FOR ACCOUNT --->

  <cfthread

  name="#getLogins.accountName#"

  action="run"

  accountName="#Trim(getLogins.accountName)#"

  email="#Trim(getLogins.email)#"

  password="#Trim(getLogins.password)#"

  resourceId="#Trim(getLogins.resourceID)#">

  <!--- DEFAULT SESSION VARIABLES --->

  <cfset SESSION["#attributes.accountName#LoggedIn"] = 0 />

  <cfset SESSION["#attributes.accountName#LoginAttempts"] = 0 />

  <!--- WHILE ACCOUNT NOT LOGGED IN AND LESS THAN 8 LOGIN ATTEMPTS MADE --->

  <cfscript>

  while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) {

  // ATTEMPT LOGIN

  THREAD.logInAccount = Application.cfcs.Login.logInAccount(attributes.email,attributes.password);

  // IF LOGIN ATTEMPT UNSUCCESSFUL

  if (THREAD.logInAccount EQ 0) {

  // INCREASE ATTEMPT COUNT

  SESSION['#attributes.accountName#LoginAttempts'] = SESSION['#attributes.accountName#LoginAttempts'] + 1;

  }

  // ELSE IF RETURNED VALUE IS 481 THEN ACCOUNT IS LOCKED

  else if (THREAD.logInAccount EQ 481) {

  // SET LOGIN ATTEMPT COUNT TO STOP LOOP

  SESSION['#attributes.accountName#LoginAttempts'] = 8;

  // UPDATE ACCOUNT TO MARK AS LOCKED

  THREAD.updLogin = Application.cfcs.Login.updLogin(attributes.email);

  }

  }

  </cfscript>

  <!--- IF ACCOUNT LOGGED IN --->

  <cfif SESSION['#attributes.accountName#LoggedIn'] EQ 1>

  <!--- SET ID FOR SEARCHING --->

  <cfset THREAD.definitionID = attributes.resourceID - 1610612736 />

  <!--- WHILE ACCOUNT LOGGED IN --->

  <cfloop condition="SESSION['#attributes.accountName#LoggedIn'] EQUALS 1">

  <!--- GET LATEST LOWEST BUY NOW PRICE --->

  <cfquery name="THREAD.getMinBIN" datasource="FUT" cachedWithin="#CreateTimeSpan(0,0,1,0)#">

  SELECT TOP 1 * FROM v_FUT14BINPrices WHERE resourceID = #attributes.resourceId# ORDER BY lastUpdated DESC

  </cfquery>

  <!--- INCLUDE FILE THAT CALCULATES BUYING AND SELLING PRICES --->

  <cfinclude template="sellingPrices.cfm" />

  <!--- IF BIDDING PRICE HAS BEEN SET --->

  <cfif StructKeyExists(THREAD,"biddingPrice")>

  <!--- MAKE SEARCH REQUEST, TIMING THE REQUEST --->

  <cfset THREAD.requestStart = GetTickCount() />

  <cfset THREAD.search = Application.cfcs.Search.dosearchOld(attributes.resourceId,THREAD.biddingPrice,0) />

  <cfset THREAD.requestDuration = GetTickCount() - THREAD.requestStart />

  <!--- IF SEARCH CONTAINS FILE CONTENT  --->

  <cfif StructKeyExists(THREAD.search,"FileContent")>

  <!--- DECLARE NUMBER OF RESULTS VARIABLE --->

  <cfset THREAD.numResults = 0 />

  <!--- IF JSON RETURNED --->

  <cfif IsJSON(THREAD.search.FileContent)>

  <!--- DESERIALIZE JSON --->

  <cfset THREAD.searchResults = DeserializeJSON(THREAD.search.FileContent) />

  <!---  IF PLAYER SEARCH RETURNS AUCTIONINFO STRUCT --->

  <cfif StructKeyExists(THREAD.searchResults,"auctionInfo")>

  <!--- SET NUMBER OF CARDS RETURNED FROM SEARCH --->

  <cfset THREAD.numResults = ArrayLen(THREAD.searchResults.auctionInfo) />

  <cfset THREAD.statusCode = "Successful" />

  <cfif THREAD.numResults EQ 0>

  <cfset THREAD.statusCode = "Successful - No Results" />

  </cfif>

  <!--- ELSE IF ERROR CODE RETURNED --->

  <cfelseif StructKeyExists(THREAD.searchResults,"code")>

  <cfset THREAD.statusCode = THREAD.searchResults.code />

  <!--- IF CODE 401 THEN SESSION HAS EXPIRED --->

  <cfif THREAD.statusCode EQ 401>

  <!--- SET SESSION AS LOGGED OUT AND ATTEMPT SESSION REFRESH --->

  <cfset SESSION['#attributes.accountName#LoggedIn'] = 0 />

  <cfset THREAD.logInAccount = Application.cfcs.Login.logInAccount(attributes.email,attributes.password) />

  </cfif>

  <!--- ELSE SOMETHING ELSE HAS HAPPENED --->

  <cfelse>

  <cfset THREAD.statusCode = "Something Else - " & THREAD.searchResults.code />

  </cfif>

  <!--- IF RESULTS RETURNED --->

  <cfif THREAD.numResults GT 0>

  <!--- LOOP ROUND RESULTS AND CHECK IF MATCH BUYING CRITERIA --->

  <cfloop index="i" from="1" to="#THREAD.numResults#">

  <!--- ***SAFETY CHECK*** - ENSURE ID OF CURRENT CARD IS SAME AS ONE SEARCHING FOR --->

  <cfif THREAD.searchResults.auctionInfo.itemData.resourceID EQ attributes.resourceId AND THREAD.getMinBIN.resourceID EQ attributes.resourceId>

  <!--- ENSURE BIN PRICE SET AND IS LESS THAN SET BUYING PRICE --->

  <cfif THREAD.searchResults.auctionInfo.buyNowPrice GT 0 AND THREAD.searchResults.auctionInfo.buyNowPrice LTE THREAD.biddingPrice>

  <!--- SET AUCTION END TIME --->

  <cfset THREAD.timeLeft = THREAD.searchResults.auctionInfo.expires />

  <cfset THREAD.auctionEnds = DateAdd("s",THREAD.timeLeft,Now()) />

  <!--- BUY CARD --->

  <cfset THREAD.buyCard = Application.cfcs.Bid.doBIN(THREAD.searchResults.auctionInfo.tradeID,THREAD.searchResults.auctionInfo.buyNowPrice,THREAD.searchResults.auctionInfo.startingBid,THREAD.searchResults.auctionInfo.itemData.ID,THREAD.searchResults.auctionInfo.itemData.resourceID,THREAD.startPrice,THREAD.binPrice,THREAD.lowestBIN,THREAD.searchResults.auctionInfo.itemData.discardValue,THREAD.auctionEnds,THREAD.requestStart,THREAD.requestDuration) />

</cfif>

  </cfif>

  </cfloop>

  </cfif>

  <cfelse>

  <cfset THREAD.statusCode = THREAD.search.FileContent />

  </cfif>

  <cfset THREAD.sleepDuration = 1000 - THREAD.requestDuration />

  <cfif THREAD.sleepDuration GT 0><cfset Sleep(THREAD.sleepDuration) /></cfif>

  </cfif>

  <!--- INSERT SEARCH RECORD --->

  <cfset THREAD.insSearchRecord = Application.cfcs.Search.insSearchRecord(THREAD.definitionID,THREAD.statusCode,THREAD.requestDuration,THREAD.numResults,THREAD.biddingPrice) />

  </cfif>

  </cfloop>

  </cfif>

  </cfthread>

</cfloop>

I would have thought that the memory would have stayed around the same usage as each loop is performing the same set of actions so once the loop has went back to the start then I thought the previous loop would have been removed from memory (freeing up space) and then the same actions would be performed so the same memory total would then be used up but it seems almost as if each loop is being kept in memory and that is why it is growing.

Could someone please help me out and offer some guidance on how I could remedy this issue? If you need any more info then just let me know

Thanks in advance

Capture.JPG

JVM looks like this thanks to gcviewer:


The diagram does not show PermGen details. Tail of log says:

PSPermGen  total 1048576K, used 80172K object space 1048576K, 7% used.

While you may have something to resolve with the CFM code, for now to keep the system up perhaps you can do better to make some JVM adjustments. Keep in mind this is probably not a fix for the overall problem more try this to see if the system stays up while you continue to work on other CFM matters.

I expect JVM details are like this currently:


Minimum heap size: 1024
Maximum heap size: 2048
-XX:MaxPermSize=1024m

The overall system is:

CF10 on a Windows 7 x64 machine with an Intel core i3-2100 processor and 8GB RAM.


Seeing not many objects are maintained in PermGen you can make that smaller. Double heap sizes and set a value for the new part of heap. EG:

Minimum heap size: 2048
Maximum heap size: 4096
-XX:MaxPermSize=324m
-Xmn256m

Or this way if you prefer in CFadmin:

-server -Xmn256m -XX:MaxPermSize=324m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log

Or this way by editing JVM.CONFIG:

-server -Xms2048m -Xmx4096m -Xmn256m -XX:MaxPermSize=324m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log


For now it might be advisable to set JVM to perform full garbage collection every 10 minutes but as I post I am undecided so will post again more thoughts on that if it seems more like a good bandaid idea to me. I think stay with UseParallelGC because it tends to keep the heap evacuated rather than alter garbage collector to something that tends to maintain objects in memory.


Java is 7 so not EOL 6 tho fair to say 7u15 is old with 7u67 current. Java 8 is also release however no support statement from Adobe with running  CF10 on Java 8. Since you are not using a newer garbage collection algorithim like G1GC (and for now I do not recommend you do) I think stay with 7u15 for now.

HTH, Carl.

TOPICS
Server administration

Views

6.6K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

CFThread GC Overhead Limit Exceeded

New Here ,
Aug 18, 2014

Copy link to clipboard

Copied

I have an application that trades virtual items and have a single page which gets all my accounts and for each one creates a thread that firstly logs the account in and then searches and buys for items for as long as the session is active.  I should point out at this point that this is my first experience of using cfthread.

I'm having issues with it. Every 30 minutes (if not less) my ColdFusion server comes to a standstill and I have to restart the service. Upon restarting the service I check the logs and there are errors that say "GC Overhead Limit Exceeded".

I have looked extensively online but as much as cfthread is new to me, so is the JVM and how it operates. I'm running on CF10 Enterprise Edition and have loaded up the server monitor and surely enough I can see the JVM memory usage grow and grow until the limit has been reached (just now I have it set as 2gb as when I had it set higher the memory seemed to fill up quicker). Even when I select the Run GC option in the monitor it does not reduce the memory usage very much, if at all.

Is this more than likely something to do with my code? At the moment I have just under 50 threads being created but as I add more accounts to the application then the more threads that will be required.

Here is the code from the page...

<script>

  /* RELOAD PAGE EVERY 65 MINUTES */

  setTimeout(function(){

    window.location.reload(1);

  }, 3900000);

</script>

<!--- GET ACTIVE ACCOUNTS --->

<cfquery name="getLogins" datasource="myDB">

SELECT * FROM Logins WHERE active = 1

</cfquery>

<!--- LOOP THROUGH ACCOUNT --->

<cfloop query="getLogins">

  <!--- HAVE A SLEEP SO IP DOESN'T GET FLAGGED FOR SENDING TOO MANY REQUESTS AT ONCE --->

  <cfset Sleep(30000) />

  <!--- CREATE THREAD FOR ACCOUNT --->

  <cfthread

  name="#getLogins.accountName#"

  action="run"

  accountName="#Trim(getLogins.accountName)#"

  email="#Trim(getLogins.email)#"

  password="#Trim(getLogins.password)#"

  resourceId="#Trim(getLogins.resourceID)#">

  <!--- DEFAULT SESSION VARIABLES --->

  <cfset SESSION["#attributes.accountName#LoggedIn"] = 0 />

  <cfset SESSION["#attributes.accountName#LoginAttempts"] = 0 />

  <!--- WHILE ACCOUNT NOT LOGGED IN AND LESS THAN 8 LOGIN ATTEMPTS MADE --->

  <cfscript>

  while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) {

  // ATTEMPT LOGIN

  THREAD.logInAccount = Application.cfcs.Login.logInAccount(attributes.email,attributes.password);

  // IF LOGIN ATTEMPT UNSUCCESSFUL

  if (THREAD.logInAccount EQ 0) {

  // INCREASE ATTEMPT COUNT

  SESSION['#attributes.accountName#LoginAttempts'] = SESSION['#attributes.accountName#LoginAttempts'] + 1;

  }

  // ELSE IF RETURNED VALUE IS 481 THEN ACCOUNT IS LOCKED

  else if (THREAD.logInAccount EQ 481) {

  // SET LOGIN ATTEMPT COUNT TO STOP LOOP

  SESSION['#attributes.accountName#LoginAttempts'] = 8;

  // UPDATE ACCOUNT TO MARK AS LOCKED

  THREAD.updLogin = Application.cfcs.Login.updLogin(attributes.email);

  }

  }

  </cfscript>

  <!--- IF ACCOUNT LOGGED IN --->

  <cfif SESSION['#attributes.accountName#LoggedIn'] EQ 1>

  <!--- SET ID FOR SEARCHING --->

  <cfset THREAD.definitionID = attributes.resourceID - 1610612736 />

  <!--- WHILE ACCOUNT LOGGED IN --->

  <cfloop condition="SESSION['#attributes.accountName#LoggedIn'] EQUALS 1">

  <!--- GET LATEST LOWEST BUY NOW PRICE --->

  <cfquery name="THREAD.getMinBIN" datasource="FUT" cachedWithin="#CreateTimeSpan(0,0,1,0)#">

  SELECT TOP 1 * FROM v_FUT14BINPrices WHERE resourceID = #attributes.resourceId# ORDER BY lastUpdated DESC

  </cfquery>

  <!--- INCLUDE FILE THAT CALCULATES BUYING AND SELLING PRICES --->

  <cfinclude template="sellingPrices.cfm" />

  <!--- IF BIDDING PRICE HAS BEEN SET --->

  <cfif StructKeyExists(THREAD,"biddingPrice")>

  <!--- MAKE SEARCH REQUEST, TIMING THE REQUEST --->

  <cfset THREAD.requestStart = GetTickCount() />

  <cfset THREAD.search = Application.cfcs.Search.dosearchOld(attributes.resourceId,THREAD.biddingPrice,0) />

  <cfset THREAD.requestDuration = GetTickCount() - THREAD.requestStart />

  <!--- IF SEARCH CONTAINS FILE CONTENT  --->

  <cfif StructKeyExists(THREAD.search,"FileContent")>

  <!--- DECLARE NUMBER OF RESULTS VARIABLE --->

  <cfset THREAD.numResults = 0 />

  <!--- IF JSON RETURNED --->

  <cfif IsJSON(THREAD.search.FileContent)>

  <!--- DESERIALIZE JSON --->

  <cfset THREAD.searchResults = DeserializeJSON(THREAD.search.FileContent) />

  <!---  IF PLAYER SEARCH RETURNS AUCTIONINFO STRUCT --->

  <cfif StructKeyExists(THREAD.searchResults,"auctionInfo")>

  <!--- SET NUMBER OF CARDS RETURNED FROM SEARCH --->

  <cfset THREAD.numResults = ArrayLen(THREAD.searchResults.auctionInfo) />

  <cfset THREAD.statusCode = "Successful" />

  <cfif THREAD.numResults EQ 0>

  <cfset THREAD.statusCode = "Successful - No Results" />

  </cfif>

  <!--- ELSE IF ERROR CODE RETURNED --->

  <cfelseif StructKeyExists(THREAD.searchResults,"code")>

  <cfset THREAD.statusCode = THREAD.searchResults.code />

  <!--- IF CODE 401 THEN SESSION HAS EXPIRED --->

  <cfif THREAD.statusCode EQ 401>

  <!--- SET SESSION AS LOGGED OUT AND ATTEMPT SESSION REFRESH --->

  <cfset SESSION['#attributes.accountName#LoggedIn'] = 0 />

  <cfset THREAD.logInAccount = Application.cfcs.Login.logInAccount(attributes.email,attributes.password) />

  </cfif>

  <!--- ELSE SOMETHING ELSE HAS HAPPENED --->

  <cfelse>

  <cfset THREAD.statusCode = "Something Else - " & THREAD.searchResults.code />

  </cfif>

  <!--- IF RESULTS RETURNED --->

  <cfif THREAD.numResults GT 0>

  <!--- LOOP ROUND RESULTS AND CHECK IF MATCH BUYING CRITERIA --->

  <cfloop index="i" from="1" to="#THREAD.numResults#">

  <!--- ***SAFETY CHECK*** - ENSURE ID OF CURRENT CARD IS SAME AS ONE SEARCHING FOR --->

  <cfif THREAD.searchResults.auctionInfo.itemData.resourceID EQ attributes.resourceId AND THREAD.getMinBIN.resourceID EQ attributes.resourceId>

  <!--- ENSURE BIN PRICE SET AND IS LESS THAN SET BUYING PRICE --->

  <cfif THREAD.searchResults.auctionInfo.buyNowPrice GT 0 AND THREAD.searchResults.auctionInfo.buyNowPrice LTE THREAD.biddingPrice>

  <!--- SET AUCTION END TIME --->

  <cfset THREAD.timeLeft = THREAD.searchResults.auctionInfo.expires />

  <cfset THREAD.auctionEnds = DateAdd("s",THREAD.timeLeft,Now()) />

  <!--- BUY CARD --->

  <cfset THREAD.buyCard = Application.cfcs.Bid.doBIN(THREAD.searchResults.auctionInfo.tradeID,THREAD.searchResults.auctionInfo.buyNowPrice,THREAD.searchResults.auctionInfo.startingBid,THREAD.searchResults.auctionInfo.itemData.ID,THREAD.searchResults.auctionInfo.itemData.resourceID,THREAD.startPrice,THREAD.binPrice,THREAD.lowestBIN,THREAD.searchResults.auctionInfo.itemData.discardValue,THREAD.auctionEnds,THREAD.requestStart,THREAD.requestDuration) />

</cfif>

  </cfif>

  </cfloop>

  </cfif>

  <cfelse>

  <cfset THREAD.statusCode = THREAD.search.FileContent />

  </cfif>

  <cfset THREAD.sleepDuration = 1000 - THREAD.requestDuration />

  <cfif THREAD.sleepDuration GT 0><cfset Sleep(THREAD.sleepDuration) /></cfif>

  </cfif>

  <!--- INSERT SEARCH RECORD --->

  <cfset THREAD.insSearchRecord = Application.cfcs.Search.insSearchRecord(THREAD.definitionID,THREAD.statusCode,THREAD.requestDuration,THREAD.numResults,THREAD.biddingPrice) />

  </cfif>

  </cfloop>

  </cfif>

  </cfthread>

</cfloop>

I would have thought that the memory would have stayed around the same usage as each loop is performing the same set of actions so once the loop has went back to the start then I thought the previous loop would have been removed from memory (freeing up space) and then the same actions would be performed so the same memory total would then be used up but it seems almost as if each loop is being kept in memory and that is why it is growing.

Could someone please help me out and offer some guidance on how I could remedy this issue? If you need any more info then just let me know

Thanks in advance

Capture.JPG

JVM looks like this thanks to gcviewer:


The diagram does not show PermGen details. Tail of log says:

PSPermGen  total 1048576K, used 80172K object space 1048576K, 7% used.

While you may have something to resolve with the CFM code, for now to keep the system up perhaps you can do better to make some JVM adjustments. Keep in mind this is probably not a fix for the overall problem more try this to see if the system stays up while you continue to work on other CFM matters.

I expect JVM details are like this currently:


Minimum heap size: 1024
Maximum heap size: 2048
-XX:MaxPermSize=1024m

The overall system is:

CF10 on a Windows 7 x64 machine with an Intel core i3-2100 processor and 8GB RAM.


Seeing not many objects are maintained in PermGen you can make that smaller. Double heap sizes and set a value for the new part of heap. EG:

Minimum heap size: 2048
Maximum heap size: 4096
-XX:MaxPermSize=324m
-Xmn256m

Or this way if you prefer in CFadmin:

-server -Xmn256m -XX:MaxPermSize=324m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log

Or this way by editing JVM.CONFIG:

-server -Xms2048m -Xmx4096m -Xmn256m -XX:MaxPermSize=324m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log


For now it might be advisable to set JVM to perform full garbage collection every 10 minutes but as I post I am undecided so will post again more thoughts on that if it seems more like a good bandaid idea to me. I think stay with UseParallelGC because it tends to keep the heap evacuated rather than alter garbage collector to something that tends to maintain objects in memory.


Java is 7 so not EOL 6 tho fair to say 7u15 is old with 7u67 current. Java 8 is also release however no support statement from Adobe with running  CF10 on Java 8. Since you are not using a newer garbage collection algorithim like G1GC (and for now I do not recommend you do) I think stay with 7u15 for now.

HTH, Carl.

TOPICS
Server administration

Views

6.6K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Aug 18, 2014 0
Adobe Community Professional ,
Aug 19, 2014

Copy link to clipboard

Copied

You seem to be generating too many threads in a loop, each of them holding live objects. There is a high chance the heap of live objects that the loop generates keeps growing. The error message is telling you that the garbage collector has been unsuccessful in clearing up (because the objects are live).

For a start, I see no motivation for the use of cfthread and for the calls to sleep(). The page implicitly runs in a thread, the main Coldfusion execution thread. To monitor the number of requests per minute, use a counter in onRequestStart instead of sleep().

You should strip your code down to its bare bones. Something like this

<script>

  /* RELOAD PAGE EVERY 65 MINUTES */

  setTimeout(function(){

    window.location.reload(1);

  }, 3900000);

</script>

<!--- GET ACTIVE ACCOUNTS --->

<cfquery name="getLogins" datasource="myDB">

SELECT * FROM Logins WHERE active = 1

</cfquery>

<!--- LOOP THROUGH ACCOUNT --->

<cfloop query="getLogins">

    <cfset accountName=getLogins.accountName>

    <cfset email=getLogins.email>

    <cfset password=getLogins.password>

    <cfset resourceId=getLogins.resourceID>

    <!--- DEFAULT SESSION VARIABLES --->

    <cfset SESSION["#accountName#LoggedIn"] = 0 />

    <cfset SESSION["#accountName#LoginAttempts"] = 0 />

 

    <!--- WHILE ACCOUNT NOT LOGGED IN AND LESS THAN 8 LOGIN ATTEMPTS MADE --->

    <cfscript>

    while (SESSION['#accountName#LoggedIn'] EQ 0 AND SESSION['#accountName#LoginAttempts'] LT 8) {

 

        // ATTEMPT LOGIN

        logInAccount = Application.cfcs.Login.logInAccount(email,password);

     

        // IF LOGIN ATTEMPT UNSUCCESSFUL

        if (logInAccount EQ 0) {

            // INCREASE ATTEMPT COUNT

            SESSION['#accountName#LoginAttempts'] = SESSION['#accountName#LoginAttempts'] + 1;

        }

        // ELSE IF RETURNED VALUE IS 481 THEN ACCOUNT IS LOCKED

        else if (logInAccount EQ 481) {

            // SET LOGIN ATTEMPT COUNT TO STOP LOOP

            SESSION['#accountName#LoginAttempts'] = 8;

            // UPDATE ACCOUNT TO MARK AS LOCKED

            updLogin = Application.cfcs.Login.updLogin(email);

        }

    }

    </cfscript>

    <!--- IF ACCOUNT LOGGED IN --->

    <cfif SESSION['#accountName#LoggedIn'] EQ 1>

        <!--- SET ID FOR SEARCHING --->

        <cfset definitionID = resourceID - 1610612736 />

     

        <!--- WHILE ACCOUNT LOGGED IN --->

        <cfloop condition="SESSION['#accountName#LoggedIn'] EQUALS 1">

            <!--- GET LATEST LOWEST BUY NOW PRICE --->

            <cfquery name="getMinBIN" datasource="FUT" cachedWithin="#CreateTimeSpan(0,0,1,0)#">

            SELECT TOP 1 * FROM v_FUT14BINPrices WHERE resourceID = #resourceId# ORDER BY lastUpdated DESC

            </cfquery>

     

            <!--- INCLUDE FILE THAT CALCULATES BUYING AND SELLING PRICES --->

            <cfinclude template="sellingPrices.cfm" />

     

            <!--- IF BIDDING PRICE HAS BEEN SET --->

            <cfif StructKeyExists(SESSION,"biddingPrice")>

                <!--- MAKE SEARCH REQUEST, TIMING THE REQUEST --->

                <cfset requestStart = GetTickCount() />

                <cfset search = Application.cfcs.Search.dosearchOld(resourceId,biddingPrice,0) />

                <cfset requestDuration = GetTickCount() - requestStart />

         

                <!--- IF SEARCH CONTAINS FILE CONTENT  --->

                <cfif StructKeyExists(search,"FileContent")> 

                    <!--- DECLARE NUMBER OF RESULTS VARIABLE --->

                    <cfset numResults = 0 />

             

                    <!--- IF JSON RETURNED --->

                    <cfif IsJSON(search.FileContent)>

                        <!--- DESERIALIZE JSON --->

                        <cfset searchResults = DeserializeJSON(search.FileContent) />

                 

                        <!---  IF PLAYER SEARCH RETURNS AUCTIONINFO STRUCT --->

                        <cfif StructKeyExists(searchResults,"auctionInfo")>

                            <!--- SET NUMBER OF CARDS RETURNED FROM SEARCH --->

                            <cfset numResults = ArrayLen(searchResults.auctionInfo) />

                            <cfset statusCode = "Successful" />

                            <cfif numResults EQ 0>

                            <cfset statusCode = "Successful - No Results" />

                            </cfif>

                        <!--- ELSE IF ERROR CODE RETURNED --->

                        <cfelseif StructKeyExists(searchResults,"code")>

                            <cfset statusCode = searchResults.code />

                            <!--- IF CODE 401 THEN SESSION HAS EXPIRED --->

                            <cfif statusCode EQ 401>

                                <!--- SET SESSION AS LOGGED OUT AND ATTEMPT SESSION REFRESH --->

                                <cfset SESSION['#accountName#LoggedIn'] = 0 />

                                <cfset logInAccount = Application.cfcs.Login.logInAccount(email,password) />

                            </cfif>

                        <!--- ELSE SOMETHING ELSE HAS HAPPENED --->

                        <cfelse>

                            <cfset statusCode = "Something Else - " & searchResults.code />

                        </cfif>

                        <!--- IF RESULTS RETURNED --->

                        <cfif numResults GT 0>

                        <!--- LOOP ROUND RESULTS AND CHECK IF MATCH BUYING CRITERIA --->

                            <cfloop index="i" from="1" to="#numResults#">

                                <!--- ***SAFETY CHECK*** - ENSURE ID OF CURRENT CARD IS SAME AS ONE SEARCHING FOR --->

                                <cfif searchResults.auctionInfo.itemData.resourceID EQ resourceId AND getMinBIN.resourceID EQ resourceId>

                                    <!--- ENSURE BIN PRICE SET AND IS LESS THAN SET BUYING PRICE --->

                                    <cfif searchResults.auctionInfo.buyNowPrice GT 0 AND searchResults.auctionInfo.buyNowPrice LTE biddingPrice>

                                    <!--- SET AUCTION END TIME --->

                                    <cfset timeLeft = searchResults.auctionInfo.expires />

                                    <cfset auctionEnds = DateAdd("s",timeLeft,Now()) />

                                    <!--- BUY CARD --->

                                    <cfset buyCard = Application.cfcs.Bid.doBIN(searchResults.auctionInfo.tradeID,searchResul ts.auctionInfo.buyNowPrice,searchResults.auctionInfo.startingBid,searc hResults.auctionInfo.itemData.ID,searchResults.auctionInfo.itemData.resourceI D,startPrice,binPrice,lowestBIN,searchResults.auctionInfo.i temData.discardValue,auctionEnds,requestStart,requestDuration) />

                                    </cfif>

                                </cfif>

                            </cfloop>

                        </cfif>

                    <cfelse>

                        <cfset statusCode = search.FileContent />

                    </cfif>

                </cfif>

                <!--- INSERT SEARCH RECORD --->

                <cfset insSearchRecord = Application.cfcs.Search.insSearchRecord(definitionID,statusCode,requestDuration,numResults,biddingPrice) />

            </cfif>

        </cfloop>

    </cfif>

</cfloop>

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 19, 2014 0
New Here ,
Aug 25, 2014

Copy link to clipboard

Copied

Having held a conversation with BKBK and given them a deeper insight in to how my application is structured and works, the issue of the GC Overhead Limit being exceeded is still requiring tackled.

The functionality of the app works very well whilst there is memory available and the cfthreads are required to allow the accounts to search synchronously.

The question at hand here is how can I go about implementing a solution that ensures the JVM does not run out of memory?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 25, 2014 0
Adobe Community Professional ,
Aug 25, 2014

Copy link to clipboard

Copied

Yes, in our private mail discussions we weighed up alternative options. For example, using event gateways for asynchronous calls instead of cfthread, and websockets to dispatch notifications to users.

This would mean redesigning and rewriting the code, with no guarantee that that would solve the memory problem. We agreed that a better solution would be to find ways to optimize the current code.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 25, 2014 0
Adobe Community Professional ,
Aug 25, 2014

Copy link to clipboard

Copied

So, here we go with suggestions for optimizing the code.

1) It would help to 'var' local function variables in all the functions in all the component classes, i.e. in Application.cfcs.Login, Application.cfcs.Search and Application.cfcs.Bids.

2) Disable server monitoring and profiling in the Coldfusion Administrator.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 25, 2014 0
New Here ,
Aug 25, 2014

Copy link to clipboard

Copied

All local function variables are set as LOCAL scope variables, which is the same difference as var-ing the variables if I'm not mistaken?

Also, when server monitor is not running the JVM issue still occurs although I'll try and start capturing up-time so there's basis for comparison.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 25, 2014 0
Adobe Community Professional ,
Aug 25, 2014

Copy link to clipboard

Copied

<cffunction>

<cfset var x1=0>

</cffunction>

is what you should have for all local function variables. That is different from

<cffunction>

<cfset local.x2=0>

</cffunction>

I meant that you should disable server monitoring altogether.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 25, 2014 0
New Here ,
Aug 28, 2014

Copy link to clipboard

Copied

So.....

I've been monitoring how the app has been running and there's no real consistency with it. At first I thought BKBK had cracked it as I changed my locals to vars and disabled the monitoring and things seemed to be running smoothly but it seems to have went down around 4am this morning (running for around 12 hours).

I restarted the service when I got in at 9 and it ran for about an hour before I got the java heap space error and had to restart the service. I then started the application again and it was only running for 15 minutes before the java heap space error appeared and the service came to a standstill and most recently the application was running for the last hour before it ran out of memory.

Is there any other approaches I can try? I'm so exasperated with this as the application accomplishes its task so well whilst it's running!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
Adobe Community Professional ,
Aug 28, 2014

Copy link to clipboard

Copied

<!--- DEFAULT SESSION VARIABLES: this line writes the session variable within the thread --->

<cfset SESSION["#attributes.accountName#LoggedIn"] = 0 />

<!--- IF ACCOUNT LOGGED IN: this line is the first test of the value of the session variable within the thread --->

<cfif SESSION['#attributes.accountName#LoggedIn'] EQ 1>

Yet, nowhere is the value of the session variable updated.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
New Here ,
Aug 28, 2014

Copy link to clipboard

Copied

<!--- IF CODE 401 THEN SESSION HAS EXPIRED --->

<cfif statusCode EQ 401>

              <!--- SET SESSION AS LOGGED OUT AND ATTEMPT SESSION REFRESH --->

              <cfset SESSION['#accountName#LoggedIn'] = 0 />

              <cfset logInAccount = Application.cfcs.Login.logInAccount(email,password) />

</cfif>

Sure it does. The response of each search is checked and the session variable maintained accordingly.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
Adobe Community Professional ,
Aug 28, 2014

Copy link to clipboard

Copied

Not quite. The session key "#attributes.accountName#LoggedIn" is an attribute of the thread of a particular account name, and so is directly coupled to it. Whereas, '#accountName#LoggedIn' is ambiguous, and may even stand for '#getLogins.accountName#LoggedIn'.

Therefore we cannot assume them to be the same. The query runs in the main execution thread. Whereas each accountname thread runs asynchronously to the main thread.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
Adobe Community Professional ,
Aug 28, 2014

Copy link to clipboard

Copied

cpb07 wrote:

<!--- IF CODE 401 THEN SESSION HAS EXPIRED --->

<cfif statusCode EQ 401>

              <!--- SET SESSION AS LOGGED OUT AND ATTEMPT SESSION REFRESH --->

              <cfset SESSION['#accountName#LoggedIn'] = 0 />

              <cfset logInAccount = Application.cfcs.Login.logInAccount(email,password) />

</cfif>

Sure it does. The response of each search is checked and the session variable maintained accordingly.

In any case, that still fails to show us how the value of SESSION["#attributes.accountName#LoggedIn"] changes from 0 to 1.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
New Here ,
Aug 28, 2014

Copy link to clipboard

Copied

Apologies, I'd actually made a change to the initial code posted in this thread as follows

<cfif THREAD.statusCode EQ 401>

     <!--- SET SESSION AS LOGGED OUT AND ATTEMPT SESSION REFRESH --->

     <cfset SESSION['#attributes.accountName#LoggedIn'] = 0 />

     <cfset THREAD.loginRequired = 1 />

</cfif>

In terms of the value being changed, the call at the start of the thread to Application.cfcs.Login.logInAccount makes all the http requests and at the very end of that function, if the account has been logged in successfully, it sets the logged in session variable to true.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
Adobe Community Professional ,
Aug 28, 2014

Copy link to clipboard

Copied

That clarifies matters somewhat, but the point I raised still has to be addressed. Here it is again:

<!--- WHILE ACCOUNT NOT LOGGED IN AND LESS THAN 8 LOGIN ATTEMPTS MADE --->

  <cfscript>

  while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) {

/* The value of SESSION['#attributes.accountName#LoggedIn'] remains 0 in this code block */

  }

  </cfscript>

   <!--- IF ACCOUNT LOGGED IN: How so? Nowhere does this value change from 0 to 1 --->

  <cfif SESSION['#attributes.accountName#LoggedIn'] EQ 1>

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
New Here ,
Aug 28, 2014

Copy link to clipboard

Copied

It gets set to 1 in the logInAccount function in the Login CFC that gets called within the while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) { } block

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 28, 2014 0
Adobe Community Professional ,
Aug 29, 2014

Copy link to clipboard

Copied

So SESSION['#attributes.accountName#LoggedIn'] is updated away from view, in the function loginAccount. That's OK. But it raises an analysis issue.

The function knows nothing about the attributes scope (as this belongs to the context of the thread). As such, you should have passed attributes.accountName explicitly to the function. You currently pass it implicitly through attributes.email and attributes.password. In any case, such indirect relationships raise the complexity of the code. More on the use of the attributes scope in a moment.

This brings me to 3 questions, which are interrelated.

1) Suppose a logged-in user were to arrive at this CFM page. Then the following code would practically log him out:

<!--- DEFAULT SESSION VARIABLES --->

<cfset SESSION["#attributes.accountName#LoggedIn"] = 0 />

  <cfset SESSION["#attributes.accountName#LoginAttempts"] = 0 />

Was it perhaps your intention to use this instead:

<cfparam name="SESSION[attributes.accountName & 'LoggedIn']" default="0">

<cfparam name="SESSION[attributes.accountName & 'LoginAttempts']" default="0">

2) Using both the attributes and getLogins scopes seems like duplication. I can see no difference in purpose between thread variables attributes.X and the respective query variables getLogins.X. Why not just use the following version of the cfthread tag, and delete every occurrence of the attributes scope?  At least, the code will then be much simpler.

  <cfthread  name="#getLogins.accountName#"  action="run">

3) Your last remark apparently contains a serious design issue. Here it is again:

It gets set to 1 in the logInAccount function in the Login CFC that gets called within the while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) { } block

This is bound to have a serious impact on CPU resources. In fact, I wonder if it would run at all for any modest number of accounts.

Suppose that there are just 10 accounts and that they are all unauthenticated at the start. Then Coldfusion will immediately create 10 threads. They will possibly run simultaneously, indefinitely.

The while-block suggests that each such thread will only stop after the account owner logs in. That is, after human interaction.

However, by the time you raise your hand to scratch your head, Coldfusion will have run each while-block ten million times, say. By the time the first of the 10 users successfully logs in, Coldfusion will have run while-loops in the hundreds, or even thousands, of millions. That is a lot of processing power, most of it spent doing nothing.

The following test will show you what I mean. You will need a tool to display memory usage(for example, Sysstat tool or Top command for Linux, Process Explorer or Task Manager for Windows).

Run a CFM page containing the code,

<cfset x=0>

<cfset y=500000000>

<cfthread name="th_x" action="run">

<cfscript>

  while (x LT 500000000) {

x=x+1;

  }

  </cfscript>

</cfthread>

<cfthread name="th_y" action="run">

<cfscript>

  while (y GT 0) {

y=y-1;

  }

  </cfscript>

</cfthread>

<cfthread action="join" name="th_x,th_y" />

<cfscript>

writeoutput('x=' & x & " ; y=" & y);

</cfscript>

View the memory consumption while the code runs. It runs on my system (Win 7, 64-Bit, 4GB RAM, CF 11) for about 5 minutes, during which Coldfusion's CPU usage stays at around 90%.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 29, 2014 0
New Here ,
Aug 29, 2014

Copy link to clipboard

Copied

Your first point is absolutely spot on, I should default the session variables as a cfparam rather than a cfset, although this page theoretically only requires to ever be called once because once the page has been loaded then each account will spawn its own thread and the criteria within the while condition should ensure that the account is always active (even if the session expires then there is a call to log the account back in).

Point 2 probably stems from my inexperience with having cfthread in a cfloop, I'll update the code accordingly.

As for point 3, there is no human interaction with this application. It is fully automated. Once I load this page once I should then not have any more interaction with it. I leave it to get on with the task in hand. The majority of the time the account will log in during the first attempt so the while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) { } block will only loop once, for each account. And after all, the loop waits on the logInAccount function to complete before completing so it will not run any more than this?

Running the code you posted, the page completes in 48 seconds each time (3 attempts) and the current percent of CPU consumption by the Adobe ColdFusion Launcher Application (coldfusion.exe) never surpasses 50%. That's running CF10 on a Windows 7 x64 machine with an Intel core i3-2100 processor and 8GB RAM.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 29, 2014 0
Adobe Community Professional ,
Aug 29, 2014

Copy link to clipboard

Copied

cpb07 wrote:

As for point 3, there is no human interaction with this application. It is fully automated.

Ah, that is a surprise!

The majority of the time the account will log in during the first attempt so the while (SESSION['#attributes.accountName#LoggedIn'] EQ 0 AND SESSION['#attributes.accountName#LoginAttempts'] LT 8) { } block will only loop once

Certainly makes for a workable design, assuming that there are no hold-ups in the logInAccount function. Under which conditions will 2 or more attempts be required?

Running the code you posted, the page completes in 48 seconds each time (3 attempts) and the current percent of CPU consumption by the Adobe ColdFusion Launcher Application (coldfusion.exe) never surpasses 50%. That's running CF10 on a Windows 7 x64 machine with an Intel core i3-2100 processor and 8GB RAM.

Judging by your times, you might have run the version with 100 000 000 loops that I posted earlier. In a later, more representative test, I used 500 000 000, and corrected my post accordingly.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 29, 2014 0
New Here ,
Aug 30, 2014

Copy link to clipboard

Copied

Sometimes the host server returns a message saying that it has reached its max connections so it sometimes takes several attempts until a successful login is achieved, although this isn't particularly common. The only time that an account won't log in at all is if the host server is down or if the account has been banned (usually for continually exceeding the request threshold). Having the login attempt set at 8 is probably a bit overkill.

Yeah it was the version with the 100,000,000 loops that I had tried, I've just ran the one with 500,000,000 loops and again the ColdFusion CPU consumption stays around the 50% mark with the overall CPU usage sitting around 59%.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 30, 2014 0
Adobe Community Professional ,
Aug 30, 2014

Copy link to clipboard

Copied

Thanks for the explanation. Your result,

I've just ran the one with 500,000,000 loops and again the ColdFusion CPU consumption stays around the 50% mark with the overall CPU usage sitting around 59%.

seems more like it. It shows us the astonishingly high memory consumption of two loops that do practically nothing besides looping. If each was to create objects, the garbage collector might not have the opportunity of cleaning up before the overhead limit is reached.

The moral is clear. We should avoid stagnation wherever possible. A function that fails to return, for example, might hold objects that cannot then be garbage-collected.

Sometimes the host server returns a message saying that it has reached its max connections so it sometimes takes several attempts until a successful login is achieved, although this isn't particularly common. The only time that an account won't log in at all is if the host server is down or if the account has been banned (usually for continually exceeding the request threshold). Having the login attempt set at 8 is probably a bit overkill.

That sounds good. It is crucial for the loops to keep going, without obstruction, and to terminate quickly in case of login failure.

By the way, how many accounts are there in your database? I ask because I have just noticed parts of the code that may cause concurrency issues.

  <!--- INCLUDE FILE THAT CALCULATES BUYING AND SELLING PRICES --->

  <cfinclude template="sellingPrices.cfm" />

I doubt that a thread can do an include like that into the current page. If it can, it will lead to a shambles. Since the threads are generated asynchronously, they run simultaneously, and in no particular order. If the include were possible, the different accounts will each include sellingPrices.cfm, in unpredictable order.

The solution is to copy the content of sellingPrices.cfm to this particular point in the code.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 30, 2014 0
New Here ,
Aug 31, 2014

Copy link to clipboard

Copied

At the moment there are about 70 (although this is likely to increase dramatically over time). In the included sellingPrices.cfm I have set all variables in the thread scope and this seems to work fine (as I keep a record of each item request and all the prices match up the way they should). There is not a great deal amount of code in that page so I shall move it in to the main thread code.

Is it better to avoid focusing on settings in the JVM at the moment as we're trying to eradicate the issue that is causing the overhead limit issue, rather than postponing the inevitable?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 31, 2014 0
Adobe Community Professional ,
Aug 31, 2014

Copy link to clipboard

Copied

The focus is not on JVM settings. It is on detecting where the application might be creating and accumulating live objects.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 31, 2014 0
New Here ,
Aug 31, 2014

Copy link to clipboard

Copied

I didn't mean that we were focusing on that at the present, I was just inquiring whether I should ignore exploring different JVM settings.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Aug 31, 2014 0
Adobe Community Professional ,
Sep 01, 2014

Copy link to clipboard

Copied

You actually brought up a good topic. We should have explored your JVM settings, if only to rule them out.

So, what are your JVM settings - particularly maximum heap (Xmx), minimum heap (Xms) and XX:MaxPermSize?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 01, 2014 0
New Here ,
Sep 01, 2014

Copy link to clipboard

Copied

I had tinkered with them previously but at present they are as follows:

-server -XX:MaxPermSize=192m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log

Minimum and maximum JVM Heap Sizes are both set to 2048mb.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 01, 2014 0
Adobe Community Professional ,
Sep 01, 2014

Copy link to clipboard

Copied

Ah, thanks. I would experiment with

Minimum heap size: 1024

Maximum heap size: 2048

-XX:MaxPermSize=1024m

I suspect that -XX:MaxPermSize:192m is too small for your purposes.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 01, 2014 0
New Here ,
Sep 01, 2014

Copy link to clipboard

Copied

Okay, thanks. Just restarting the service so the changes take effect and will keep you updated.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 01, 2014 0
New Here ,
Sep 01, 2014

Copy link to clipboard

Copied

Okay...so that's ColdFusion just grind to a halt after 4 hours (which has been an improvement on recent up-time) and from keeping an eye on the resource monitor in task manager, the ColdFusion CPU consumption was sitting below 20% but it is currently at 85%+ (as I've not stopped the service yet) so I'm wondering if it is just a gradual filling up of the memory (although I'd have expected the CPU consumption to gradually increase) or if there is something going wrong and demanding all the memory suddenly until such point as there is no more free memory.

I'll restart the service and keep an eye on the resource monitor again and see if there is sudden spike in CPU consumption.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 01, 2014 0
Guide ,
Sep 01, 2014

Copy link to clipboard

Copied

Hello,

I see thread has moved to discussing JVM arguments so thought I would join in as I refrain from commenting on CFM syntax.

What Java version is CF10 using? Early CF10 installer applied Java 6 (which is Oracle EOL) and latter installer used Java 7.

I see Java logging is enabled so perhaps the details in your last cfjvmGC.log would be interesting. Can you ZIP and host that somewhere so I can download and read log?

Regards, Carl.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 01, 2014 0
New Here ,
Sep 02, 2014

Copy link to clipboard

Copied

Progress report...

The service has ran for 3-4 hours on three occasions, the other occasion I ran it the GC overhead limit reached message was encountered after only the 10th account logged in. I'm starting to think that rather than the memory filling up gradually (although increasing the MaxPermSize seems to have helped greatly) that eventually a request is getting stuck and this is causing all the subsequent requests to throttle the memory. Sound plausable?

Carl, I've uploaded my last cfjvmGC.log which can be found at http://www.filedropper.com/cfjvmgc

EDIT: The service just ran for 20 minutes before the GC overhead limit exceed message appeared so that would suggest that a request is getting stuck (for lack of a better term).

Now, with the inclusion of more accounts I have a cfsetting variable at the very start of the file that sets the request timeout to 600 seconds (10 minutes - at present it takes around 6-8 minutes to log all accounts in but the more accounts added the longer this will take obviously).

Could it be that because this overwrites the default CFAdmin timeout setting of 60 seconds that for some reason, quite randomly, a cfhttp or cfquery request is not returning a response in a timely manner and this is causing any subsequent requests to have to wait for this request to finish and this is throttling the memory?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 02, 2014 0
Adobe Community Professional ,
Sep 02, 2014

Copy link to clipboard

Copied

cpb07 wrote:

I'm starting to think that rather than the memory filling up gradually ... that eventually a request is getting stuck and this is causing all the subsequent requests to throttle the memory. Sound plausable?

My instincts exactly. There is filling up all right, and not gradually. Should there be talk of 'subsequent requests', in plural? From what I understand, there is just the one request, albeit one that spawns 70+ threads.

EDIT: The service just ran for 20 minutes before the GC overhead limit exceed message appeared so that would suggest that a request is getting stuck (for lack of a better term).

When you consider the 20 minutes together with the line <cfset Sleep(30000) />, it tells you the code never made it past 40 threads. [20 minutes = 1200 seconds. Coldfusion sleeps for 30000 ms, that is, 30 seconds during creation of subsequent threads. 1200 / 30 = 40 ]

Now, I'd mention to BKBK in a discussion we had that there is a scheduled task that runs every 10 minutes that basically gets a list of all the items I'm trading and gets the current lowest buy now price for each. This can often take anywhere between 4-8 minutes to complete so I have a cfsetting variable in the .cfm file that sets the request timeout to 600 seconds (10 minutes - so that even if the task has not completed, it will start again as the lowest buy now price data will now be out of date).

Could it be that because this overwrites the default CFAdmin timeout setting of 60 seconds that for some reason, quite randomly, a cfhttp or cfquery request is not returning a response in a timely manner and this is causing any subsequent requests to have to wait for this request to finish and this is throttling the memory?

To answer these questions, it might help to have a closer look into the threads. Suppose you wish to know how they stand after, say, 30 minutes.

You could then include an extra thread, named sleeper30min, say, that does nothing but wait for 30 minutes. Next, join the main page execution thread to sleeper30min.

At this point, dump all the threads.

It goes something like this:

<cfloop query="getLogins">

   <!--- HAVE A SLEEP SO IP DOESN'T GET FLAGGED FOR SENDING TOO MANY REQUESTS AT ONCE --->

  <cfset Sleep(30000) />

  <!--- CREATE THREAD FOR ACCOUNT --->

  <cfthread ...>

etc.

</cfthread>

<!--- Join page thread to thread sleeper30min --->

<cfthread action="join" name="sleeper30min" timeout="1800000" />

<cfif sleeper30min.STATUS is "TERMINATED">

<!--- Get out of loop --->

<cfbreak>

</cfif>

</cfloop>

<cfif sleeper30min.STATUS is "TERMINATED">

<cfloop query="getLogins">

  <cfset thread_name = getLogins.accountName>

   <cfdump label="Thread #thread_name#" var="#evaluate(thread_name)#">

</cfloop>

</cfif>

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 02, 2014 0
Guide ,
Sep 02, 2014

Copy link to clipboard

Copied

You will need to make a copy of cfjvmGC.log since when CF starts a new log is created and existing log with the memory details and garbage collection information lost. The log you attached has 1 full garbage collection at 29 seconds for 0.03
seconds duration with just over 2 minutes of up or log time.


Be sure to save your next log when fail occurs and host that.

HTH, Carl.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 02, 2014 0
Guide ,
Sep 02, 2014

Copy link to clipboard

Copied

As for finding out what Java version is in use, in CFadmin select System Information i button. What are the details EG:

JVM Details 

Java Version  1.7.0_51   

Java Vendor  Oracle Corporation   

Java Vendor URL  http://java.oracle.com/ 

Java Home  D:\Program Files\Java\jdk1.7.0_51\jre   

Java VM Specification Version  1.7   

Java VM Version  24.51-b03   

Java VM Name  Java HotSpot(TM) 64-Bit Server VM

Regards, Carl.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 02, 2014 0
New Here ,
Sep 03, 2014

Copy link to clipboard

Copied

Carl, had a fail this morning and the log can be found at http://www.filedropper.com/cfjvmgc_1. I'd forgot you'd asked about Java version, details are as follows:

Java Version1.7.0_15 
Java VendorOracle Corporation 
Java Vendor URLhttp://java.oracle.com/ 
Java HomeC:\Services\web\jre 
Java VM Specification Version1.7 
Java VM Version23.7-b01 
Java VM NameJava HotSpot(TM) 64-Bit Server VM 

BKBK, I'd reduced the sleep time to 5 seconds so all accounts log in okay. I'll implement the suggestion you made and let you know what I find.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 03, 2014 0
New Here ,
Sep 03, 2014

Copy link to clipboard

Copied

Not even getting 30 minutes up-time anymore, memory threshold seems to be getting exceeded after 10-15 minutes for whatever reason...grrrrrrr! Will report back further.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 03, 2014 0
Guide ,
Sep 03, 2014

Copy link to clipboard

Copied

Capture.JPG

JVM looks like this thanks to gcviewer:


The diagram does not show PermGen details. Tail of log says:

PSPermGen  total 1048576K, used 80172K object space 1048576K, 7% used.

While you may have something to resolve with the CFM code, for now to keep the system up perhaps you can do better to make some JVM adjustments. Keep in mind this is probably not a fix for the overall problem more try this to see if the system stays up while you continue to work on other CFM matters.

I expect JVM details are like this currently:


Minimum heap size: 1024
Maximum heap size: 2048
-XX:MaxPermSize=1024m

The overall system is:

CF10 on a Windows 7 x64 machine with an Intel core i3-2100 processor and 8GB RAM.


Seeing not many objects are maintained in PermGen you can make that smaller. Double heap sizes and set a value for the new part of heap. EG:

Minimum heap size: 2048
Maximum heap size: 4096
-XX:MaxPermSize=324m
-Xmn256m

Or this way if you prefer in CFadmin:

-server -Xmn256m -XX:MaxPermSize=324m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log

Or this way by editing JVM.CONFIG:

-server -Xms2048m -Xmx4096m -Xmn256m -XX:MaxPermSize=324m -XX:PermSize=192m -XX:+UseParallelGC -Xbatch -Dcoldfusion.home={application.home} -Dcoldfusion.rootDir={application.home} -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -verbose:gc -Xloggc:cfjvmGC.log


For now it might be advisable to set JVM to perform full garbage collection every 10 minutes but as I post I am undecided so will post again more thoughts on that if it seems more like a good bandaid idea to me. I think stay with UseParallelGC because it tends to keep the heap evacuated rather than alter garbage collector to something that tends to maintain objects in memory.


Java is 7 so not EOL 6 tho fair to say 7u15 is old with 7u67 current. Java 8 is also release however no support statement from Adobe with running  CF10 on Java 8. Since you are not using a newer garbage collection algorithim like G1GC (and for now I do not recommend you do) I think stay with 7u15 for now.

HTH, Carl.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 03, 2014 0
New Here ,
Sep 05, 2014

Copy link to clipboard

Copied

I added the sleeper thread that you suggested and the information it gives me on the thread is elapsed time, name, output (which is empty), priority, start time and status. Were you expecting there to be more information?

I check the error logs whenever ColdFusion is grinding to a halt and cannot see any other errors than the GC overhead limit exceeded one, but cfhttp requests are taking 30-40 seconds to return rather than a few hundred milliseconds.

Would anyone be willing to cast their eye over the application on their own development environment to see if they A) encounter the same issue and B) see if they can find what the cause is?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 05, 2014 0
New Here ,
Sep 05, 2014

Copy link to clipboard

Copied

Carl

I implemented the suggestions you made but unfortunately this does not seem to have had any impact on the availability of the application, for the last couple days I've never had longer than 25-30 minutes up-time

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 05, 2014 0
Adobe Community Professional ,
Sep 05, 2014

Copy link to clipboard

Copied

At this point, the key values we wish to know are those of status. You want to know which threads have run completely (status=COMPLETED), which were prevented from running (status=TERMINATED) and which have yet to run (status=NOT_STARTED).

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 05, 2014 0
cpb071 LATEST
New Here ,
Dec 31, 2014

Copy link to clipboard

Copied

I assume these are found in a thread dump?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Dec 31, 2014 0