Skip to main content
B
Bardnet
Inspiring
December 10, 2020
Question

"Phantom" calls to long running scripts (connector issue)

  • December 10, 2020
  • 4 replies
  • 793 views

Hi,

 

I find myself in a ColdFusion 2018 environment on a Linux server behind an Apache server.

I had issues and tried to track them down. It turned out my long running script is called over and over again.

I call it manually through a webbrowser once. I see it initiates only one request through the browser's F12 tool.

After I realized that requests multiply I searched the internet for possible solutions and found this thread on this site: https://coldfusion.adobe.com/2019/02/apache-crashes-serving-multiple-thread-requests-using-cf2018/

 

I eventually set heartbeat_interval to 0 and noticed no change. Only after httpd was restarted the "Segmentation fault" messages stopped.

 

Now I start my long running script and enter some data into a database when it is started, including a timestamp. This is how I see that the script is started again and again every 6 minutes.

I guess this is still a connector issue.

Does this behaviour ring a bell? I like to believe some instance (connector, httpd) sees no response from CF's Tomcat and repeats the request. Is there such a timeout? The Request Timeout in CFAdmin is disabled - the checkbox is unchecked.

 

Best,

Bernhard

    This topic has been closed for replies.

    4 replies

    P
    paule12345
    Inspiring
    December 18, 2020

    We've observed the same behavior on our CF2016 Enterprise install on Windows 2012.   We see CF scripts that take longer than 6 minutes spawn a duplicate thread.   I can see it in FusionReactor, at approx 377,000 ms, a duplicate thread will fire up.   cfsetting requesttimeout has no effect.   Timeout settings in CF admin have no effect.   IIS app pool recycling is not causing it.  This is not a scheduled task.

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    December 19, 2020

    That this has been observed on CF2018+Linux+Apache and CF2016+Windows+IIS suggests an issue in the ColdFusion engine. So I wonder whether we can reproduce it. For example, using the following code:

     

    <cfscript>
        // This request will run for 9 minutes
	setting requestTimeout=540;
	sleep(540000);
</cfscript>

     

    Steps:

    1) Run the above code in a CFM page.

    2) Wait 6 minutes. Then use the jstack tool of the Java that ColdFusion is running on to take ColdFusion's thread-dump.

    For example, my ColdFusion installation runs on C:\Program Files\Java\jdk-11.0.8, as Windows process 22816. So, I navigate in DOS to C:\Program Files\Java\jdk-11.0.8\bin, and take thread-dumps using the following DOS command (the dump is saved as a text file on my desktop):

     

    jstack -l 22816 > C:\Users\BKBK\Desktop\CF_threadDump_1.txt

     

    3) Continue taking thread-dumps every 20 to 30 seconds, renaming subsequent files CF_threadDump_2.txt,  CF_threadDump_3.txt, ... till the end of the request.

    4) Study the dumps to identify any additional or extraneous threads. 

     

     

    P
    paule12345
    Inspiring
    December 19, 2020

    BKBK, I tried your code today and the script ran to completion.  It did not spawn a duplicate thread after 6 minutes.   Something else in our code must be causing it.  I'll have to do some more testing with our actual production code.

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    December 13, 2020

    Any error messages in coldfusion-error.log and coldfusion-out.log?

    B
    BardnetAuthor
    Inspiring
    December 17, 2020

    Sadly, the log files are not helpful.

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    December 11, 2020

    Are you running any scheduled tasks?

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    December 11, 2020

    How long does your long-running script run for? Let's say 15 minutes.

     

    Then you could try the following. In the Administrator, go to Server Settings > Settings and set Timeout Requests after seconds to 1200.
    Then proceed to Server Settings > Request Tuning and set Queue Timeout Settings to 1200.

    Restart ColdFusion, just to be sure the change kicks in.

    Any joy?

      B
      BardnetAuthor
      Inspiring
      December 17, 2020

      Thanks alot for your pointers! I  try this setting.

      As a quick fix I tried to identify if the script is already running and cancel every other request.

      Still looking for the issue, bud I could limit the negative effects.

      Terms & ConditionsAccessibility statement
      Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices