Skip to main content
I
impalerware
Participating Frequently
July 26, 2023
Answered

Long running scheduled tasks kicking off multiple times

  • July 26, 2023
  • 3 replies
  • 1039 views

Anyone else have an issue with 2021 and long running scheduled task running multiple times?

I've narrowed it down to only occurring using a JDK higher than 11.0.12. 

 

I've been replicating this by scheduling a task to a page with the following content:

 

<cfset sleep(360000)>

<cfmail to="youremail@email.edu" from="testtask@email.edu" subject="test task" type="html">
	<cfdump var='#timeFormat(now(), "HH:mm:ss")#'/>
</cfmail>

 

    Correct answer pichardov

    I have looked into the issue some more, and continue to do so. As far as Java changes go, I have been unable to find any explicit acknowledgement from Oracle or OpenJDK saying something like, “We changed ScheduledThreadPoolExecutor behavior in 11.0.13+". Therefore, the behaviour you observed (ColdFusion scheduled tasks overlapping under JDK 11.0.13 or newer) seems to be an undocumented runtime behavior change rather than a documented API change. The research continues.


    I did a lot of troubleshooting and finally narrowed it down and have a workaround. 

     

    We offload SSL termination to a F5 load balancer. The issue occurs when the scheduled task url hits the F5 first. So going F5 -> IIS -> CF. Still not sure why this only happens with newer Java though.

     

    Since we have a lot of scheduled tasks and I didn't want to update all of them, I hardcoded the IP of the server running CF in its hosts file. So all http requests initiated on the CF server itself will go to itself vs going over to the F5 first.

     

    I did have to create a self signed IIS cert with the same name as the cert on the F5 and trust it into the java keystore. Initially I tried a self signed cert with the hostname of the CF server but for whatever reason, that fails the ssl check even when trusted into the java keystore.

    3 replies

    D
    default8rs3k49y1nan
    Participating Frequently
    September 15, 2025

    I've ran into this issue in the past with long running scheduled tasks and the resolution i've found is wrapping the whole task in a <cfthread> and right after that cfthread open tag <cfouput>any text</cfoutput>. I'm not sure of all the mechanics that make that work, but seems like since scheduler is executing the "task" like it's a page request using a http request,  and that it needs confirmation that the http request resolved and if it doesn't then it trys again to resolve thinking there was a network issue along the way. If you run the task without the cfthread tag by putting the URL in a browser and execute it the page just spins and spins, but if you do the same with the cfthread and the output it resolves the page immediately while the work of the task is still being done behind the scenes. 

    P
    paule12345
    Inspiring
    August 11, 2025

    We observed this behavior on CF2021 when doing bulk email sending, no scheduled task involved.  When we would send, say, a 10,000 email blast via the SMTP service on the local Windows 2019 server,  after X number of minutes the CFM doing the email sending would start running again.   I could see it happening in FusionReactor monitor in real time.   Our solution at the time was running smaller email batches, say, no more than 5000 at a time.   We now use a third party cloud based email sender (SMTP2GO) and don't have the issue anymore because their system processes each SMTP transaction much faster.

    Charlie Arehart
    Community Expert
    Charlie ArehartCommunity Expert
    Community Expert
    August 11, 2025

    @paule12345, your situation sure sounds quite different that what @pichardov had experienced. In fact, your comment here arrived at the very same minute as theirs, offering their workaround.

     

    As for your saying that a cfm page that created mails started running again, if you are thinking there's a connection there, I'll note that the process in cfmail will by default simply put emails into the cf mail spool folder. Cfmail itself doesn't talk to the smtp server.

     

    Instead it's cf itself (a background thread) that wakes up (by default every 15 seconds) to find any such spooled mail to be delivered, and THAT'S what talks to the smtp server. Any failure or hangup in that process would NEVER (itself) cause the cf page doing the cfmails to run again.

     

    It's great to hear you have FR. With that, you should definetky) definitely be able to track down the "repeat offender", finding such details as what ip address kicked it off, what client user agent that was, what web server processed it, what headers came in from the client or web server, and more. 

    /Charlie (troubleshooter, carehart. org)
    P
    pichardov
    Participating Frequently
    August 5, 2025

    We just upgraded to 2023 and this is still happening.

    Charlie Arehart
    Community Expert
    Charlie ArehartCommunity Expert
    Community Expert
    August 5, 2025

    Well, I have not succeeded in replicating that test that the OP offered. As such, whatever it is that you and ther are experiencing (even if you may find others confirming it), it would seem there must be something environmental affecting it.

     

    So let's talk about a few things:

    • first, how are you tracking that it runs more than once? The OP showed sending an email. One could also add cflog or the like, or some people judge based on database updates done by the task
    • second, can you do anything to prove that the task is running twice from that one CF instance setup for the sceduled task? I have helped some people find out that they unepectedly had scheduled tasks on more than one machine or instance of CF, calling the same URL (and thus fooling them)
    • if you have access to the web server that is processing the page being called as the url, do you have any access logs for that web server which do clearly show the request running twice?
    • do any CF logs on the machine kicking off the scheduled task show it running more than once (at the time expected)?
    • When you do feel they run more than once" per task, do you have evidence that tracks the timing of that second run? Is it some set number of seconds after the first one?
    • If you have FusionReactor, it of course can track the url called by the task if it's a CF page, but FR does ALSO track the scheduled task processing (as what's called a "transaction", of type "httpclient", for those familiar with using FR).

     

    Some more background: since CF 10 the cf scheduled tasks run under the covers based on an open-source library called quartz (with config files in the cfusion/quartz folder). And for some number of releases, there has indeed been a problem were the default timeout for tasks (in QUARTZ) is 60, which means the task gets kicked off (and runs, wherever the URL says to run--and it runs as long as it needs to run), but the quartz task itself "fails"--as is logged in CF's scheduler.log (if you have enabled logging of scheduled tasks, in the CF Admin logging settings page).

     

    I am working up a blog post on how to enable additional logging from the quartz perspective. That too can help in better assessing if it's true (or not) that this quartz scheduler in CF is responsible for this seeming double-execution of tasks that you guys are observing.

     

    There may well be another known reason for that seemnig to happen, and maybe someone else will chime in with more thoughts. I wanted to offer the above in the meantime.

    /Charlie (troubleshooter, carehart. org)
    P
    pichardov
    Participating Frequently
    August 6, 2025

    Hi Charlie, I'm actually the OP as well. My username changed for some reason since my initial post.

     

    We've been stuck on CF21 with java 11.0.2 this entire time. Any newer version of java triggers this behavior, no matter what jdk varient (oracle, amazon, adoptium). Now that CF21 is getting close to eol, I've upgraded our test server to CF23 and got the same behavior.

     

    I've since modified the sample code:

    <cfset starttime = '#timeFormat(now(), "HH:mm:ss")#'>
<cfset sleep(360000)>

Start time : <cfoutput>#starttime#</cfoutput> <br>
End time: <cfdump var='#timeFormat(now(), "HH:mm:ss")#'/>

<cfmail to="testuser@email.com" from="testtask@email.com" subject="test task" type="html">
	Start time : <cfoutput>#starttime#</cfoutput> <br>
	End time: <cfdump var='#timeFormat(now(), "HH:mm:ss")#'/> <br>
	<cfdump var="#cgi.CERT_SERVER_ISSUER#">
</cfmail>

    Running this page via task schedule once, produces 3 emails. each one has a start time of 3 minutes after the previous and the end time is as expected with 6 minutes after the start time. The coldfusion schedule log shows the task kicking off once. I did try changing the default task retry to 0 instead 3 with no luck. IIS access logs has the page getting hit only once also.

     

    I was going to try setting up a linux environment to see if it happens there

     

     

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices