Scheduled task issue error code 1010

[paulm1677332]

Hello,

Have an issue running a scheduled task on CF2018. Hitting the url in a browser window it runs fine, but via the schedule task when I output to a file all im getting in the file is error code: 1010

Not seeing anything wrong in execption.log or scheduler.log. Anyone any ideas?

Thanks

[Priyank Shrivastava.]

Hi @paulm1677332 

It is happening with all the scheduled tasks or one particular task. Could you please confirm it? For testing, create a test task with a simple webpage and then run it as a Scheduled task.

[paulm1677332]

Hi Priyank,

Its just for this task. We only have 2 but the other one is working fine. I edited the page just to show the date, nothing else and still the error code 1010 so it isnt anything within a component or function. 

Thanks

[Priyank Shrivastava.]

Thanks, could you please let me know what code that you have in CFM, let me quickly try and run it on my end.

[paulm1677332]

I have a lot of component calls etc. so cant post that here but like i said earlier when I commented all that out and just had the below I also got same error code 1010

<cfoutput>
			Complete #Now()#
</cfoutput>

I see that error code 1010 has some relevance in Cloudflare and we do use Cloudflare so im wondering if it has something to do with that. Cloudflare reference 

[Priyank Shrivastava.]

Thanks, let me try and get back to you.

[Charlie Arehart · Accepted solution]

Yes, this is about Cloudflare blocking the request for the URL being called in the scheduled task, because of its "Browser Integrity Check" feature.

If that URL is going to your own site, then you can easily solve it in your CloudFlare configuration. If the URL goes to someone else's site, then THEY have to solve this in THEIR setup of CloudFlare and there is NOTHING you can do. (I write that as much for future readers of this post.) 

Let me explain the problem, then the solution (or skip ahead to "how to solve the problem" if you feel you must).

1) What's happening, and how to prove it

The issue is that when you run a CF scheduled task, it sends the request (whatever URL is set in the scheduled task ) with a user-agent header of "CFSCHEDULE". And Cloudflare's "Browser Integrity Check" feature is only allowing user agents it recognizes, and blocking requests whose agent it does not recognize. Currently, it's blocking the user-agent of CFSCHEDULE.

FWIW, it is NOT currently blocking the user-agent of "ColdFusion", which is what is sent by default when you run a CFHTTP request. In fact, if you tried that, you should find that it WOULD be allowed.

In fact, you can confirm the problem from the command line (of linux or Windows) by using CURL to test the page you are wanting to call (in the task) and indicating the user agent. 

curl -A "CFSCHEDULE" -X GET https://domaintotest

If you visit a site that is being BLOCKED in Cloudflare this way, the response will be:

error code: 1010

If the site is NOT being blocked, the command would display the expected response from the page.

You could even change the useragent value (in the CURL command above) to be "firefox" and it should work (that works for a request that would be blocked with cfschedule. In fact, you can use "https://www.cloudflare.com" as the URL to get in the CURL, and experience what I am talking about.

2) Options for solving the problem

So let's talk about solving it, though first I want to note that this is not a problem unique to ColdFusion, in that if the user agent of a call (routed through CloudFlare) is not one that CloudFlare recognizes, that "Browser Integrity Check" feature will block it, by default.  In fact, you will find people complaining that CloudFlare has blocked calls from other "legitimate" user agents, such as those from calls from payment processors and other sorts of automated agents.

And you may well find blog posts or forum threads, or indeed the Cloudflare docs, where most discussions of this problem suggest that the "solution" is to disable this "Browser Integrity Check" feature. I'll show how to do that, but it's throwing the baby out with the bathwater, when they leave it at that (as most do) and makes your site accessible from "any" useragent.

Instead, I'll show how you can take just one more step to simply allow CFSCHEDULE as an allowed user agent in Cloudflare.

Both can be done with a free Cloudflare account, by the way.

And while you might think you could solve this with a Cloudflare WAF firewall rule (which even lets you configure the rule as being based on user-agent), I confirmed that a Cloudflare WAF firewall did not work. Here is a) a simpler and b) a more complete solution.

Option a)

Go to your site (in CloudFlare), then choose "Security" on the left, then "Settings" within that (/security/settings on the cloudflare URL for your site), then on that page scroll down to "Browser Integrity Check". There you could, again, disable the feature with the checkmark offered there.

But again that would remove the feature ENTIRELY from protecting your site, allowing ALL unrecognized useragent values to access your site, for better or worse.

Option b)

Instead, note that under that setting (in Cloudflare, select a site and then choose "Security">"Settings">"Browser Integrity Check") there is a link offered right under that feature which allows you to "Create a configuration rule". That will open a page (/rules/configuration-rules) where you can allow SPECIFICALLY the CFSCHEDULE user agent  to work (as opposed to turning off the feature entirely as above).

On that "Create new Configuration Rule" page (/rules/configuration-rules/new):

• give the rule a name like "allow cfschedule"
• then choose the option, "Custom filter expression" (instead of "all incoming requests")
• in the "field" drop-down choose "user agent"
• set "operator" to "equals"
• type CFSCHEDULE in the "value" field
• Then scroll down the list of things you can "control" to find that "Browser Integrity Check" option and click the "add" option to its right. This will enable the rule to control that feature, and note that it will show as being unchecked (off) by default (which is what we want, thus disabling the browser integrity check feature for this rule)
• Don't forget to scroll to the bottom and click "deploy" .

Now test your scheduled task, or test that CURL command with CFSCHEDULE as the user agent, and it should work.

You could of course go up to the top of the rule configuration page and add still other controls, if you wanted to limit WHO was allowed to make a call using cfschedule as a user agent (limiting by ip address, and more), but that may not be necessary for most folks (and legit ip addresses can change, of course). 

A hoped-for better option

Finally, we can also hope that someday perhaps CloudFlare will make the user agent "CFSCHEDULE" be trusted/allowed by default, just like they do now for "ColdFusion" (which allows through calls that come from CFHTTP).

Hope that solves the problem for you. Let us know. (I will turn this into a blog post as well, so I welcome feedback on the discussion above.)

[paulm1677332]

Hi Charlie,

Thanks for that comprehensive explanation. The Cloudflare route was the only plausible issue as I couldnt find anything similiar for anyone else out there. I was thinking that it may be in the way cfschedule calls the url under the hood but wouldnt have had any idea how to dig any deeper there. I will make the requested change tomorrow and add a follow up just to confirm. The url is on our own servers so we shouldnt have an issue making the required changes. 

Thanks again

[paulm1677332]

Quick update. We made the change above Option 2, but it stopped all traffic to our sites so we had to revert. We may have configured it wrong but we will test it more next week and will add a follow up here.

Thanks

[Charlie Arehart]

That's indeed odd. I'd done the same to my own cloudflare account before suggesting the solution, and my site was not blocked otherwise, then or now.

So please clarify: did you add ONLY that one rule with that one allow for cfschedule? That's all I did, to solve the problem (and not cause another).

While you wait to try again/investigate, there's a workaround: create a page to do a cfhttp to the url in the task, and change the task to call that page doing the cfhttp. While I'm not aware of any way to change the user agent for sched tasks to be other than "cfschedule". Recall that I'd said cfhttp calls use "ColdFusion" as the user agent and that IS NOT BLOCKED by cloudflare. (BTW, a cfhttp can also set the user-agent header to be anything else. I'm not aware of any way to change the user agent for sched tasks to be other than "cfschedule".)

Let us know if you try that. 

[paulm1677332]

Hello there,

Sorry for the delay in replying. Long story short we didnt get to retry modifying the Cloudflare settings (not my area). We ended up creating a Windows Scheduled job which contained a call to the url using CURL. This got us what we wanted but im sure we could have got it working via Cloudflare with what you outlined above Charlie. Thanks again for the help.

[Charlie Arehart]

I want to add here, for the sake of future readers, that while Paul reported that the Cloudflare "fix" I offered above "broke his site", he was never able to confirm what he'd done (nor could we investigate) because he reverted to using the Windows scheduled tasks feature in PLACE of the CF Scheduled Tasks feature. Anyone who hits this error should in fact still consider the solution I offered above. 

Indeed, I just helped someone use the same approach tonight, and it worked--and his site did continue to work fine otherwise (as does my own, where I'd done this a long time ago). And I've helped others implement the solution.

One last thing: the person tonight wasn't getting this "error code 1010" reported originally here by Paul. Instead, they found that in CF's scheduler.log, the call to the task was logging an error of "403 forbidden", as in:

"Error","DefaultQuartzScheduler_Worker-2","09/23/24","15:21:00","","403 Forbidden".

That's what he presented instead, and it wasn't until we dug further that we realized (he acknowledged) that he was routing his sites through cloudflare, and had done that for a new server he was running, where this problem was happening. That's when I proposed we apply the technique I listed above, and now tasks worked.

In particular I wanted to add this mention of the "403 forbidden" error in the log so that others searching for that might find this post. (Recall that Paul instead had been looking at the output saved in the file as named in the CF scheduled task. Just two different results for the same error, it seems.)