Slow cfmail spooling and cfadministrator

[stuartw81]

Hi

We're running ColdFusion 2018 Standard with Update 1 installed on a Windows 2016 server.

When sending emails using <cfmail> we have two issues...

1) We fire email frequently, but not heavily. Maybe 100-150 email per hour during the working hours. These emails are backing up in the /cfusion/mail/spool folder and clearing very slowly. Each one takes approx 6-10 seconds to fire, but it does fire successfully.

2) In the Administrator > Server Settings > Mail > Maintain connection to mail server is not checked. When checking the box and submitting the form, the Administrator does not update this checkbox to stay on. When looking at the cfusion/lib/neo-mail.xml file this value selected as true.

<var name="maintainconnections"><boolean value="true"/></var>

Does this imply there's a glitch with the administrator?

Has anyone any suggestions to help?

Kind regards, Stuart

[WolfShade]

The only thing that I can think of is to add an attribute to the CFMAIL tag:  spoolenable="no".  This should send the mail immediately, avoiding the spooling.

HTH,

^ _ ^

[Dave Watts]

1) I'm not sure this is an actual problem. Mail deliver is not supposed to be instantaneous. And it is intentionally slower with Standard than with Enterprise. Enterprise includes a much more robust mail engine, to deliver a larger volume of mail. But if you're saying you send 150 emails per hour with a delay of 10 seconds per email, and assuming that each email is handled serially - which generally is not the case - you're looking at a 15 minute delay for the last email in the chain. That's well within the boundaries of acceptability for email delivery. It takes me more than that when I make a payment to my bank most of the time.

I would recommend against disabling the spool, as you'll really have no way to handle mail disconnection issues.

2) There is probably a bug with the CF Admin. The value that matters is what's in neo-mail.xml after a service restart. If that's true, then you're all set.

Dave Watts, Eidolon LLC

[Charlie Arehart]

Stuart, there can be many explanations  but let's start with your last point: it's possible you had an error during the one update you've applied. For more on investigating that, see me post:

https://www.carehart.org/blog/client/index.cfm/2016/9/6/solve_common_problems_with_CF_updates_in_10_...

Indeed, has the mail spool lag been since that update? Even if not, do check that log I point to, for the admin issue.

Finally, as for other possible explanations of the slow mail send issue, that could be caused by an anti-virus or firewall issue  Check that out.

(And if you may prefer direct help with these things--with satisfaction guaranteed or you won't pay, I can help remotely via screensharing. More at the consulting page at carehart.org.)

[Wilgeno1]

I did a lot of testing on the ColdFusion Mail Spool processing speeds for Standard and Enterprise editions and reported those results here.  https://coldfusion.adobe.com/2017/11/coldfusion-mailspoolservice-performance/

[stuartw81]

I'd already seen this, and re-read it several times, thanks for posting it. Have you done any similar testing for the 2018 version?

[stuartw81]

Thanks for your replies

WolfShade​. We've selecting the option without spooling and it's faster, but it's not the solution we're after. We definitely need to spool the emails to keep a decent user experience when multiple emails are fired.

Dave Watts​. You mention that the Standard version is intentionally slower than Enterprise. I've seen the tests done on ColdFusionMuse for CF11, but have you any other data for this?

Our tests today and the logs shown in HMailServer suggest there's exactly a 9 second delay between

"SENT: 220 mail.mailnameserver.com ESMTP"

and the

"RECEIVED: EHLO serverid" messages.

This could work out to be throttled to 400 emails an hour... Does this correlate with your experience?

Charlie Arehart​. Thanks for the link. We installed CF18 as a developer edition, installed Update 1 and then tested the server before buying the license, so we've not actually used the server without the update. We've checked the update log file (as mentioned) and there were

1417 Successes

0 Warnings

0 NonFatalErrors

0 FatalErrors

We've done a lot of testing on this today, checked the firewall and anti-virus (temporarily disabling scanning on the CF folders) and found no improvements.

------

We're lucky to be able to try our old server alongside the new one - CF11 vs CF18. Both running standard versions of CF and HMailServer for SMTP, both set with a Spool Interval of 15 seconds.

- On the new server running CF18 in CF Administrator, switching from the new mailserver details (127.0.0.1) to the old mailserver (old IP) works in exactly the same way as the new mailserver. It spools the emails virtually instantly, but then takes one from the spool folder every 15 seconds. When you watch the logs in HMailServer, the time from when it starts to send the email to having finished sending it is always 9 seconds.

- On the old server running CF11, in CF Administrator, switching TO the new mailserver details works as we think it should. It spools the emails virtually instantly, and then when the spool interval is reached it sends a good chunk of emails (25+ tested) - if not all - in one go.

This last test for us ruled out any problems within HMailServer, antivirus or the firewall and we believe that there is now a speed limit/spool throttling that has been added in after CF11.

------

Does anyone have any additional information that would help improve the spool speed? Or other options for firing emails from ColdFusion?

As it stands we're going to have to consider recoding our emails in something other than ColdFusion.

[Charlie Arehart]

I appreciate your diligence in trying things, but I still think you have missed something on the "new server". For instance, you say you have "disabled the av", but you may have disabled *some* feature of it and not the one causing this problem. Or there may be more than one a/v and you're not dealing with the one causing the problem (this is more common than you may think).

If you really feel that your conclusions "prove" that it's not the firewall or av on the new machine, but instead prove that the problem is in CF2018, I would propose you try one more test: put CF11 on the NEW machine. If you do that, and the problem happens, only THEN can you say that it's clearly CF2018. I am really strongly willing to bet that the problem would happen with CF11 on the new machine.

So your choices seem: keep digging into what may be holding up CF2018 calling to the web server, or put CF11 on the machine to prove if it really is CF2018. I've got my popcorn out, waiting to hear the conclusion. 🙂

[Dave Watts]

"You mention that the Standard version is intentionally slower than Enterprise. I've seen the tests done on ColdFusionMuse for CF11, but have you any other data for this?"

Just the documentation from Adobe saying they included a better mail engine with Enterprise. I mean, you can find it in the Standard vs Enterprise comparison matrix on the Adobe site.

Dave Watts, Eidolon LLC

[Wilgeno1]

The mail spool in ColdFusion Standard is single threaded while the Enterprise version is multi-threaded. That's the huge difference between the two. Additionally with the Enterprise version you can set the number of threads. The default appears to be 10 threads, but during my testing I was able to increase the thread numbers without any issues.