CF9 spooling mail VERY slowly

[John_Sieber]

I recently moved my site from CF7 to CF9. We normally send out several bulk emails and newsletters in the 20,000 - 50,000 message range. I send these emails in chunks of around 500 messages at a time and then wait for the spool directory to have less than 50 messages before sending another chunk. This has worked great for us for several years but now with the move to CF9 the messages are leaving the spool directory very slowly. It seems to process one message every 1 to 2 seconds. A list of 40,000 emails would in the past take say 8 hours is now taking 30 + hours to go out.

Here are things that I have already looked into:

Running Symantec Endpoint Protection and have set a rule up to exclude the CF Mail directory as well as it's child directories. I had to make the same change on my old server as the AV scanning was slowing the process tremendously.

When running CF7 I was using a smtp server on the same machine, but now the smtp server is a different machine on the same network. This should not cause these issues and the smtp server is not bogged down or having issues.

Has anyone else run into these issues with cfmail on CF9? How many messages should leave the spool on each interval? I know on my CF7 server it did not send only 1 message per interval, why might this be the case on CF9? Any help or insight on this issue would be greatly appreciated as I desperately need to solve this issue.

[Adam Cameron.]

What's your spool interval set to?  Have you switched mail logging to "debug" and see if the logs tell you anything?

Have you verified it's the speed CF is sending them to the SMTP server, or whether it's the SMTP server taking its time to send them on?

--

Adam

[John_Sieber]

Thanks for the reply Adam! I should have added all of these details from the start. My spool interval is set at 20 seconds. I also tried at the default setting of 15 seconds as well. I did set the logging to debug and the only thing that I noticed was an occasional, "Error","scheduler-0","02/18/10","14:39:02",,"Exception reading response". Looking at the exception log it appears to be a timeout waiting for a response from the smtp server. There was several of these in a row over a ten minute period in the middle of the night and then 2 more spread out over a couple of hours today.

It appears to be an issue with the CF server to me, from observing the behavior, but I could be wrong. When I watch the spool directory on the CF9 server it just consistently sends one email at a time about every one and a half to two seconds. As it was taking over 24 hours and my newsletter still had a 1/4 of its recipients to go, I sent the remainder of emails this evening from my CF7 server and they went from the spool to the mail server in large chunks like usual. The behavior that I have watched will typically send around a messages during a spool interval if there are around 500 in the spool directory. As the total number of messages in the spool directory decreases so does the number of messages sent from a spool interval.

The big difference in this is that the CF7 is also the mailserver so it is not having to send the mail across the network. All of my experience with using cfmail for large emails has been with using a local smtp server. Could it be that there is just this large of performance drop when using a remote smtp server? If I have to use a local smtp server I can do that but I thought it would be nice to use my existing Exchange server for sending out mail from the site.

[Adam Cameron.]

What is the ping time like between the two boxes?  Unless you have a network problem (which could - of course - be the case!), I can't see your performace drop off like that.

I was gonna suggest perhaps sending an email or two manually using telnet and see if you notice anything, but I'm not sure you'd be able to get much useful info from doing that.

What sort of server is the CF one?  If it's a Windows box (it sounds like it is), how about using the in-built SMTP server for a bit whilst you're troubleshooting?  Even if you can't actually send email from it, you could at least check the speed that emails go from the spool to the undeliverable dir, which might tell you something.

I wonder whether there's any authentication overhead between the CF and SMTP boxes, if they're on disparate systems?

--

Adam

[John_Sieber]

Hi Adam,

I did a a couple of pings between the two servers and found the connection to be very fast at 3ms. I also tried sending a few emails from the new server running CF9 using telnet to the old server hosting the mail and they seemed to go just fine. My typing skills seemed to be the speed limitation in that test.

Both the old and new servers are running Windows. The mail server is running Server 2003 and Exhange 2003, it also used to run CF7 unitl I moved the website to the new server. The new Server is running Server 2008 and CF9. I think the next step is to setup a smtp server on the Server 2008 box and try to see if that allieviates the problems with the mail leaving the spool so slowly.

I wondered a bit about the overhead of the authentication as well after reading that each message is sent individually. I configured the exchange server to allow relaying from the CF9 servers IP address without authentication to see if that helped but the end result is the same.

I'm wondering if most people run a smtp server along with CF for sending mail or if it is normal practice to configure a remote mail server in the cfadmin?

I don't have another large mailing to send until the middle of next week, but if I find out more before then I will update this thread as it may save someone else a bunch of time down the road.

[Adam Cameron.]

You're doing all the same things I would be doing to troubleshoot, so I'm not sure I can be of too much more help here.  Although I'll keep it ticking over in the back of my mind, in case anything new occurs to me.

Our SMTP servers have always been on separate boxes, but we never to huge mail send outs.  Plus... there ain't no way I'd be running CF9 in production yet anyhow.  Not until at least 9.0.1, anyway (no dis' to CF... I would never run a "fresh off the blocks" release of any software in production!).  So I've no experience at all with how CF9 handles it.

I cannot find a reference now, but I seem to recall it was either CF8 or CF9 in which the email engine was revamped and was supposed to be orders of magnitude faster than it used to be.  So your experiences are interesting (if inconvenient for you!)

--

Adam

[John_Sieber]

Adam, I appreciate you helping me to think it through. I was able to do a little more testing this afternoon with interesting results. I installed a mail server on the same box that was setup to relay the mail to the other exchange server. I'm seeing an immediate performance increase but not yet the same performance that I had with my old CF7 setup on the same machine as the exchange server.

The testing I did today was with a 100 messages so it was not an apples to apples test. What I saw was .mail files leaving the spool one or two at a time at a rate of several a second. You could see the pause about half way through each interval where it would wait for the next threshold interval. On my CF7 server the whole chunk would disappear at once instead slowly leaving one or two at time and then it would wait for the next interval. So, we will see if it slows down the entire process next week when I have to send a larger mailing.

I think that you are right that they supposedly made improvements in CF8 to the mailing engine. Right now I would be very happy with the performance that I had on CF7. Hoping I still stumble upon a reason why it seems so slow for me right now. 

[Hedge]

Did you by chance ever find a solution to this? We have the exact same problem but it is from an upgrade of CF7 to CF8. worked find on CF7 and sent the messages quickly from the spooler tot he mail server but since the upgrade it only sends out 1 or 2 messages every 15 seconds. Using the same network and same mail server the only difference is CF8. I have other CF installation on the same network using the same mail server with no problems at all. Very frustrating.

[John_Sieber]

I eneded up installing a SMTP server on my ColdFusion server that relays the mail to our actual email server. This ended up taking care of the speed issues for me.

[Guest]

I am experencing the same issue. Is Adobe aware of the problem?

It seems a bit silly, that it would take 3 months to get a patch out for this critical of a problem.

[John_Sieber]

Not sure, after installing a local mail server that is relaying to my actual server the problem went away for me. I would be curious if it will ever be resolved as I did not have the issue at all when running on CF7.

[Guest]

I can confirm that it has definately NOT been resovled. All this money for CF 9 and it doesn't even perform as well as 2 versions ago.

[Kaif_Akbar]

As it was working fine in CF 7 but after migrating to CF 9, you are facing this issue, whereas you also performed following:-

• Tested it with local machine exchange server, which speeds it up.
• Tested it with remote exchange server with around 100 emails, which also worked faster.
• Tested network connection, which is also working fine.

Further you can change following "Server Settings">> "Mails" >> "Mail Spool Settings":-

Mail Delivery Threads -> 10 , you can increase this to higher value if you machine has several processors.

Spool mail messages for delivery  -> Set it to "Disk" so that those thousands of emails won't require additional memory from ColdFusion memory whereas Spooling to disk would be more robust too.

[jeremylittle777]

I can confirm that my spool on CF9 was working great for a couple of months and now BAMM! slow as ever!! 1-2 messages a second. This is rediculous!! I have not done anything to the settings! @Adobe can you please fix this!! or atleast give me a good option to speed this back up?

Thanks

[Anit_Kumar]

Did you check, wether the problem lies at SMTP end or at ColdFusion? Also, please check the logs after changing the mail logging to "debug" for any errors.

[Hedge]

Has any solution been found to this problem? We have 3 separate CF machines and a separate mail server. Two of the machines are CF 9 and 1 is CF 8. Cf 8 had been working find sending emails in big chunks at the time then all the sudden like 3 days ago it sends one every 5 seconds. I can FTP the cfmail files from the CF8 machine to either of the CF9 spool folders and it goes out quickly. Very frustrating because we had a completely different CF8 server do this to us in the past and the only way we could fix it was to completely rebuild the box.

[kesharmehra]

Don't know if I have found a solution to this problem.

I am having this problem with standard version of CF9. I notice that the moment you add more scheduled tasks, the mail spooling goes very slow.

Now I have removed the tasks and shifted them to another server and its working fine.