An exception occurred while performing action getall in cfimap tag

[Melvebak]

We upgraded from CF2018 to CF2023 a few months back and since the upgrade we have a cfimap call getting hung on emails here and there.  Most of the emails retrieval work as they should, however, one will bring ColdFusion down and hang the rest of the sites on the server.  Once the offending email has been removed from the queue the server will recover.  The process has been allowed to run for 5-10 mins and the cfimap process will not be able to clear it.  The other sites are down during this time.

We are not sure what is happening for sure.  One thing we are leaning towards is the email attachment size.  The emails that this has happen on seem to have attachments that are around 14 MB in size.  We are wondering if there is a JVM setting that may need to be updated to allow for the processing of a bit larger files? What would be different between CF2018 and CF2023 for cfimap that would cause this?

Any insight would be helpful.

Thanks.

The error message we recieve when this happens is:

coldfusion.mail.core.MailExceptions$IMapTagException: An exception occurred while performing action getall in cfimap tag.

Message: An exception occurred while performing action getall in cfimap tag.
Detail: The cause of this exception was: javax.mail.MessageRemovedException.
Type: Application

[BKBK]

The cfimap documentation shows no change between ColdFusion 2018 and ColdFusion 2023. So the issue may be happening because of a difference in the settings or in the environment, rather than a difference between the ColdFusion versions. So check the following:

1.  Recently expunged messages
   The exception javax.mail.MessageRemovedException  suggests that one or more of the messages requested by cfimap have been expunged. Verify whether mail messages have been deleted or whether attachments no longer exist or are no longer accessible.
2. JVM settings

Have a look at /cfusion/bin/jvm.config in ColdFusion 2023. Make sure that the heap size is significantly larger than maximum total attachment size.  For example, if the largest attachment is 500 MB, then use at least

-Xms1024m -Xmx1024m

3.  Cfimap timeout

Cfimap's default timeout is 60 seconds. Increase the timeout value to allow the application more processing time.

[Melvebak]

Thanks for your reponse.  We have tried all 3 suggestions and the issue continues unfortunately.  

We are still searching for a solution.

[BKBK]

Could you share the code? When you do, remember to hide any private or sensitive information.

[Melvebak]

There is not much to the code.  It is the standard cfimap getall tag.  We have been using the same code for the last 8 years with ColdFusion 2016 > current.

The script loops through the email and pulls the informaiton and inserts the data into the database.

[BKBK]

@Melvebak , I would disagree with you. There is in fact much to the code. After all, it keeps bringing the application to a standstill. 🙂

It is crucial to get feedback on your code, especially when it has a show-stopping issue like this one. More pairs of eyes debug better than one.

Could you also share the entire stacktrace of the exception?

@Charlie Arehart ,  thanks for sharing the link to the Tracker bug report. That is helpful. Just as well Adobe is also looking into the issue.

[Charlie Arehart]

I'm sorry that I missed this thread back in Feb. First, there is indeed a difference between cf2018 and cf2023, in that the former runs on Java 11 and the latter on Java 17. I share that as much to help interested folks here (Adobe or otherwise) consider that, though it may well not be where the problem is.

And it would help some to hear that I just found Melonie had opened a tracker ticket on this issue back then, https://tracker.adobe.com/#/view/CF-4220689, and she's updated it today with this response to bkbk's idea about the heap (and after Adobe offered a reply on the ticket last week).

I do have some other things to consider on the problem. 

First, Melonie, you'd reported there in March being on update 4. Update 5 (from October) had been a bug fix update. Had that also not helped? (I realize Adobe's not mentioning it last week may diminish the chance of it helping)

Second, if you have fusionreactor or the pmt or any Java monitoring, do you see what the thread (running the cfimap) is doing at the time of the hangup, in terms of its stack trace? And how about the other requests that seem to be hanging? Indeed, just knowing if a monitor CAN return this info at the time could be informative.

Finally and especially if getting that info would be difficult), it seems processing of attachment is a common denominator.  I wonder if you could try this the next time it happens. It's not a solution (and it entails restarting cf a couple of times), but it may identify the precise cause:

• First, don't delete the given email from the inbox as before 
• Instead, restart cf (annoying, I know) 
• Confirm if the request asking for the mail fails again (again, annoying. But it would confirm consistency/repeatability of this specific email being the cause)
• Then restart cf again 
• Then edit the cfml to remove the cfimap attributes related to pulling down the attachment. Since it's just a getall, the email and its attachment should remain on the mail server
• Does the cf code now run? That proves it's about the attachment(s)
• Next try pointing to a different, hard-coded path. A problem could be with the location to which cf was saving them. (it could even be a problem with whatever path was resolved for the variable naming the path. You may want to output that variable for this specific email you're processing, as a debugging step while you're at it.)
• If the cfimap now runs, I wonder if it would hang or fail if you set it back to the original attachmentpath but changed it to not use the generateuniquefilenames (that you show using in the tracker ticket). If it it fails (because another file exists with the name of the attachment being downloaded), does it not hang of you temporarily rename that other attachment file it conflicted with? Maybe the hangup is in dealing with such conflicts, but that's just speculation.

Again, having the stack trace at the time of the hang of the request may help most, but I realize that's not always easily obtained. (A full thread dump of all running threads might also help, but that's often less useful without the details about what requests are running, which tools like fr and the pmt can offer.)

I've offered all this as info to consider, for you or others who may find this thread in the future. Of course, I'm also available to help directly via remote screenshare consulting. But I appreciate that some can't avail themselves of that sort of help, or won't, or won't until exhausting all other avenues first. Clearly, I've tried to help above in that respect.

Hope it may help you move the ball a little further down field. If not, perhaps your reply on any of it may help me or others to suggest more to consider. 

[BKBK]

@Melvebak , I hope Charlie's suggestions and help will solve the problem.

Nevertheless, let me offer some more thoughts and suggestions, should you need them. 

From your description, cfimap "waits" or "blocks".  This suggests synchronous communication between the ColdFusion server (cfimap) and the mail server. Hence all the suggestions below are based on one main consideration: connection. 

• Between Java 11, which ColdFusion 2018 runs on, and Java 17, which ColdFusion 2023 runs on, there is one significant change that involves HTTP connections. From Java 11.0.11 onwards, TLS 1.0 and TLS 1.1 are deprecated. 
  So verify whether the mail server uses TLS 1.0 or TLS 1.1. You can do so by scanning the mail server's domain name on https://www.ssllabs.com/ssltest . If the mail server uses TLS 1.0 or TLS 1.1, then that is a red flag.
• On the subject of HTTP connections and TLS, make sure cfimap has the following attribute: secure = "yes"
• Still on the subject of HTTP connections and TLS, add the following settings to java.args in /{CF_INSTANCE}/bin/jvm.config, restart ColdFusion, and see if that helps:
  

  -Djdk.tls.client.protocols=TLSv1.3,TLSv1.2 -Dhttps.protocols=TLSv1.3,TLSv1.2 ​

• Let's say that the maximum total load of e-mails and attachments is 50 MB. Let's assume that, given the speed of your server, the download time is 90 seconds.
  Then use the following attributes for cfimap: timeout="90" stopOnError="true"
• Make sure that the connection is closed after use. You can achieve this by placing an action="close" tag after the action="getAll" tag:
  

  <cfimap action="getAll" connection = "my.cf.mail">
  <cfimap action="close" connection = "my.cf.mail">  ​

• Enclose all the cfimap code within try/cactch, for example:
  

  <cftry>
  	<cfimap action="getall" connection="my.cf.mail" name="myMail" timeout="90" stoponerror="true">
  	<cfimap action="close" connection = "my.cf.mail">  
  <cfcatch type="any">
      <!--- Dump the exception to an HTML file --->
  	<cfdump var="#cfcatch#" format="html" output="#server.coldfusion.rootDir#\logs\cfimap_exception.html">
  </cfcatch>
  </cftry>​

  Share the exception with the forum.

  
  
  

[BKBK]

Second thoughts:
In my latest suggestion above, you will see, on two occasions, that an action="close" tag directly follows the action="getAll" tag:

We might be able to improve on that. The code might be more robust if we applied a named lock to both the cfimap "getAll" action and the cfimap "close" action. Then both actions can never run simultaneously. The close-action will only run after the getAll action has completed. 

<cflock timeout="90" name="cfimapLock" type="exclusive">
	<cfimap action="getall" name="myMail" connection="my.cf.mail">	
</cflock>
<!--- Note: same lock name! --->
<cflock timeout="90" name="cfimapLock" type="exclusive">
	<cfimap action="close" connection="my.cf.mail">	
</cflock>