Skip to main content
B
Bill Tudor
Known Participant
September 21, 2023
Question

CF21 FastMethodInterceptorDelegatorBoot class broken by AppDynamics (ClassNotFoundException)

  • September 21, 2023
  • 2 replies
  • 1852 views

We have found that when an AppDynamics agent is running in the JVM our scheduled taks fail to run

error something like this:

 

[ERROR] runwar.context: java.lang.NoClassDefFoundError: com/singularity/ee/agent/appagent/entrypoint/bciengine/FastMethodInterceptorDelegatorBoot

[ERROR] runwar.context:             at coldfusion.cache.ehcache.GenericEhcache$CacheCleanUpAgent.run(GenericEhcache.java)

[ERROR] runwar.context:             at coldfusion.scheduling.Scheduler.run(Scheduler.java:245)

[ERROR] runwar.context:             at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] runwar.context: Caused by: java.lang.ClassNotFoundException: com.singularity.ee.agent.appagent.entrypoint.bciengine.FastMethodInterceptorDelegatorBoot not found by scheduler [19]

It seems others (including Atlassian and Oracle) have encountered similar when implementing AppDynamics apm, and the accepted solution seems to be a variation of adding "org.osgi.framework.bootdelegation=com.singularity.*" to either JVM or properties files depending on your stack.

Cannot find an equivalent 'place' to add something like this to CF and was wondering if anyone has encountered this and found a solution in CF. Note that CF2016 played nicely with AppDynamics, so this has manifested only after an upgrade to CF2021 and therefore Java 11 and a later version of AppDynamics.

 

      This topic has been closed for replies.

      2 replies

      A
      Community Manager
      AbhishekJhaCommunity Manager
      Community Manager
      September 25, 2023

      Hi Bill,

       

      Could you please share the ColdFusion2021\cfusion\logs - coldfusion-out, coldfusion-error and exception logs to cf.install@adobe.com

       

      Thanks,

      Abhishek

      B
      Bill TudorAuthor
      Known Participant
      September 26, 2023

      Thank you Abishek - I have sent logs as requested

      Charlie Arehart
      Community Expert
      Charlie ArehartCommunity Expert
      Community Expert
      September 22, 2023

      Assuming that desired jvm args is the right solution, you ask where to put it. That depends on how cf itself has been deployed. 

       

      1) The place to add such jvg args in a typical cf deployment would be either a) the cf admin's "java and jvm" page and its Java args field, or b) the jvm.config file (found in cfusion/bin or [instancename]/bin) and its java.args line.

       

      Always be very careful editing either of those (changing one changes the other), as a mistake can keep cf from restarting. To be safe, make a backup of the file before touching it.

       

      2) That said, your error message mentions runwar, and that suggests you may using Commandbox to run cf. Is that so? If so it's important to clarify. It offers still other ways to effect a JVM change, though the cf admin page above would still work. 

       

      3) Or you could be deploying cf in yet another way where you're deploying it as a war file, like on Azure App Service or your own implementation of Tomcat or another app server. In that case, the admin does not offer jvm page and there is no jvm.config file--because all jvm arg definition is dependent on the specific app server platform/servlet engine. Again, you'd need to clarify which you're using as there are many variants. 

       

      4) Finally,you could be deploying cf as a container/image, in which case the Adobe cf images offer no means to easily modify the jvm args. But I show an example of doing that among my many sample config files at my "awesome cf compose" GitHub repo. 

       

      5) As for your error, if your arg does not indeed solve it, note that it may not be about the cf version upgrade but instead (as you mentioned) the jvm upgrade which was implicit in the cf upgrade. 

       

      Cf2018 and cf2021 run on Java 11, and earlier versions could depending on cf updates applied, otherwise they ran on Java 8 or earlier. (I have a blog post with a table of what cf versions/updates support what jvm versions.)

       

      6) And as you may know Java 9 introduced a new modularity control mechanism, which defaults to limiting what Java libraries/classes can be called from within Java (and thus cf) code--in the name of greater control/security. 

       

      And that jvm version (and later) added new jvm args to control/override that default control of library loading, such as --add-opens, --add-modules, and --add-exports. Some may have noticed that Adobe has implemented those in the jvm args for cf versions whose installers were created to support implementing versions beyond java 8.

       

      And you may therefore need to add one of those, but the error message you offer doesn't indicate that directly, so again I just offer this for consideration if the above answers to your question about "where to put such args" doesn't get you going.

       

      Let us know how it goes. 

      /Charlie (troubleshooter, carehart. org)
      B
      Bill TudorAuthor
      Known Participant
      September 25, 2023

      Update: we have both commandbox and silent Windows installs of coldfusion using the standard msi installer and we have tried both the in-built Cf jre and pointing to a different java_home. All flavours exhibit the problem. We also have other JVM arguments that work as expected and we haven't stopped Cf loading and running. It's just that once the AppD javaagent is running CF can no longer run scheduled tasks as it cannot load dependent com,sigularity.* classes (it's as if the AppD agent has overritten Cf's com.singularity classpath and we see exceptions like posted above 'NoClassDefFoundError'). This only adversly affects the scheduler, even though we also see similar errors in coldfusion-error.log  for mail and ehcachce packages, they still seem to work ok)...so we contacted AppDynamics support....

       

      After a v.long and in-depth call with Appynamics it looks as if the solution that works for other organisations will not work with Cf because any JVM argument to tell apache.felix framework where bootdelegation com.sigularity classes are (or where they are not) is ignored - usually it's a flag something like (in Atlassian's case)

      -Datlassian.org.osgi.framework.bootdelegation.extra=com.singularity.*

      Their advice "For other OSGI-based runtime containers, add this package prefix to the appropriate OSGi configuration : 

      file.org.osgi.framework.bootdelegation=com.singularity.*

      Also didn't make sense in context of Cf - as 'where' would we add such a property.

      So I was trying to ask how/where can add this fix for Cf; either as a JVM flag or a jar property file.There appear to be no .properties files under the hood of Cf (we exploreed the jars mailnly in cfusion\lib). Hence my vagueness as to 'where' to add this fix. It is definitely a JVM 11 issue as you hint at -  AppD older version JAVA 8 agent jar worked fine with CF2016 (and no, we cannot use the old AppD agent.jar unfortunately - that would be too easy!). 

      Not sure it's relevant but we use IIS or whatever Tomcat/Catalina comes out-of-the-commandbox - what I mean is nothing unusual in terms of  application or webservers.

      D
      Community Expert
      Dave WattsCommunity Expert
      Community Expert
      September 25, 2023

      I'm pretty sure you'll need to add this to jvm.config. I don't know what CF uses for "file.org" in the string here:

       

      file.org.osgi.framework.bootdelegation=com.singularity

       

      Anyway, follow up with the CF support team as Abhishek recommends. Good luck!

       

      Dave Watts, Eidolon LLC 

      Dave Watts, Eidolon LLC
      Terms & ConditionsAccessibility statement
      Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices