zero-day exploit affecting the popular Apache Log4j utility (CVE-2021-44228)

[Jdsplicer]

Does anyone know if the zero-day exploit affecting the popular Apache Log4j utility (CVE-2021-44228) that was announced on 12/9/2021 will affect ColdFusion version 10 and 2018?

[BKBK]

Hi @Priyank Shrivastava. 

Thanks for the update. It raises an immediate question. One of the JARs has been renamed from 

log4j-to-slf4j-2.16.0.jar / log4j-to-slf4j-2.17.0.jar

to

log4j-slf4j-impl-2.16.0.jar / log4j-slf4j-impl-2.17.0.jar.

Doesn't that matter?

[jhansen-cf]

I deleted log4j-to-slf4j-2.16.0.jar and added log4j-slf4j-impl-2.17.0.jar

CF2018u13 started up and appears to be running fine. 

[Priyank Shrivastava.]

@BKBK @jhansen-cf  Can you please re-download the zip using incognito windows, I have added the correct jar again. 

please remove the one that you have added and add the from the zip.

Thanks @BKBK for letting us know. 

[aliciam29605858]

When I tried the new zip file, I get this message: "Unable to initialise CFStartupServlet:Unable to initialise Logging service: coldfusion.server.ServiceException: log4j-slf4j-impl cannot be present with log4j-to-slf4j". 

[AjasHadi]

Per my understanding and the confusion it created with first set of instructions, as of now, you just need these 3 files below if you are on CF 2018 or CF 2021. You dont need the log4j-slf4j-impl-2.17.0.jar that was present in original zip and caused confusion.

Just use these below

• log4j-core-2.17.0.jar,
• log4j-api-2.17.0.jar
• log4j-to-slf4j-2.17.0.jar

[Priyank Shrivastava.]

Hi @aliciam29605858 

There was some issue with old zip and we found that a jar was missing. I have updated the link with correct jar file, you can download the zip using incognito window and as @AjasHadi mentioned, please replace the jars. 

Do check the article once if there is any doubt. 

[aliciam29605858]

Thank you, I should have read the instructions more carefully.  I guess I assumed that only the needed jar files would be in the zip, but that was a bad assumption.  Thanks again.

[Charlie Arehart]

Priyank, even today (a day after your last comment here), the problem with the zip remains. It still has six log4j 2.17 jars, while CF2018 and 2021 both have only three log4j 2.16 jars in the instance's lib folder. While the technote does tell people to only replace the three jars, it's unclear why and confusing that the zip has six files.

And someone shared previously here that they put in all six (figuring that was what was expected) and they got an error on starting CF. (I can offer a clarification that in my default deployments of CF2018 and 2021, after doing the CF update from last week, these three are the ONLY log4j 2 jars, in all folders of CF (other than the hf-update backup folders, which hold copies of files CHANGED by an update, which are not on any classpath and so should not concern people.)

You suggest here, Priyank, that the zip was changed and that people should open a new incognito window to get it. To be clear, I am obtaining the zip for the first time on a given machine, so it's not a caching issue.

Or is anyone seeing otherwise? I tried this at 945a US Central on Dec 22. If anyone might see this and try it and find that the zip DOES now have only three jars, please do share your observation. Thanks.

[jeffh65754959]

I just downloaded the Zip file and see all six jar files.  It appears they are all the files you would need if you had the API Manager and Monitoring Tool installed, as well.  The API Manager has log4j-slf4j-impl-2.17.0.jar and log4j-jul-2.17.0.jar in addition to the api and core log4j jar files.  I can see how people would be confused and think they had to copy all files into the lib folder.  

PS, I am hoping to be contacting you early next year, Charlie.  Things just move slower now.

Jeff

[Charlie Arehart]

Ah, good point, Jeff. That probably explains it, but yes indeed, potentially confusing (or worse), so hope they'll address it some way. OK on the rest, and happy to help. 🙂 

[BKBK]

Thanks, @Priyank Shrivastava. , for your help and timely support!

[michaelm87459285]

FYI-Nessus updated today to enumerate the log1j 1.x  https://www.tenable.com/cve/CVE-2021-4104

While 1.x is not immidately vulnerable unless configured to use JMSAppender (not default), we belive that we will soon be required to update all log4j libs to the current version 2.17.  Joy. Any chance we'll see CF 21 update 4 that does that for solr?

[michaelm87459285]

if interested, here's Apache guidence on manual upgrades to v 2. https://logging.apache.org/log4j/log4j-2.12.3/manual/migration.html

[Charlie Arehart]

You CAN update to the 2.17 jars, after having applied update 3. Same for u13 of cf2018.

See comments here from earlier this week, as well as an update made to

"the answer" for this thread. Both point to the Adobe technote from earlier this week, detailing that AND offering the 2.17 jars. 

Also, the updates DID indeed remove the JMSAppender class (and other vulnerable ones) from any log4j 1.x jars that cf came with. If a scanner looks only at jar version numbers, a better scanner will look to see if the vulnerable classes are there in the jar. 

[Charlie Arehart]

And as for whether the next update to cf would include these 2.17 jars, surely it would. But we should NOT expect an update in coming days or weeks that would do ONLY that, no.

It would seem the next update, in weeks or months, will be whatever was originally in the works before this blow-up, with whatever additions or fixes they'd been working on. And it would fold in bug fixes from the PREVIOUS updates (2 and 12) which were NOT folded into the most recent update, and any fixes between now and then.

I know you'll want to hear that from Adobe rather than me. Perhaps they will clarify. 

[michaelm87459285]

Yup- That's where we are at. Nessus only looks for what it looks for, not for the config 😉

[Charlie Arehart]

First, when you say "that's where we're at", are you referring to the 2.17 jars or 1.x jars? To be clear, you CAN fix "the problem" of your "seeing CF using 2.16 jars" by droppign in the 2.17 jars, as I just discussed. And one can fix the problem of CF "still using older 1.x jars" by doing the CF updates released on Fri Dec 17, which removed the vulnerable classes within those jars.

If it's that second point about log4j 1.x jars in the cfusion/lib (which don't exist by default in CF2021, and there's just the one in a CF2018 cfusion/lib), again that DID get updated by the Dec 17 updates. And I gather your concern is that you want the file to be "gone" instead, because its name fails the scan. If Adobe could have removed that jar, it sure seems they would have. 

So then we're back to a concern about the scanner searching only for name patters. That's really not reliable, frankly. First, bad guys can manipulate file names. Second, one may find merely a file named log4j.jar (as is the case in CF2021's cfusion/lib) 

I would be surprised if the scanner didn't have another option to scan the files more deeply--not for "config" as you say, but just for more file names. As you may know, a jar is just a zip, and so most any tool even looking for files by name should also be looking IN jars (treating them as zips), since a vulnerable file may be found there (even if those also are sought only "by name").

I was saying that if such a scan did look at a jar (as a zip) it could detect if within that zip (jar) there were certain named files, such as the jmsappender.class that all recent discussions of log4j 1.x  vulns have been sugesting should be removed...and again which the CF updates from last week do, for any log4j 1.x files found.

I get it, some people (and some tools) simply "don't want to see any log4j files with numbers that are presumed to be vulnerable". You may feel you have no choice but to have CF pass this scan or you have to stop using CF. And so you may assert Adobe "has to fix this". They may, but they may not. (And as you may know, you can't just DELETE the log4j 1.x files. More in a moment)

If you're adamant about this (and Adobe does not move fast enough on this front for you), you could technically rename the "bad" jar files yourself, once you've confirmed that it does not have those vulnerable classes within it. FWIW, Java really doesn't care about file names. It looks in all jars in its classpath, and tries to load classes (and packages) as may be found in any jar found in the classpath. If you renamed a jar, Java wouldn't care.

For example, in CF2018 evenr after is upate 13, there is still a log4j-1.2.15.jar in its cfusion/lib folder. If one stops CF and simply REMOVES that file, then starts CF2018, while it will START, if you try to go to the CF Admin (or run any page), you will get a 500 error/nullpointer exception--in CF's logs, even if something keeps you from seeing that on screen.

Yet if you stop CF and put that file back, and then RENAME it to even simply bob.jar, then start CF and test a page, it will now work. So if you want to rename the file to something that will "pass a scan" that's focused solely on file name, I'm just saying that technically you CAN.

(One risk with that is that if Adobe or anyone else does later implement a NEW version of that jar, they would be looking to remove that current jar by its known name, and they would NOT think to remove the bob.jar you called it. Workarounds are usually two-edged swords. But desparate times call for desparate measures.)

Anyway, I suspect you (or anyone in your situation) would say you don't want to take the responsibilty of renaming that file, even after confirming it was "safe" in terms of it having the embedded class files removed.

I'm just offering info while you await any better answer from Adobe. 🙂

[michaelm87459285]

Charlie-what I meant by "that's where we're at" was we applied the CF realted patches and 2.17 updates- so good to go there. On 23 Dec we got new Nessus findings on all the 1.x versions associated w Solr etc. Nessus shows the existanece of any 1.x as critical becuase it is not supported and several years beyond end of life and Apache is not evaluating security of the components.  The focus on log4j is super high right now so presume Tenable is trying to err on the side of caution. I'm sure we will be required to "fix" those in short order, even though they are not realted to the current log4j 2 RCE and even with the jmsappender.class removed.

[Charlie Arehart]

Yep. And we can hope that Adobe will indeed undertake efforts to remove any and all log4j 1.x files from CF2021 and 2018. Perhaps that will be in whatever is the next update, which could be weeks or even months from now. Whether they regard this "need to remove log4j 1.x files ASAP" as something to prompt them to do another emergency update (like the Dec 13 one was) is another question. That's why it may take weeks or months. (To be clear, they should not be expected to offer any update for CF2016 or earlier versions, as those are no longer supported).

If someone may say "the US Federal Government is mandating removal of software that is not updated", my read of things (such as at the primary UG gov doc on the log4j vuln , and the related alert on how things are to be addressed, updated yesterday) is that those are focused solely on the log4j2 lib vulns. I see no mention of log4j 1.x (in any variation). But each org needs to read such docs and make their own conclusions. 

Again, I'm not arguing against orgs who may want to demand remediation or even removal of any log4j 1.x jars. I was just offering info, context, and perspective that some may want to consider. I don't claim expertise in all this, and it's all a fast-moving subject, such that something we may say one day is trumped by something learned the next.

[Charlie Arehart]

Adobe folks, why has nothing been done about updating the class files within the log4j-1.2.17.jar, found within cfusion\jetty\lib\ext\ in both CF2021 and 2018, even after the Dec 17 updates?

To be clear, I do understand that the log4j-1.2.15.jar in the cfusion/lib of CF2018 (after update 13) HAS in fact been modified to remove the vulnerable classes, like jmsappender.class. But I am NOT seeing that those vulnerable classes have been removed from the log4j-1.2.17.jar in the cfusion/jetty/lib/ext. 

That seems an oversight. Might you perhaps create another technote that points this out and offers folks an updated version of the jar file that DOES remove those vulnerable classes?

I realize also that some people are raising the concern that their security scanning tools are triggering on the mere PRESENCE of files with names like that log4j-1.2.15.jar in CF2018's cfusion/lib, even when that HAS been updated to remove the vulnerable classes. At some point, it seems you really will have to just update CF to REMOVE those log4j 1.x jars, whatever it takes. Otherwise, too many folks will be unable to fend off their less-sophisticated security folks, who don't generally want to stop at hearing or even seeing prood that "the vulnerable classes were removed from the jars".

But again, that's separate from the current problem that the jetty/lib/ext folder has a truly vulnerable log4j 1.x jar, even after the Dec 17 updates.

(If somehow I am misunderstanding or misrepresenting things, I do apologize. I really don't want to raise the panic level any further, esp as we enter the holiday season.)

[EvanSinceCF31]

There is also the version of log4j 1.x embedded inside cf-logging.jar.  

On CF2021, comparing before and after, the Adobe patch removes some of the known vulnerable classes from the coldfusion.org.apache.log4j folder in the .jar and Priyank reported "We will be upgrading the library in future update".

Besides the open CVEs logged against log4j 1.2, likely mitigated by removing the vulnerable classes, our security folks also point out that log4j 1.2 is no longer maintained and may be subject to other vulnerabilities.  I realize if we tried to remove every library that is not "actively maintained" we would quickly decimate a good part of the open source universe, but seeing as how log4j 1.2 is under the lens and is going to be an ongoing sticky point with IT security experts, Adobe needs to at least announce a plan to resolve this as soon as practical.  

[ccsimmons_FAVER]

Are you aware of step by step published anywhere to mitigate the presence of C:\ColdFusion2021\cfusion\jetty\lib\ext\log4j-1.2.17.jar?  The install we have is with a government agency and this file is causing a Nessus security scan failure.  If we don't find a way to pass that scan they will pull the plug on the server.

[ccsimmons_FAVER]

I meant to reply to this post with the following:

Are you aware of step by step published anywhere to mitigate the presence of C:\ColdFusion2021\cfusion\jetty\lib\ext\log4j-1.2.17.jar?  The install we have is with a government agency and this file is causing a Nessus security scan failure.  If we don't find a way to pass that scan they will pull the plug on the server.

[Ripley Casdorph]

The Jetty folder is used for one of the monitoring services that we don't use and I was able to replace with a 2.17 version, no issues for us.  We were getting killed on that too!

[Charlie Arehart]

To be clear, the jetty folder is about the cf add-on service (not monitoring), which is the solr or pdfg/cfhtmltopdf feature.

There are no documented mitigation steps. One could uninstall the cf add-on service if you're not using it. One could even just yank the log4j jar, if you wanted to just stop the add-on service without installing it. 

We can hope that some future update (perhaps coming even this month) will address this more appropriately. 

BTW, Ripley, the jetty aspect related to monitoring is yet ANOTHER jetty that Cf has had, starting back in cf9, which was about offering a separate port and web server through which to access the enterprise server monitor. It has its own jetty.xml config file in the cfusion/lib folder, which relies on a jetty jar in that folder. That is NOT the cfusion/jetty folder, added starting in cf11 (if one enabled the add-on service at cf install), which is the focus of the log4j concern in this thread. It can get confusing! 

Hope that's helpful.