Enabling "Automatically Write Changes to .XMP" changes orig file creation, modification dates!

[$$PhotoHobby]

WARNING, a bit of an angry rant here given the damage... This issue was encountered with LR Classic on Mac.  I have no idea if it occurs elsewhere.  I sure hope not.

I have a catalog with about 180K images and videos, all organized into folders by date.

In each folder, I have the original images that should NEVER be modified.  That includes the OS file creation time (which will match the date/time when the file was created by the camera if you don't do anything silly when importing images). Preserving the file creation time is very useful! Yesterday, I decided to enable the "feature" to "Automatically Write Changes to .XMP" in the Catalog settings.  I did this as it seemed like a good idea to keep the metadata on disk as a sidecar to the original image (in addition to it being in the database). My hope was that this would provide an additional layer of security in case a catalog got corrupted, etc.  You could sync the folder and get back all the metadata from the .XMP file.  In theory at least.

To my horror, I just discovered that turning on this feature launched some LR operation that is walking the entire image collection, presumably to read metadata that will be copied into the .XMP sidecar, but in the process LR IS UPDATING THE ORIGINAL IMAGE CREATION AND MODIFICATION TIME TO THE CURRENT TIME!  Even worse, I discovered this problem while backing up my Photos drive using rsync as I usually do.  I couldn't figure out why the heck so many old, totally unmodified, files were being transferred to the backup so I looked at the file metadata and discovered the problem.  To add insult to injury the rsync had already overwritten thousands of files on the backup drive.  All that file metadata is GONE.

This seems like a spectacularly bad bug.  AFAICT there's ZERO reason for LR to update the modification time, let alone the file creation time!

Is this a known issue?

Is there anyway to undo this? (e.g by using the image capture time from the image metadata to update the file creation time to match.  Not perfect, but better than being totally wrong thanks to LR).

Hoping the LR engineering team gets on this one quickly.

[Ian Lyons]

You haven't mentioned the file format (e.g. proprietary raw, DNG, etc). The reason I mention this is that the proprietary raw file should never be touched when 'Automatically Write Changes to XMP' is active, instead an XMP sidecar should be created/updated. However, with DNG there is currently no XMP sidecar option. Therefore, the original DNG file is updated when the file is changed in LrC.

BTW, this is not new behaviour, but as you have indicated, not necessarily expected or desirable. Also, just in case my memory was playing tricks, I installed version 10.4 and 10.0 to confirm that the behaviour you describe did occur prior to 11.x, which it did.

> To add insult to injury the rsync had already overwritten thousands of files on the backup drive.  All that file metadata is GONE.

When 'Automatically Write Changes to XMP' is active the metadata is updated to reflect the catalog contents. There is no read back from file to catalog or sync. Therefore, not sure why you've lost metadata in files.

[$$PhotoHobby]

This has affected at minimum JPEG, HEIC as well as Canon Raw (.CR2) files.   After doing more investigation in the forums, I learned that for JPEG images (and possibly others) it seems that the image is updated IN PLACE (i.e. the file is modified).  This on its own seems like a bad idea. For JPEG and HEIC files, no .xmp sidecar seems to have been written for the files I saw.  Bear in mind I didn't review all 180K images!  For Canon Raw files, .xmp sidecars have been added but the file creation and modification times have been changed.

It's unfortunate that I just stumbled on this apparenly long-existing issue now.  I did not realize the seriousl consequences of the setting to write metadata to .XMP.  A reasonably assumption which I made was that LR would honor the principle of "Never modify originals!" but apparently that's not the choice the LR team made.

What's even more annoying is that on many operating systems it's possible to update the file creation and modification times so even if LR did save metadata back into the file, it could keep the original datetime for creation and modification.   I get that the latter might be contentious but the former seems appropriate.

[Jao vdL]

Yeah it is pretty annoying that you can't make Classic write separate sidecars for files that can contain the xmp in their metadata. This is especially important for backups where having separate sidecars would allow you to do incremental backups of just the small xmp files instead of having to backup the entire file every time you make a small change to the image. 

[Ian Lyons]

The only edit to a proprietary raw file that Adobe has ever allowed in Lightroom or Camera Raw is the Capture date/time. So, not sure how or even if, it was 'Automatically Write Changes to XMP' that resulted in modification date changing.

[johnrellis]

"For Canon Raw files, .xmp sidecars have been added but the file creation and modification times have been changed."

To clarify, .xmp sidecars were created with Date Created and Date Modified set to the current time, but the Date Created and Date Modified of the .cr2 files were left unmodified, right?

There's an existing feature request for LR to use .xmp sidecars for all file types, not just raws:

https://community.adobe.com/t5/camera-raw-ideas/p-store-the-xmp-metadata-outside-dng-jpeg-etc-file-t... 

But it's seven years old and only has 26 votes (I voted for it), so at this point I think it's very unlikely Adobe will change this behavior. (They're putting little effort into LR Classic, except for Develop.)

[Jao vdL]

What kind of files are these? If they are proprietary raw (I.e. Nef, arw, etc.), automatically writing metadata will only touch the xmp  sidecars which is the behavior I see on my machines and no originals will be touched and updated on my backups. But if the originals are jpeg, tiff, psd, png, or dng than yeah writing metadata to them will rewrite the entire file every time. Unfortunately (there is a very long standing feature request on this forum to change this but Adobe hasn't acted on it) Lightroom Classic will not write xmp sidecars for files that can have embedded metadata but will always write it into the file which will update its filesystem metadata. It has always done this ever since the automatic metadata writing showed up and this has always annoyed me as it impacts my backup strategy.

[Jao vdL]

Sorry just noticed Ian already gave basically the same answer! Not sure why the forum only showed that to me after I typed my answer.

[johnrellis]

"Is there anyway to undo this? (e.g by using the image capture time from the image metadata to update the file creation time to match.  Not perfect, but better than being totally wrong thanks to LR)."

You could use the free Exiftool utility to set FileCreateDate and FileModifyDate from EXIF:DateTimeOriginal (the EXIF capture date). (Unfortunately, and very unusually, exiftool.org has been down the last couple of days.)

Or you could use the Mac app A Better Finder Attributes to copy EXIF:DateTimeOriginal to files' Date Created and Date Modified.

[Ian Lyons]

> Or you could use the Mac app A Better Finder Attributes to copy EXIF:DateTimeOriginal to files' Date Created and Date Modified.

BFA is an excellent and easy to use application for making date/time changes. Been using it since V2.x

[addison0]

Yes this just happened to me! And I am PISSED! Lightroom has gone through and changed all of my file modification and file creation times for practically every single photo in my catalog. Luckily not all of my external drives/network drives were connected at the time so it's only a few thousand files I need to fix. So much for Lightroom leaving the originals intact. I am now going to have to use an external software to go through and fix this disaster.

I am now having to rethink my entire photo workflow. I might just make all my photo folders read access only and then enable write access if I need to rename or write a specific edit to a certain file. Or create a separate user account and only use Lightroom through it to access the files and then create certain permissions on a user scale. Maybe I just need to find an entirely different software at this point.

It's also making me rethink having one large master catalog. Everything I read has people saying there's no use in splitting up catalogs into smaller ones since there's no performance increase and Lightroom can handle tons of photos at a time. I thought this made sense so I went with it for convenience of not having to close and reopen Lightroom anytime I needed to switch catalogs. Well now I don't feel like this is a smart idea at all. I don't feel l can even trust my files with this software since clicking one simple button can overwrite THOUSANDS of photos at once WITHOUT a prompt or notification. The least it could do is have a popup note that states "WARNING! .XMP files will only be created for raw image types. Otherwise, your metadata edits will overwrite the original file. Would you like to proceed?"  

This is such a major problem that truly needs to be fixed. I know everyone says that file creation/modification dates don't matter and the EXIF/IPTC info is all you need, but I know I'm not the only one whose workflow needs those creation/modification dates intact. I am so frustrated right now with this. And there is really no true undo option in this situation. Only going back through manually and setting it to match the photo time created. Huge oversight that Lightroom is marketed as a non-destructive way to edit your photos when in reality the simple click of a button can destructively change thousands and thousands of photos.