P: "Save Metadata to File" overwrites EXIF tag SubSecTimeOriginal

[dansmith21]

Issue: When I save metadata from Lightroom Classic via "Save Metadata to File", I lose subsecond EXIF metadata that was originally there in the file. This is new behavior, did not happen in previous versions (though I'm not sure exactly when it started).

Lightroom Classic version: 13.5 [ 202408062022-6258095b ]

System version: macOS 13.6.9

To reproduce:

1) Import a photo that has a subsecond creation date—for my test, I used an iPhone photo imported via mobile sync.

2) Check SubSecTimeOriginal from the command line:

$ -> exiftool -G -s -SubSecTimeOriginal /Volumes/Media/Photos/Import/Mobile\ Import/IMG_6067.HEIC 

[EXIF]          SubSecTimeOriginal              : 389

3) In LrC, run the "Save Metdata to File" command

4) Check SubSecTimeOriginal from the command line:

$ -> exiftool -G -s -SubSecTimeOriginal /Volumes/Media/Photos/Import/Mobile\ Import/IMG_6067.HEIC 

Warning: [minor] Bad format (16) for MakerNotes entry 14 - /Volumes/Media/Photos/Import/Mobile Import/IMG_6067.HEIC

(That warning does not seem directly connected to the issue, but may be a sign of something unexpected happening...)

[dansmith21]

Clarification on (3): I don't believe the command does anything until you change some metadata first. I think I flagged or unflagged the file as part of this step.

[John R Ellis]

Hmm, following your steps, I don't see the misbehavior -- EXIF:SubjectTimeOriginal is preserved, with Iphone HEICs, JPEGs, and .xmp sidecars of .CR3s.

In these cases, I've found the most reliable way of demonstrating a bug is by sharing a small test catalog including the photo(s) illustrating the problem.

1. Select one or more problem photos that haven't yet been polluted by Save Metadata To File.

2. Do the menu command File > Export As Catalog with just the option Export Negative Files checked.

3. Zip up the exported catalog folder and upload it to Dropbox, Google Drive, or similar, and post the sharing link here.

Adobe generally won't pay attention unless they can easily reproduce the problem.

[dansmith21]

Here's a test library:

https://www.icloud.com/iclouddrive/0b3ffGjOxRmVDHtdo6YhcvalA#subsecond-test

I did notice some intermittency while testing. But it seems I can pretty reliably reproduce by:

1) Download & unzip the library

2) Inspect the files

$ -> cd ~/Downloads/subsecond-test/Photos/Import/Test/

$ -> exiftool -G -s -SubSecTimeOriginal IMG_6067.HEIC IMG_6097.HEIC IMG_6175.HEIC 

======== IMG_6067.HEIC

[EXIF]          SubSecTimeOriginal              : 389

======== IMG_6097.HEIC

[EXIF]          SubSecTimeOriginal              : 747

======== IMG_6175.HEIC

[EXIF]          SubSecTimeOriginal              : 366

    3 image files read

3) Open the library, select all 3 files, edit metadata (I remove the "Review Place" keyword), do "Save Metadata to Files"

4) Inspect the files again

$ -> exiftool -G -s -SubSecTimeOriginal IMG_6067.HEIC IMG_6097.HEIC IMG_6175.HEIC 

======== IMG_6067.HEIC

Warning: [minor] Bad format (16) for MakerNotes entry 14 - IMG_6067.HEIC

======== IMG_6097.HEIC

Warning: [minor] Bad format (16) for MakerNotes entry 13 - IMG_6097.HEIC

======== IMG_6175.HEIC

[EXIF]          SubSecTimeOriginal              : 366

    3 image files read

[John R Ellis]

@Rikk Flohr: Photography, please consider moving to Bugs -- I get the same misbehavior following the bug recipe with the sample catalog.

[Rikk Flohr: Photography]

Updating Status - adding bug number

[John R Ellis]

[This post contains formatting and embedded images that don't appear in email. View the post in your Web browser.]

@dansmith21, LR normally preserves EXIF:SubSecTimeOriginal in the catalog and uses it internally for sorting images in Grid view, though the only place it displays it is in the Date Created field of the Metadata panel.  In your test catalog (before doing Save Metadata To File), the fractional time isn't displayed for the two problem images, though it is for the third:

This indicates the problem is likely in reading the metadata from the file on import, rather than writing it back with Save Metadata To File.

Also, I noticed that the files had been modified with Exiftool 12.70, probably with the  plugin dansmith.customizations.

- Does the problem occur with virgin unmodified files straight from your Iphone?  I don't see any anomalies or standards nonconformance with the photos in the test catalog, but I recall in years past a few times some bugs where LR choked on files whose metadata was modified by other apps in ways LR wasn't expecting.

- The latest version of Exiftool is 12.93. Does the problem occur with that version?

[John R Ellis]

Also, be sure to preserve the uploaded catalog at that Icloud link until Adobe fixes the issue. They generally download sample files until they start working on a problem.

[dansmith21]

> I noticed that the files had been modified with Exiftool 12.70, probably with the  plugin dansmith.customizations

Yep, that's my workflow to manage date/time metadata.

> Does the problem occur with virgin unmodified files straight from your Iphone?

I'll have to experiment more with the entire workflow to figure out at exactly what point the issue comes up, but it's not just one corrupt file or something—I've got whole batches of photos that came in from iPhone and ended up in this same state. (Good news, then, is that I should hopefully be able to reproduce the entire workflow again...)

[dansmith21]

Here's a second library, this one with some images that came directly through mobile sync from my iPhone (so, untouched by exiftool). Follow the steps, get the same behavior:

https://www.icloud.com/iclouddrive/0c4PlWAGhrVAY-JWaL7NP148Q#subseconds-test2

[dansmith21]

And here's a third test library:

https://www.icloud.com/iclouddrive/00cGMltBWAWrhkR6nCopUw2Ow#subseconds-test3

These images came directly from the iPhone without going through mobile sync. (Specifically: took the photo, hit the "Share" button in the Photos app and exported to my iCloud storage, then manually imported to my LrC library.) They show subseconds in the "Date Created" LrC field, and do not overwrite the SubSecTimeOriginal tag on "Save Metadata to File".

So it seems there's something about the metadata that is being corrupted by the mobile sync process? When these last shots show up via mobile sync (it's a painfully slow process), I'll try to compare their exiftool tags and see if I notice any anomalies.

[dansmith21]

Ok, looking at the same identical photo side-by-side, imported two ways: (1) directly (as in subseconds-test3), and (2) via mobile sync (as in subseconds-test2):

- The exiftool output is identical (other than directories and file modify/access times)

- The LrC "Date Created" field is different—the first says "2024-08-27T10:05:36.130-07:00", the second says "2024-08-27T10:05:36-07:00"

[John R Ellis]

Good detective work. I did some more testing, and EXIF:SubSecTimeOriginal is getting dropped by the sync protocol uploading from the clients (LR Mobile and LR Desktop) to LR Cloud. 

This path demonstrates that the IOS Camera App sets both SubSecTimeOriginal and SubSecTimeDigitized:

Camera App > Share Airdrop to Mac: ...Original and ...Digitized preserved

This path demonstrates that the LR Mobile app preserves both fields when it imports from the camera roll:

Camera App > LR Mobile > Export As DNG > Share Airdrop to Mac: ...Original and ...Digitized preserved

This path demonstrates that the LR Desktop app preserves both fields when it first imports from a file on the Mac:

file > LR Desktop > Share Export Original: ...Original and ...Digitized preserved

This path demonstrates that LR Cloud itself preserves both fields (LR Web doesn't use the sync protocol -- it uses an HTTP REST protocol to access the LR Cloud):

file > LR Web upload > LR Web download: ..Original and ...Digitized preserved

These paths demonstrate it's the sync protocol uploading from the clients to the cloud that's dropping SubSecTimeOriginal:

Camera App > LR Mobile > Sync > LR Web > Download: ...Original not preserved, ...Digitized preserved

file > LR Desktop > Sync > LR Web > Download: ...Original not preserved, ...Digitized preserved

* * * 

Tested with Iphone 11 Pro / IOS 17.3, Lightroom IOS 9.5.0, Lightroom Desktop 7.5 / Mac OS 14.6.1, Lightroom Classic 13.5.

[John R Ellis]

@Rikk Flohr: Photography, more testing by @dansmith21 and myself indicate the bug is in the sync between LR clients (Mobile and Desktop) and LR Cloud.  Thus this bug should be refiled accordingly, thanks.

[Rikk Flohr: Photography]

@John R Ellis  Does it not involve Classic?

[John R Ellis]

"Does it not involve Classic?"

No. It's the sync upload from LR Mobile and from LR Desktop to LR Cloud that drops the EXIF capture time's fractional seconds. The apps themselves (LR Mobile, LR Desktop, LR Web, LR Classic) and LR Cloud itself all preserve fractional seconds, and the sync upload preserves the digitization time's fractional seconds.

[Rikk Flohr: Photography]

Moving to Lightroom Eco

Moving back to Lightroom Classic

[Rikk Flohr: Photography]

I have rerouted the existing bug to the proper place. Thanks for the head's up!

[dansmith21]

One additional odd thing to observe is that the LrC "Read Metadata from File" does not seem to shake files out of this broken state. Once the catalog has decided they don't have subsecond creation times, there doesn't seem to be anything I can do with the file metadata to correct that. (Eventually, besides addressing the bug for future synced images, I'd really love to know what to do to get my existing images out of this broken state!)

[John R Ellis]

"LrC "Read Metadata from File" does not seem to shake files out of this broken state."

That's because the sync from LR Mobile to LR Cloud didn't preserve EXIF:SubSecTimeOriginal. Then when LR Classic synced the images to your computer, they were already missing that field, so Read Metadata From File won't do anything.

"I'd really love to know what to do to get my existing images out of this broken state!"

My testing shows that the sync preserves the EXIF Date Time Digitized and that the Cameras app sets Date Time Original and Date Time Digitized to the same value. (Beware that Exiftool confusingly uses the non-standard name EXIF:CreateDate for the standard name EXIF:DateTimeDigitized.)

So you could do this recovery:

1. In LR, select all the possibly affected photos.

2. Do Metadata > Save Metadata To File.

3. At the command line, use Exiftool to copy EXIF:SubSecTimeDigitized to EXIF:SubSecTimeOriginal for all the selected photos.

4. In LR, do Metadata > Read Metadata From File.

[dansmith21]

> That's because the sync from LR Mobile to LR Cloud didn't preserve EXIF:SubSecTimeOriginal. Then when LR Classic synced the images to your computer, they were already missing that field, so Read Metadata From File won't do anything.

No, in my testing I can manually add back SubSecTimeOriginal, do "Read Metadata From File", and it doesn't matter—the catalog internal metadata does not get updated, and when I later do "Save Metadata to File", it overwrites my change.

[John R Ellis]

"in my testing I can manually add back SubSecTimeOriginal, do "Read Metadata From File", and it doesn't matter—the catalog internal metadata does not get updated"

Poking into this further, sync is doing two things to photos it uploads to the LR Cloud:

- Removing EXIF:SubSecTimeOriginal

- Adding an XMP metadata section containing, among other fields, XMP:DateCreated set to the capture date without fractional seconds.

When both EXIF:DateTimeOriginal and XMP:DateCreated are present, the Metadata Working Group standard (which LR follows in general) says the EXIF field should be preferred. But LR is preferring the XMP field, contrary to the standard. So when you do Read Metadata From File, LR reads the capture time from XMP:DateCreated (from which sync removed fractional seconds).

Thus step 3 of my workaround should be:

exiftool -use mwg "-composite:datetimeoriginal<xmp:createdate" filename

This sets the EXIF and XMP fields representing capture date to XMP:CreateDate (date/time of digitization), from which sync hasn't removed fractional seconds. (I've tested this step.)

I can't believe my head is filled with this inane legacy industry crap.

[dansmith21]

I've done more testing and found an additional problem.

Sure, something happens during sync to mess up the metadata. But that's ok, I can fix it.

The problem I'm having is: after I fix up all the metadata, I do "Read Metadata from File", and then in the LrC UI, the "Date Created" field says "2024-02-03T12:13:09.675-08:00". Everything seems to be in sync. But about 50 seconds later, the field reverts to "2024-02-03T12:13:09-08:00"—without any action on my part. Turns out if I pause syncing, though—cloud icon in the top right corner of the LrC window—this doesn't happen.

So: not only is mobile sync removing the subsecond metadata, it is actively overwriting my local changes after I try to fix it. Oddly, it will let me make changes to the seconds or time zone—those changes will stick. Just not the subseconds.

[dansmith21]

Now that I know what to look for, I'm noticing it elsewhere: for photos with a blank GPS field, I can set it manually, but sync comes in 50 seconds later and blanks it out again.