why is adobe folder synchronization still broken?

Report · May 27, 2019

why is adobe LR 8.3 changing images when you synchronize a folder?

why is it touching images that are not edited/changed in any way?

i actually searched the forum and i read this a few times .... and it´s a PITA.

i just synchronized a folder and LR was touching EVERY images in the folder and subfolders and changed it´s timestamp.

the images are not changed in any way by me. so in my opinion LR has to do NOTHING to these images. yet they are all set to 27.05.2019.

that means that my automatic backup will copy them over to all my backup drives.

can someone explain this to me?

these are the changes LR has made to the images:

all i have done is synchronizing a folder i have not touched in month.

i read that some people had thousands of images that are altered.
that means that hundreds of gigabytes will be be copied to backup drives because the synchronize feature is basically broken.

Report · May 28, 2019

johnrellis schrieb
Over the years, there have been a number of releases where LR automatically updates catalog entries for existing photos incrementally in background as you do various operations (scrolling through thumbnails, editing photos). Sometimes this has generated updates to the metadata (both Library and Develop settings). That may be what's happening here, either by intentional design or not.

how do you explain the fact that only the parts of metadata shown in the screenshots is changed then?

that honestly makes no sense.

But without two sample before and after photos,

as i said it happend on 16000+ images when i tested it on a 32000+ image test catalog.

i posted the differences. they are the same for all images.

how should having a random couple of images out of the 16000 images help nailing down the issue?

there is nothing special on these images.

the images are from these dozend of different cameras (all major brands, canon, nikon, sony included):

Report · May 28, 2019

how do you explain the fact that only the parts of metadata shown in the screenshots is changed then?

I won't try to explain anything without having full access to the image and metadata.

Report · May 28, 2019

johnrellis schrieb
how do you explain the fact that only the parts of metadata shown in the screenshots is changed then?
I won't try to explain anything without having full access to the image and metadata.

here you go:

http://www.filedropper.com/temp_1

while another person, using different tools and with different experience, will see important clues. This is basic engineering practice.

a binary compare should find all differences. but hey prove me wrong.

Report · May 28, 2019

I have at home two copies made from an XMP sidecar file (for Raw) - I cannot access those at present. One copy ("before") was made just after changing this sidecar file in a text editor - I just typed a space, saved, deleted the space, saved again. This naturally changed the file modification date, and made it different than it had been when LR last wrote out XMP - which, I imagine, would have also been reported in the metadata timestamp within this XMP data.

Then I ran a folder sync in LR, with metadata scan. Clearly LR saw some difference in this particular file as compared to the others in the folder - perhaps the file modification date differing from the metadata date within - because the file modification details on just this image, then updated to reflect the day and time when the folder sync happened.

I then made a second copy ("after").

Comparing these "before sync" and "after sync" copies, their text content is identical so far as I can tell, including the metadata timestamps within the text. If LR had decided there was a change to metadata, I would have expected this timestamp to renew also. If LR had decided there was no change to metadata, I am not sure why it should have re-written (identical) XMP out again to the file system?

Report · May 28, 2019

With those two sample TIFFs, here are the metadata differences identified by "exiftool -a -G":

< [File] File Name : Burgruine Hanstein_0007 original.tif
> [File] File Name : Burgruine Hanstein_0007.tif
< [File] File Modification Date/Time : 2018:02:17 05:10:51-08:00
> [File] File Modification Date/Time : 2019:05:27 06:02:54-07:00
< [XMP] Metadata Date : 2018:02:17 14:10:51+01:00
> [XMP] Metadata Date : 2019:05:27 15:02:54+02:00
< [XMP] Instance ID : xmp.iid:f71f7f2e-ad1b-e745-85a8-28f56692f725
> [XMP] Instance ID : xmp.iid:eb3f0c53-28b2-244a-84ab-c3d9eb8f59c0
< [XMP] History Instance ID : xmp.iid:10e76656-82fa-b542-981a-fcef8304bebd, xmp.iid:17112483-14d4-c14b-ba64-29cc04613f00, xmp.iid:7c2604e3-2a0e-544d-a7e4-971391a0c139, xmp.iid:f71f7f2e-ad1b-e745-85a8-28f56692f725
> [XMP] History Instance ID : xmp.iid:10e76656-82fa-b542-981a-fcef8304bebd, xmp.iid:17112483-14d4-c14b-ba64-29cc04613f00, xmp.iid:7c2604e3-2a0e-544d-a7e4-971391a0c139, xmp.iid:eb3f0c53-28b2-244a-84ab-c3d9eb8f59c0
< [XMP] History When : 2014:10:31 18:21:53+01:00, 2014:10:31 18:23:06+01:00, 2014:11:02 13:14:35+01:00, 2018:02:17 14:10:51+01:00
> [XMP] History When : 2014:10:31 18:21:53+01:00, 2014:10:31 18:23:06+01:00, 2014:11:02 13:14:35+01:00, 2019:05:27 15:02:54+02:00
< [XMP] History Software Agent : Adobe Photoshop Camera Raw 8.6 (Windows), Adobe Photoshop CC 2014 (Windows), Adobe Photoshop Lightroom 5.6 (Windows), Adobe Photoshop Lightroom Classic 7.2 (Windows)
> [XMP] History Software Agent : Adobe Photoshop Camera Raw 8.6 (Windows), Adobe Photoshop CC 2014 (Windows), Adobe Photoshop Lightroom 5.6 (Windows), Adobe Photoshop Lightroom Classic 8.3 (Windows)

The History fields are curious:

The last entry for Lightroom 8.3 has replaced the entry for Lightroom 7.2, which is unexpected. That could be a symptom of a LR bug, or it could indicate a more complicated modification history for the two files.

The history doesn't indicate the changes made by the unknown geotagging app. It's possible that the geotagging app made changes to the metadata (perhaps at a low binary level) that were incompatible with LR (i.e. because one or the other wasn't strictly following the standard), and that confused LR. This has happened many times before. But that's just educated guesswork at this point -- the unknown geotagging app could just as likely be irrelevant to this, considering that Richard has reproduced the problem, presumably without that app. But I wouldn't rule the interaction of that app yet.

I have some other avenues to explore for reproducing the problem.

Report · May 28, 2019

I don't have all those previous versions of Camera Raw and LR (and Adobe's new policy prevents me from getting them). But I tried the following, which didn't reproduce the problem:

1. Imported "Burgruine Hanstein_0007 original.tif" into a new LR 5.7 catalog, with Automatically Write Changes Into XMP enabled.

2. Opened that LR 5.7 catalog in LR 8.3, converting it.

3. Verified that Automatically Write was enabled in LR 8.3.

4. Synchronized the folder with Scan For Metadata Updates.

I tried converting the 5.7 catalog to 2015.14 (6.14) and then converting that to 8.3, but that didn't cause the problem either.

Report · May 28, 2019

The XMP:HistorySoftwareAgent field of your sample files provided an important clue as to the prior versions of LR to examine (7.2 and nearby versions). I am able to reproduce the problem in 8.3 with most JPEGs that were last edited with 7.2 and 7.1, and with a few of the JPEGs edited by 7.3. I wasn't able to reproduce the problem at all with raws last edited with those versions.

Using "exiftool -a -G" to examine the metadata before and after the Synchronize Folder command, LR was adding these metadata fields in JPEGs last edited in 7.1 and 7.2 (in addition to the Metadata Date and History fields):

XMP:ToneMapStrength, XMP:OverrideLookVignette, XMP:LookName, XMP:CameraProfileDigest, XMP:Texture

All except the last are fields that were introduced in 7.2 in support of the new enhanced profiles. XMP:Texture is a new develop setting introduced in 8.3.

In the photos last edited in 7.3, only XMP:Texture was edited.

This is similar to other behavior observed over the years: When LR introduces new develop settings, process versions, or other metadata fields, it incrementally updates the cataloged photos on demand when it "touches" those photos. For example, I recall that when a new process version was introduced, LR sometimes updated photos as you scrolled through Library grid view and it regenerated previews.

It appears that the Scan For Metadata Updates command requests from the catalog the photo's metadata, which in turn triggers the "update to the latest metadata fields" logic. That adds the new enhanced profile fields (for photos last edited with 7.2 or earlier) and XMP:Texture (for photos last edited before 8.3). Then, since the metadata fields have changed and Automatically Write Changes is enabled, LR writes the new fields to the photo.

With your sample photo, none of those new develop settings were added. I don't know why that was, but I'm pretty confident that its metadata was written as a consequence of the same update logic.

You could file a revised bug report on this in the official Adobe feedback forum, but I think it's likely that Adobe will ignore it. This is the way they've done metadata updates for new versions for many years, and given the current level of support for LR Classic and that relatively few people have complained about this, I doubt Adobe would prioritize a change. (Again, don't shoot the messenger.)

Adobe might be slightly more willing to consider the bug report if you presented them with a precise recipe and sample files that would allow the engineers to reproduce the behavior. (Without an easy way to reproduce the behavior, I can guarantee that they'll ignore the bug report.) You could supply a copy of your .lrcat file (not the entire catalog folder), along with a folder containing a few of the photos that illustrate the behavior. They could open that catalog, relink the photos to the new location of the folder, and then do Synchronize Folder. But you'd need to spell out each step in exact detail and test out those steps. To include the .lrcat file and folder in your bug report, zip them, upload the .zip to Dropbox or similar, and include the sharing link the bug report.

Report · May 29, 2019

johnrellis schrieb
Using "exiftool -a -G" to examine the metadata before and after the Synchronize Folder command, LR was adding these metadata fields in JPEGs last edited in 7.1 and 7.2 (in addition to the Metadata Date and History fields):
XMP:ToneMapStrength, XMP:OverrideLookVignette, XMP:LookName, XMP:CameraProfileDigest, XMP:Texture

i have checked a few more of my changed files and none have these added metadata fields.

With your sample photo, none of those new develop settings were added. I don't know why that was, but I'm pretty confident that its metadata was written as a consequence of the same update logic.

why?

it is just changing the DATE.

why changing the date of thousands of files for no reason.

that is a bug if you ask me. it messes with peoples file updates

Report · May 29, 2019

Lightboxx schrieb
The history doesn't indicate the changes made by the unknown geotagging app. It's possible that the geotagging app made changes to the metadata (perhaps at a low binary level) that were incompatible with LR (i.e. because one or the other wasn't strictly following the standard), and that confused LR.

as i wrote the geotagging app has nothing to do with this.
not all of the 16000 file are geotagged.

i only geotag my travel photos, not studio shots, macros etc.

pure coincidence that the example files are geotagged.

Adobe might be slightly more willing to consider the bug report if you presented them with a precise recipe and sample files that would allow the engineers to reproduce the behavior.

knowing adobe and seeing bugs in photoshop CC 2019 i (and millions of others) noticed in 2014 i will not waste my time.

Report · May 28, 2019

johnrellis wrote
Over the years, there have been a number of releases where LR automatically updates catalog entries for existing photos incrementally in background as you do various operations (scrolling through thumbnails, editing photos). Sometimes this has generated updates to the metadata (both Library and Develop settings). That may be what's happening here, either by intentional design or not.

One possibility is that Adobe created a new camera or lens profile and set it to be automatically applied. Adobe names the new profile v2 and normally it has to be applied manually by the user to update the image files. However, there have been a few cases where it was early in the camera's production and Adobe applied the new v2 profile automatically without any warning during the LR version update. Here are two Canon camera examples where this has happened. I'm sure there are others.

Re: Camera Profile missing

BUG!!! Profile changes to V2 automatically on previously edited files! possible to prevent?

Lightboxx what camera model and lenses were used for the files that are changing? Check the DNG file copy XMP metadata for Camera Profile and Lens Profile names and see if they are different than the names showing inside LR's Develop module.

Report · May 28, 2019

https://forums.adobe.com/people/Todd+Shaner schrieb
johnrellis wrote
Over the years, there have been a number of releases where LR automatically updates catalog entries for existing photos incrementally in background as you do various operations (scrolling through thumbnails, editing photos). Sometimes this has generated updates to the metadata (both Library and Develop settings). That may be what's happening here, either by intentional design or not.

again... why is nothing of that written to the files then?

you can see all the changes that are made to the files. no changed camera profiles nothing, no changed lens profiles.

this guessing is all fine but it should be based on the info that is provided.

and as far as i see the metadata changes are not supporting this:

Lightboxx what camera model and lenses were used for the files that are changing? Check the DNG file copy XMP metadata for Camera Profile and Lens Profile names and see if they are different than the names showing inside LR's Develop module.

i wonder if you guys really read what is writen here?

in my post above i even posted a screenshot of the cameras in question.

as for lenses... around 50-80 different lenses i guess.

for canon alone i had around 20 over the years.

the XMP data in the file is unchanged.

it´s is the same as in the LR catalog and the changed and original files are the same except for the changes i posted in the screenshots.

Report · May 28, 2019

With all due respect, it helps to have multiple people examining image files and metadata. One person can be absolutely certain that it's "unchanged" and "the same as in the catalog", while another person, using different tools and with different experience, will see important clues. This is basic engineering practice. I have not infrequently been absolutely certain there was nothing there to be seen, but someone else comes along and spots something.

If you don't want to provide us with two sample before and after images, fine, but I'm not going to waste more of my time trying to help improve LR on this issue without them.

Report · May 27, 2019

You also need to provide the name of the geotagging app and how you use it. If you can duplicate the issue with a workflow that uses only Adobe apps (LR, PS Bridge) so much the better!

it has nothing to do with the geotagging app.

the geotagging app is the REASON i want to syncronize a folder but it has nothing to do with the files that are changed.

the same happens when i synchronize folders that have not a single geotagged file in it.

Report · May 27, 2019

If you check (optionally) "scan for metadata updates" then your Sync Folder has carried out a check and compare on all images. You may feel that this is not a per-se alteration, but it is a potential one at least; and, an event in the image's history. So if you have got LR (optionally) set to "automatically write XMP" then a timestamped record of having performed this metadata scan check, is apparently written to the file, hence the Modified date is altered.

If that file's a proprietary Raw then this has only affected a small sidecar file which is not such a huge issue to re-write to backup... assuming you are including these even. Sure, DNG will be a much bigger issue practically speaking.

If your only reason for doing "Sync Folder" was in order to see whether there were new images that may need to be imported - then you could omit this "scan" and the metadata / file timestamps should be left alone otherwise.

Or, you could try a conventional Import instead, excluding already-imported images from that.

If you actively did want to scan for external changes to metadata, then I would have to ask: what is the overall workflow in which this is happening?

Report · May 27, 2019

richardplondon schrieb

If you actively did want to scan for external changes to metadata, then I would have to ask: what is the overall workflow in which this is happening?

i edit RAW files in LR then i send them to photoshop for further editing.

from photoshop i save a copy as TIFF files into the appropriate folders.

sometimes i need to re-import (synchronize) these files because i do edits with a geotagging app (lightrooms geotagging is subpar).

so i want LR to check for metadata changes... but i don´t want it to touch all the files and change them.

ou may feel that this is not a per-se alteration, but it is a potential one at least; and, an event in the image's history. So if you have got LR (optionally) set to "automatically write XMP" then a timestamped record of having performed this metadata scan check, is apparently written to the file, hence the Modified date is altered.

your explanation could be the reason... nevertheless it is absolutely stupid if you ask me (i guess the other people who complained about it would agree).

imagine you do this for a root folder of your image database.

that would mean that an backup program would copy the whole image database (in my case 120000+ images) to the backup drives.

for nothing.. only because you wanted to synchronize a few files.