P: Photos are marked for republish even though there is no change.

[Dr. Fred Mbogo]

I noticed after updating my new v10 converted catalog that LrC was acting slow in the Library module, so I disengaged the usual culprit for this, the "Automatically write changes into XMP" setting. Then to my dismay, I noticed that it proceeded to mark the majority of the photos in the current grid view as needing a metadata update, even though the catalog metadata was fully saved to XMP prior to the upgrade, as far as I was aware.

Then I paged down, and the pattern repeated: Lightroom scanned all the now-visible photos and found that almost all of them also needed to be updated on disk.

And I did it again. And again.

Eventually I wrote a script to send "Page Down" events to the program periodically to ensure that Lightroom looked at every photo in the catalog, then let it run overnight with the library filter set to "Metadata Status = Up to date", so that it would give me the list of photos that need no metadata update. The next morning, I scrolled the grid back up to the top and let it go again, to catch any photos it missed on the first pass.

In the end, it marked over four-fifths of my catalog as out of date. This beggars belief, since I normally keep "Automatically write changes into XMP" engaged.

Then I did an experiment: I ran exiftool on one of the photos marked as still needing an update, saving the result to a text file, told Lightroom to save the metadata (⌘-S) and ran exiftool on the result, saving the output to a different text file, and diffed the two outputs, and found only timestamp and program version differences!

Observe:

5c5< File Modification Date/Time     : 2020:10:23 18:31:53-06:00---> File Modification Date/Time     : 2020:10:25 10:46:00-06:007c7< File Inode Change Date/Time     : 2020:10:23 18:31:53-06:00---> File Inode Change Date/Time     : 2020:10:25 10:46:00-06:0026c26< Instance ID                     : xmp.iid:9acec219-b6a4-4918-a592-9fc6a0ab3486---> Instance ID                     : xmp.iid:b5e76784-9638-44a1-a8b6-4e17799d003f28c28< Metadata Date                   : 2020:10:23 18:31:53-06:00---> Metadata Date                   : 2020:10:25 10:46-06:0040,41c40,41< History Instance ID             : xmp.iid:eac2ee0b-0b2d-4143-8da1-cafb424d66cc, xmp.iid:9acec219-b6a4-4918-a592-9fc6a0ab3486< History When                    : 2014:04:27 20:29:30-06:00, 2020:10:23 18:31:53-06:00---> History Instance ID             : xmp.iid:eac2ee0b-0b2d-4143-8da1-cafb424d66cc, xmp.iid:b5e76784-9638-44a1-a8b6-4e17799d003f> History When                    : 2014:04:27 20:29:30-06:00, 2020:10:25 10:46-06:00

For that particular file, the "Metadata Date" inside Lightroom is in July of 2019!

You may then wonder why the file modification time in the diff isn't in 2019, but instead two days ago. I dug into a recent pre-upgrade backup of my photos, and indeed, the prior file's mtime was in 2019. So, not only did LrC v10 decide it needed to do a bogus update to the file's metadata, it touched the file prior to actually being told it was okay to do so!

This upgrade has entirely invalidated all of my photo backups. Everything has to be backed up again, all because LrC is being stupid about touching files unnecessarily.

Surely the only defensible case for updating the application version number in the file's XMP metadata is that I've changed the photo, so now the program is properly reporting the last application to update the metadata?

I see other posts in the forum here on related topics, such as complaints that publish services are forcing a re-publish of unchanged photos. I'm posting this because I think I've diagnosed this to a deeper level than most users.

[pipurs]

Do you bu any chance have the files on a NAS?

[Dr. Fred Mbogo]

No, it's a local filesystem. However, it's ZFS, not APFS or HFS+. That's how I was able to dig back and find the file metadata prior to the upgrade: I mounted a snapshot and checked it.

I do understand your idea here, being that most NAS file sharing protocols do strange things with file metadata, but you can't extend that worry to ZFS. It is considerably more advanced than either HFS+ or APFS, but it does present a fully POSIX-compliant view to the host OS, unlike SMB, the most common problem source with files on a NAS.

[Schmidtze]

I had the same problem, that a lot of files have been marked as changed... For some files it was not possible to save the changes to disk. Immediately after doing it, it occured again as changed...

Strange!

Best regards

Friedemann

[kimballisms]

This upgrade has entirely invalidated all of my photo backups. Everything has to be backed up again, all because LrC is being stupid about touching files unnecessarily.

Ah, the joys of using DNGs and JPGs in Lightroom.

It blows my mind that Adobe can boast about LR being a parametric editor that doesn't touch original images while in fact it is constantly modifying original files.  Their hubris is off the charts.

[pipurs]

There has been some seriously digging into this ancient flaw during the years. I wish I could provide you a link to some of the threads, but the consensus is that it's a bug in how LR handles it. Something that your research seems to validate. 

I'm so tired of how increasingly buggy this software has become... 

With your apparent knowledge please try to locate the other threads - hopefully some rock solid evidence can come out of it which might make Adobe move their apparent lazy and ignorant asses someday...

[Dr. Fred Mbogo]

Yes, I've seen that symptom as well. I reported it separately as point #4 in this post.

[Dr. Fred Mbogo]

Yes, there should be an option for "external XMP" with JPG and DNG, not only to solve problems like the one in this thread, but also because it would keep my ZFS snapshots and external backup deltas small. It'd probably be quite a performance improvement to the Library module as well.

[kimballisms]

Strongly agree.  Sadly I believe this argument has been made many times before, and Adobe continues to think their method of re-writing our originals is the right way.

[Victoria Bampton LR Queen]

> It blows my mind that Adobe can boast about LR being a parametric editor that doesn't touch original images while in fact it is constantly modifying original files.

I fully support the request for sidecar files for all formats, however modifying the metadata of the originals files is completely optional, and was only made available for cross-app compatibility. Automatically write to XMP is not enabled by default and you can leave it turned off if you don't want to modify your originals.

[mk29376512]

Hi did the library module speed up for you once you did that, for myself and other users (see other threads) it is being very slow post v10 and looking for a fix?

[Schmidtze]

But I checked the difference which has been written, it's nothing important, no development change, it's simply the new xml version of LR 10 and a date. That's absolutely not necessary, to invalidate thousands of files. Also in the cloud, for me about 40.000 files had to be uploaded and downloaded again on my devices...

[Schmidtze]

I forgot to mention again, that after saving the metadata manually to the DNG files, they will appear again and again as changed. So that's of course a bug...

Best regards

Friedemann

[Dr. Fred Mbogo]

@Mphot43: Nope, it's still dog slow when working on the full catalog, and now I can't carve off a slice to get some reasonable speed back. This is deeply disappointing, Adobe!

[mk29376512]

They basically bricked the programme for me then, and because of the catalog update much harder to roll back because I slogged through a few projects already, crazy. I'm never auto-updating again. They had a really good track record up to this point in my book.

[alexskunz]

Could this be why photos in Publish Collections get marked as "changed" and requiring to be updated in the Publish service, in LrC 10?

I posted a "Problem" about that here: https://feedback.photoshop.com/conversations/lightroom-classic/lightroom-classic-10-marks-photos-as-...

[Sunil_Bhaskaran]

Thanks for the details.
We are investigating.

Thanks,
Sunil

[MusicalBarry]

Obviously still investigating as v10.2 doesn't seem any better 😞

[Dr. Fred Mbogo]

This ridiculous behavior is still happening with 10.3.

Just a few weeks ago, I went back through the same exercise that led to this thread with 10.0, this time using 10.2: turn off the automatically write XMP pref, then page through the catalog from start to end, with the result that Lightroom marked about 4/5 of my catalog as needing a metadata update. I then toggled the preference back on and let it write gigs of pointless metadata changes back to my disk and re-backed-up all of those "changed" photos.

Now we've got the 10.3 update, and it's looking to do it all over again! It'll take me days of processing to be sure, but I see the same symptom in the first batch of photos it's done this to:

33c33<  <System:FileModifyDate>2021:06:01 04:08:51-06:00</System:FileModifyDate>--->  <System:FileModifyDate>2021:06:12 12:12:01-06:00</System:FileModifyDate>35c35<  <System:FileInodeChangeDate>2021:06:01 04:08:51-06:00</System:FileInodeChangeDate>--->  <System:FileInodeChangeDate>2021:06:12 12:12:01-06:00</System:FileInodeChangeDate>74c74<  <XMP-xmp:MetadataDate>2021:06:01 04:08:51-06:00</XMP-xmp:MetadataDate>--->  <XMP-xmp:MetadataDate>2021:06:12 12:12:01-06:00</XMP-xmp:MetadataDate>171c171<  <XMP-xmpMM:InstanceID>xmp.iid:9ee46f3f-bef7-426b-baa4-587741f62058</XMP-xmpMM:InstanceID>--->  <XMP-xmpMM:InstanceID>xmp.iid:3228d987-e2b8-4d09-941e-9078edf18895</XMP-xmpMM:InstanceID>183c183<    <rdf:li>xmp.iid:9ee46f3f-bef7-426b-baa4-587741f62058</rdf:li>--->    <rdf:li>xmp.iid:3228d987-e2b8-4d09-941e-9078edf18895</rdf:li>189c189<    <rdf:li>2021:06:01 04:08:51-06:00</rdf:li>--->    <rdf:li>2021:06:12 12:12:01-06:00</rdf:li>195c195<    <rdf:li>Adobe Photoshop Lightroom Classic 10.2 (Macintosh)</rdf:li>--->    <rdf:li>Adobe Photoshop Lightroom Classic 10.3 (Macintosh)</rdf:li>

Replication method:

1. With the XMP pref turned off, run the command "exiftool /path/to/photo/marked/as/changed.jpg > a"
2. Turn the XMP pref back on and either wait for Lightroom to get around to saving the bogus changes or explicitly Save them (Cmd-S).
3. Run the same program as in step 1, but direct the output to a "b" file.
4. Run "diff a b", resulting in above output.

This diff shows that only meaningless date stamps, GUIDs, etc. are changing, plus the Lightroom version number.

I think that last part is the critical part: I believe Lightroom's internal metadata checker is saying, "Oh, the last program to write this XMP to disk was Lr 10.2, and I'm 10.3, so the metadata's changed! Let's rewrite the XMP data!"

Please stop this. I really really really don't want to back up 4/5 of all photos I've merely happened to look at in the interval between each Lightroom version upgrade.

[Dr. Fred Mbogo]

This pass ended up marking about two-thirds of my photos as "changed" even though I keep preference to keep the XMP up-to-date almost always engaged.

I could detect no pattern in the criteria it's using for this, so I not only do not know why so many photos are so-marked, I can't tell you why the remaining third were considered up-to-date.

[DVABAU]

I found several threads that mention the issue of the system marking photos for republishing by itself but I didn't find any fix...I can't be alone with this issue. It's absolutely maddening to try and work in on my published folders only to see all the photos start to move around as the system seems to arbitrarily decide to start marking photos to republish.  I've had the issue since I went on the subscription model.

I'm on a mac with the latest software.

Please help!

[John R Ellis]

The only workaround I'm aware of is from six years ago. Since it appears there could be multiple causes for these symptoms, that workaround may or may not work for you:

https://community.adobe.com/t5/lightroom-classic-bugs/p-photos-incorrectly-considered-as-quot-change... 

[tomtel49]

Hope I am posting this is the correct place....

This BUG (marking images a needing to be "re-published") is driving me nuts and really making LRC loose some of it value to me.   For a long while I have experienced this "bug" not really realizing the severity of it (more LRC editedphotos now than before).   Everytime I open LRC (ver 11.1 now) I find more and more photos needing to be "re-published".  

I can understand and accept a bug is s/w as long as it is fixed, but looking at all the postings this bug goes way back to to 2016 or ealier (think I read on posting from 2015) with LRC ver. 2015.6 - we are now on LRC ver 11.1 and 5-6 years later with no fix.  NOT fixing something like this after so long really does not have an excuse.   What is worse, one does not even know if images are being affected in other ways when being constantly marked for replublising.

I can have folders not show images needing to be republished and then turn around the next day/hour/week, etc. and then they will show up as needing to be replushing.   What is even worse I can have 1 or several images needing to be replushed and then "republish" them and get and according affect of images needing to be republished while updating.  It can just from a few images to a large number and back down and back up before completing.

I can mark the image as "Mark as up-to-date" and then will change their status - for this time period.  Open up LRC again, and the same (or some of the same) images are flagged as "Modified Photos to Re-Publish" again.

Needless to say, this is getting very, very tiring and wasting a lot of timeon managing one LRC catalogue.  MORE IMPORTANTLY - it is making one loose faith in LRC and almost looking for alternatives to managing/editing my photo catalogues - as much as I like the Adobe Photography package plan.

PLEASE, PLEASE FIX THIS ONCE AND FOR ALL .....

[tomtel49]

Following up on a reply from Rikk Flohr - thanks for moving this posting to the proper place.  As for your question about reproducing....   for me it is actually simple - just open LRC (any of the recent versions) and images in "Published Services". Make sure all are "Published" and then go back to LRC later and you (I and other according to post I find) will see "Modifiewd Photos to Re-Publish" above the "Published Photos".   One really has to do nothing other than open up LRC and check the images in Published Services.  IMPORTANT to note that it is not always the same images - although it can be the same folders.

As an example - I did a "Mark as Up-To-Date" on all the photos in a sub-folder of a folder earlier today, and this evening 11 images are requiring to be "re-published" - when all was OK earlier today.   If I "Re-Publish" rather than do A Mark as Up-To-Date" it will act like an according increasing the number of images and reducing the number of images and then increasing again - before re-publishing all.   Sample screen shots are attached.

[DVABAU]

Thansk but from what I read, the cure is worse than the disease and since it's 6 years later and it's still happening on the current version, it would seem that Adobe really doesn't care about it.

[DVABAU]

I tried the save metadata and read metadata trick to no avail. It immediately began marking photos for republishing.And certainly marking them all as up to date is a waste of time. It's funny, as I am wrining this in Chrome, I can see LR adding photos to be republished behind this window. It added about 50 while I typed this...an I'm not in LR right now!