P: Photos are marked for republish even though there is no change.

[Dr. Fred Mbogo]

I noticed after updating my new v10 converted catalog that LrC was acting slow in the Library module, so I disengaged the usual culprit for this, the "Automatically write changes into XMP" setting. Then to my dismay, I noticed that it proceeded to mark the majority of the photos in the current grid view as needing a metadata update, even though the catalog metadata was fully saved to XMP prior to the upgrade, as far as I was aware.

Then I paged down, and the pattern repeated: Lightroom scanned all the now-visible photos and found that almost all of them also needed to be updated on disk.

And I did it again. And again.

Eventually I wrote a script to send "Page Down" events to the program periodically to ensure that Lightroom looked at every photo in the catalog, then let it run overnight with the library filter set to "Metadata Status = Up to date", so that it would give me the list of photos that need no metadata update. The next morning, I scrolled the grid back up to the top and let it go again, to catch any photos it missed on the first pass.

In the end, it marked over four-fifths of my catalog as out of date. This beggars belief, since I normally keep "Automatically write changes into XMP" engaged.

Then I did an experiment: I ran exiftool on one of the photos marked as still needing an update, saving the result to a text file, told Lightroom to save the metadata (⌘-S) and ran exiftool on the result, saving the output to a different text file, and diffed the two outputs, and found only timestamp and program version differences!

Observe:

5c5< File Modification Date/Time     : 2020:10:23 18:31:53-06:00---> File Modification Date/Time     : 2020:10:25 10:46:00-06:007c7< File Inode Change Date/Time     : 2020:10:23 18:31:53-06:00---> File Inode Change Date/Time     : 2020:10:25 10:46:00-06:0026c26< Instance ID                     : xmp.iid:9acec219-b6a4-4918-a592-9fc6a0ab3486---> Instance ID                     : xmp.iid:b5e76784-9638-44a1-a8b6-4e17799d003f28c28< Metadata Date                   : 2020:10:23 18:31:53-06:00---> Metadata Date                   : 2020:10:25 10:46-06:0040,41c40,41< History Instance ID             : xmp.iid:eac2ee0b-0b2d-4143-8da1-cafb424d66cc, xmp.iid:9acec219-b6a4-4918-a592-9fc6a0ab3486< History When                    : 2014:04:27 20:29:30-06:00, 2020:10:23 18:31:53-06:00---> History Instance ID             : xmp.iid:eac2ee0b-0b2d-4143-8da1-cafb424d66cc, xmp.iid:b5e76784-9638-44a1-a8b6-4e17799d003f> History When                    : 2014:04:27 20:29:30-06:00, 2020:10:25 10:46-06:00

For that particular file, the "Metadata Date" inside Lightroom is in July of 2019!

You may then wonder why the file modification time in the diff isn't in 2019, but instead two days ago. I dug into a recent pre-upgrade backup of my photos, and indeed, the prior file's mtime was in 2019. So, not only did LrC v10 decide it needed to do a bogus update to the file's metadata, it touched the file prior to actually being told it was okay to do so!

This upgrade has entirely invalidated all of my photo backups. Everything has to be backed up again, all because LrC is being stupid about touching files unnecessarily.

Surely the only defensible case for updating the application version number in the file's XMP metadata is that I've changed the photo, so now the program is properly reporting the last application to update the metadata?

I see other posts in the forum here on related topics, such as complaints that publish services are forcing a re-publish of unchanged photos. I'm posting this because I think I've diagnosed this to a deeper level than most users.

[robertv47531707]

And you don't think anyone from Adobe reads these forums?

[John R Ellis]

@robertv47531707: "And you don't think anyone from Adobe reads these forums?"

Senior Adobe employee Rikk Flohr actively moderates this forum and curated this thread, linking it with the internal bug report.

[DClark064]

Does anybody know if using Friedl's Smugmug publish module eliminates this issue?

I believe I saw someplace that it did but I don't want to read seven pages of posts.

[John R Ellis]

These symptoms have been reported for many different publish services (each with its own plugin). Personally, my Flickr (Adobe plugin) and Zenfolio (Friedl plugin) services are affected. There's a report about Smugmug, though it doesn't specify which plugin. I'd be very surprised if the Friedl Smugmug plugin didn't experience the same problems.

[DClark064]

I found it.  dj_paige reported Friedl's Smugmug did not show the problem, https://community.adobe.com/t5/lightroom-classic-discussions/lightroom-creates-duplicates-when-repub... 

That's another thread that should probably be merged with this one.

[kimaldis]

That doesn't look l ike the same problem to me.

[Kim Kruse]

Adobe reading it: Yes. Adobe create Corrective Actions: No

[Aaron Roberts]

For those asking about Friedl plugins. It seems that the one that I have, has a dedicated area which is used to avoid republishing. I don't know when this feature was added, but it certainly seems to have stabalised things for me.

 

[DClark064]

Ooops, in my two comments above I should have said Flickr not Smugmug. 

I find that my Adobe Smugmug publish module keeps marking large numbers of files to be republished, but if I do republish no duplicates are created.  Republishing is very slow and aggravating, but it does not make a mess of the web site.  The Adobe Flickr pubish module does make a mess of the web site with disconnected duplicates, triplicates, etc.  So I meant to inquire whether Friedl's Flickr publish module avoids the duplicates. 

[John R Ellis]

[This post contains formatting and embedded images that don't appear in email. View the post in your Web browser.]

@Aaron Roberts: "For those asking about Friedl plugins. It seems that the one that I have, has a dedicated area which is used to avoid republishing. I don't know when this feature was added, but it certainly seems to have stabalised things for me."

With the Friedl Zenolio plugin, I've disabled all the metadata fields that could trigger republishing, but that doesn't help my situation:

Note that it says, "Develop updates always trigger a republish" -- i.e. you can't disable that.  There may be multiple causes of the republishing problem, but over the years one common theme is that rendering a photo (e.g. for Library previews or for export) can trigger a false republish.  At least a couple of the instances have been caused by LR doing incremental upgrades of the internal catalog representation of develop settings (the upgrades can happen on the fly as LR accesses photos, rather than all at once after a catalog upgrade or a new version of LR is installed).

After poking around this for many years, my educated guesses:

- There is an architectural design flaw with the internal mechanism used for tracking photo changes.

- The current developers don't know how to use the mechanism.

- The mechanism works but is expensive (in engineeering costs) to maintain as LR changes the internal representation of Develop settings.

Regardless, it's factually true that Adobe has not prioritized a fix.

[DVABAU]

It's been my experience that there is nothing the user does or doesn't do that causes the issue so there's no point in trying to stop it. In my case, there have been far too many instances where I'm working on a picture, after marking all photos as published, and I'll still see random photos added to the republish list by the time I'm finished with the editing of the one photo.

[sebmsg]

25,000 photos in 463 rcTreesync mirrored collections. Somewhere I have a photo that has a missing original that's giving the annoying use smart previews prompt but I can't find it with metadata matching to FindMissingOriginals vs the published services. And I need to find it because it's breaking out of new code I've had written for Treesync to bypass missing orginals (eg. old hard drive disconnected). So I am going through all collections manually publishing. And at each one of 463 collections I'm having to wait whilst it decides that all of the photos need republishing (three to five minutes for the big collections), and then hitting publish now.  This bug is very boring.