P: Photos are marked for republish even though there is no change.

[Dr. Fred Mbogo]

I noticed after updating my new v10 converted catalog that LrC was acting slow in the Library module, so I disengaged the usual culprit for this, the "Automatically write changes into XMP" setting. Then to my dismay, I noticed that it proceeded to mark the majority of the photos in the current grid view as needing a metadata update, even though the catalog metadata was fully saved to XMP prior to the upgrade, as far as I was aware.

Then I paged down, and the pattern repeated: Lightroom scanned all the now-visible photos and found that almost all of them also needed to be updated on disk.

And I did it again. And again.

Eventually I wrote a script to send "Page Down" events to the program periodically to ensure that Lightroom looked at every photo in the catalog, then let it run overnight with the library filter set to "Metadata Status = Up to date", so that it would give me the list of photos that need no metadata update. The next morning, I scrolled the grid back up to the top and let it go again, to catch any photos it missed on the first pass.

In the end, it marked over four-fifths of my catalog as out of date. This beggars belief, since I normally keep "Automatically write changes into XMP" engaged.

Then I did an experiment: I ran exiftool on one of the photos marked as still needing an update, saving the result to a text file, told Lightroom to save the metadata (⌘-S) and ran exiftool on the result, saving the output to a different text file, and diffed the two outputs, and found only timestamp and program version differences!

Observe:

5c5< File Modification Date/Time     : 2020:10:23 18:31:53-06:00---> File Modification Date/Time     : 2020:10:25 10:46:00-06:007c7< File Inode Change Date/Time     : 2020:10:23 18:31:53-06:00---> File Inode Change Date/Time     : 2020:10:25 10:46:00-06:0026c26< Instance ID                     : xmp.iid:9acec219-b6a4-4918-a592-9fc6a0ab3486---> Instance ID                     : xmp.iid:b5e76784-9638-44a1-a8b6-4e17799d003f28c28< Metadata Date                   : 2020:10:23 18:31:53-06:00---> Metadata Date                   : 2020:10:25 10:46-06:0040,41c40,41< History Instance ID             : xmp.iid:eac2ee0b-0b2d-4143-8da1-cafb424d66cc, xmp.iid:9acec219-b6a4-4918-a592-9fc6a0ab3486< History When                    : 2014:04:27 20:29:30-06:00, 2020:10:23 18:31:53-06:00---> History Instance ID             : xmp.iid:eac2ee0b-0b2d-4143-8da1-cafb424d66cc, xmp.iid:b5e76784-9638-44a1-a8b6-4e17799d003f> History When                    : 2014:04:27 20:29:30-06:00, 2020:10:25 10:46-06:00

For that particular file, the "Metadata Date" inside Lightroom is in July of 2019!

You may then wonder why the file modification time in the diff isn't in 2019, but instead two days ago. I dug into a recent pre-upgrade backup of my photos, and indeed, the prior file's mtime was in 2019. So, not only did LrC v10 decide it needed to do a bogus update to the file's metadata, it touched the file prior to actually being told it was okay to do so!

This upgrade has entirely invalidated all of my photo backups. Everything has to be backed up again, all because LrC is being stupid about touching files unnecessarily.

Surely the only defensible case for updating the application version number in the file's XMP metadata is that I've changed the photo, so now the program is properly reporting the last application to update the metadata?

I see other posts in the forum here on related topics, such as complaints that publish services are forcing a re-publish of unchanged photos. I'm posting this because I think I've diagnosed this to a deeper level than most users.

[Aaron Roberts]

This issue is indeed very frustrating. I have decided to capture a video of this problem here: https://youtu.be/E92STPHYemE 

I've just ended up messing up my entire collection of photo exports because of this problem. It seems that the older the photo, the more likely it is to decide that it needs to be republished, even if nothing has changed with that image for YEARS!

I wish the Adobe team would get on top of this or add a feature to "lock" the publishing status.

[helen_glazer]

This is outrageous. I'm encountering the bug, too, and when I looked it up I found this discussion. It's been going on for years. I guess the publish to Flickr module in Lightroom is essentially useless. I do not want to sit here modifying the publishing dates for over 600 photos.

[doug_bailey]

I'll add my "me too" to this chain.  I've been digging into the symptoms a little bit tonight, and I've got an interesting way to reproduce the problem.

I'm working with 12.0.  I flushed out a published-to-harddrive album (marked everything to republish, deleted the contents of the directory on the harddrive, then republished the album).  Grid mode viewing the album and everything is marked as published.  Examine one of the photos, zoom in to 100%, zoom back to "fit".  Switch back to grid mode.  That one photo is now in need of republishing.  I reproduced this a bunch of times, but it doesn't seem to happen every time.

[DVABAU]

Trust me, there is no rhyme or reason, it arbitrary marks things to republish regardless of my activity or lack thereof.

[Jonathan25940150zxd9]

Pretty much anytime I open an album Lightroom moves all of the pictures (very slowly) to 'modified re-publish'. So if I want to go back and add a keyword tag or pretty much anything, it's impossible to do until I republish the album. This makes lightroom almost useless for me. I've noticed that there are never any posts from Adobe. Is this actually a support group owned by Adobe or just a place to complain about the problems we find? I'm getting the feeling that they have the market so dont' really care about customer service. 

[johnrellis]

@Jonathan25940150zxd9  wrote, "So if I want to go back and add a keyword tag or pretty much anything, it's impossible to do until I republish the album."

Can you say more about this?  You can add keywords to photos needing republishing.

[alexskunz]

@Jonathan25940150zxd9you can mark all photos in the "modified to republish" section as "up to date" (Cmd/Ctrl+A, then right-click) so that you don't have to actually re-publish if they haven't changed.

[DVABAU]

Yes that works if you also haven't been editing otherwise, your changes won't be published. I'll often do this before I start editing and hope that by time I done there aren't that many more that get added.

Regards,

Dana

[Aaron Roberts]

I also have this issue for YEARS. It seems that Adobe are doing sweet FA about it and it's driving me nuts. Some of the publishing services I use will overwrite the "published" files, in other cases, it'll simply just keep adding to the published files.. resulting in duplicates of images which are already publish and have NOT been edited.

About half a year ago, I posted a videon on this also: https://youtu.be/E92STPHYemE

There have been numerous updates to LrC over the past months and yet this bug still remains. It even seems to get triggered by simply re-generating previews. I fricking hate it!

[Aaron Roberts]

Just to add to my last comment. Even marking all photos to republish and then ACTUALLY re-publishing them all, does not solve the issue. Even if the entire collection was published 5 minutes before, scrolling through the images will randomly mark images to re-publish again. 

[johnrellis]

"Even marking all photos to republish and then ACTUALLY re-publishing them all, does not solve the issue. Even if the entire collection was published 5 minutes before, scrolling through the images will randomly mark images to re-publish again."

That's what I've observed in LR 11 and 12 too. Prior to that, once I republished, the photos usually stayed published, but no longer. These days I just ignore the publish status and try to remember what collections need publishing or republishing, sigh.

[robertv47531707]

They just continue to ignore all the complaints. I guess there's not a
software engineer who can figure out how to put a button that says "Do not
republish."

[johnrellis]

"a button that says "Do not republish.""

You can select one or more photos in a published collection, right-click, and do Mark As Up-To-Date. But that's broken too.  Soon enough the photos will be marked as needing publishing.

[kimaldis]

It does seem difficult to replicate but I found a couple of things that sometimes make it more or less likely that an image gets marked to repuplished.

1. Select an image, hit 'Z' to zoom, hit 'Z' again. Pausing for 10 seconds or so between hitting keys can increase the likelihood of the photo being marked. Sometimes more than one photo gets marked.

2. It seems to settle down after working within a collection for a while. I messed around with 1. above until most images had been marked, re-marked them all as published and the collection behaved itself for a couple of days.

3. Collections that hasn't been worked in for a while seem more prone to being problematic.

4. An image that doesn't get marked when following the steps in 1. tends to be immune thereafter, at least for a few days.

5. For me, if I re-mark a problematic photo as up to date, it tends to stay that way, at least for a few days.

The problem seems to be worse in collections that haven't been visited for some time.

Here's an interesting one: I have a photo that's in two published collections. I visit one collection, run the steps in 1. above and it pops into the photos to republish section. I then visit the other collection and it's not marked for republishing. If I hop between the two collections the photo remains unpublished in one collection, published in the other. So, I run steps in 1. on the photo in the collection where it's marked to publish. The photo remains unpublished but becomes accompanied by another photograph. I go back to the other collection, which has both photos in it and both are in the published section.

That last paragraph is a bit rambling but the takeaway is this; if  a photo in a published collection gets marked as unpublished in one collection, it may still be marked as published in any other collections it's held in. And if you re-mark that photo as published, it may well re-mark a different photo in a different published collection as unpublished it that other collection also contains the first photo.

If that's all a bit convoluted, I can sit down and make a step by step that's more rigorous.

Conclusion; Published Collections are fundamentally broken. And have been so for at least 1-2 years, without comment from Adobe, which is concerning.

[alexskunz]

The photos that get marked to republish are not using the latest (internal) revision/version of the Develop process. This is not the "Process Version", it's just some metadata fields that are out of the user's sight and control. This happens every time new features are added.

Since Lightroom's develop features are updated pretty much constantly (the latest being the addition of Grain and Tone Curves as local adjustments), that metadata gets updated, constantly. The Publish Collections themselves are working absolutely as intended, tracking those changes and marking those photos to republish.

For example, when Split Toning was changed to Color Grading, the photos looked exactly the same because Color Grading was backwards compatible with Split Toning, but when you "touched" a photo that used Split Toning, Lightroom would automatically update the metadata of that photo, to include the additional/new Color Grading information. Result: the photo is "changed" internally, even though it looks the same to us.

"Touching" a photo is anything that will have Lightroom evaluate and re-render the photo based on updated metadata. This could be an outdated preview in the Library grid, for example, switching to the 1-up view in the Library, or switching to Develop. When that happens, some internal ID or IDs in the metadata will be updated as Lightroom touches the photo, which registers as a change, which causes the photo to be market as re-published.

This can be reproduced easily by selecting older photos in a Publish Collection and telling Lightroom to render new Previews (Library module, Library menu, Previews -> Build Standard-Size Previews).

If you have "Automatically write changes into XMP" active, you can even watch it happen: the sidecar file in Finder/Explorer is updated with a new file date as this happens. If you compare the old and new XMP sidecar file, you will find fields like "Mask ID" (long hexadecimal string) or something like that have changed.

YMMV. These are just my observations. I'm not a programmer and don't know how this could be improved.

[DVABAU]

It's been my experience that all I need to do is open Lightroom and it will start adding photos to the republish folder...sometimes a few, sometimes dozens...all randomly. Before I start any new editing, I'll flag all photos in my publish folder as up to date and hope that by time I'm done editing, only a few random ones will have been added.

[johnrellis]

The symptoms described by @alexskunz, where changes in the internal representation of Develop settings causes LR to think the visual representation of a photo has changed and thus needs republishing, have long occurred with LR. But that doesn't account for most of the current problems with spurious republishing.

For example, the problems occur with recently imported and edited photos. Such photos have the latest internal representation already, so behind-the-scenes updating of the representations isn't happening.

You can also delete LR's Library previews and rebuild all previews, forcing LR to re-render the photos and update the internal representations as needed.  If you then publish those photos , they'll still end up getting marked as needing re-publishing.

There's a straightforward path for Adobe to replicate and debug this issue: They can take a couple of our catalogs, observe the problem happening, and point the debugger at LR.   But it's clearly been deprioritized.

[alexskunz]

It just so happened that Lightroom decided to throw my entire preview cache away, after a system migration 🥴 and because I'm aware of the republish issue, I had it then render new "Standard" previews for published photos so that I could mark them as "up to date" in one fell swoop.

After I have done that, no photos are marked as updated for republishing anymore. I tested this with a walk through the entire published collection in the Develop module (I tested it with one that contains over 1000 photos from 2007-2023).

So at present, I can't confirm the randomness that @johnrellis mentioned — which isn't to say that it isn't happening, I'm sure there are outliers...

I'm mentioning this because having LR render Standard previews for all published photos in a collection seems to update whatever metadata change triggers this. Together with John's plugin to mark all publish collections as up to date, it can help stop the constant pain of seemingly random photos getting marked as updated to republish.

I just fear this might become necessary again with the next Lightroom update... 🤪

[AZMyke]

After reading the forum, having had the same problem for a while now I reach out to Adobe and began to chat with their virtue chats person.  Long story short, he had me delete my Flickr connection with LRC and redo it.  It seemed he could see this connection from his end.  This by the way was on my MAC ... I couldn't get the problem I was having to duplicate itself on my Windows LRC.  For now,  after publishing a couple of pictures (on the Mac) I'm not getting the flood of re-publishing as I did before.  None have occurred yet.   Keeping my fingers crossed.

[kimaldis]

I'd be surprised if this works. For one, I don't have the Flickr plugin installed.

[johnrellis]

"I'm mentioning this because having LR render Standard previews for all published photos in a collection seems to update whatever metadata change triggers this."

Glad that's working for you.  Forcing LR to re-render previews hasn't helped me and others in this thread, unfortunately.

[aliceinwl]

I was really hoping there was a solution to this. I have over 20,000 photos in my Flickr collection. Ever since I upgraded to Lightroom 11, every time I go to publish new photos or republish select old photos Lightroom starts pulling up random photos from my catalog by the 1000s. Republishing them doesn't help and telling Lightroom the photos are up to date only helps temporarily. Even marking them up to date is like whack a mole because Lightroom keeps pulling up more photos by the 100s as fast as I can mark them. It's hugely frustrating 😞

[alexskunz]

@aliceingdid you already try letting Lightroom Classic re-render previews for all photos in the entire Flickr collection?

Cmd/Ctrl+A, menu Library, Previews -> Render Standard-Sized Previews.

[alexskunz]

@johnrellisquite honestly, I'm surprised that this problem hasn't reappeared for me 😅 but so far, so good. 🤞🏻

I'm wondering whether it might have to do something with the preview cache itself. As I mentioned, LRC decided to nuke my entire preview cache after a system migration.

[AZMyke]

On 8/13 through the insistence of  Adobe support on the phone with me, I removed my Flickr link service to LRC and reinitiated it.  I am pleased to say to date I haven't had any random republishing events in my Publish Services for Flickr. (Mac Computer)