Export and retain publish service data?

[JBenfordPhoto]

I'm having some issues with LR classic and I can't tell if it's a bug or an issue with my catalog. I suspect the catalog but wanted to check here.

I'm running LR classic 14.3 on Windows 11 Home v24H2 on a one year old Lenovo Legion 5 Pro. My catalog has about 139K images in it; the catalog file is about 2.5 GB. I back up the catalog regularly, including testing integrity and optimizing - no recent integrity issues found. 

Here are the issues I'm noticing:

• When I add color labels or star ratings, it takes a long time (>1 minute) for it to tint the cell or show the stars in the expanded cells. The label and rating shows up in the image metadata viewer in the right panel.
• When I close the catalog, it frequently (but not always) says it is still writing metadata changes to xmp, even if I haven't done much (or even sometimes when all I've done is navigate, no develop or metadata changes at all).
• When I close the catalog, it takes a looong time for the app to actually finish closing and for Windows to release the memory (so that, for example, I could open a different catalog). The WAL and SHM often files persist and the WAL file can get quite large (as large or even larger than the main catalog). Task Manager shows that it is using the network even though the catalog is on a local drive (some of the images are on a NAS drive, so I'm guessing it's writing to XMP for files on the NAS drive - even though I haven't changed any metadata or develop settings for any of them). 

I exported the entire catalog to a new catalog and the new catalog did not seem to experience these issues, so I assume that means the catalog has gotten corrupted. I would just move forward with this, but I use a publish service (Jordy Meow's WP/LR sync plugin) to sync files with my website and, of course, publish service data doesn't get exported.

So it seems my options are: 

• export all images to a new catalog and lose the publish service info, or
• try to find a backup catalog from before the corruption occurred and import any new images and metadata/develop updates from the current catalog. 

Any other possibilities?

Anyone have insight into the problem or other suggestions of things to try? 

[JohanElzenga]

I would start by not automatically writing changes to XMP. There is no compelling need to do that.

[JBenfordPhoto]

Could be but it's only become a problem recently. Why would it suddenly think there are so many changes to write when I only touch a few images? 

[JohanElzenga]

I don't know, but the point is that writing to XMP, especially on a NAS, will definitely slow Lightroom Classic down.

[JBenfordPhoto]

This thread (Writing Metatdata Changes into XMP on exit takes f... - Adobe Community - 14148414) suggests that it shouldn't impact performance significantly unless you're making a lot of changes - which I'm not. And the few images that I am touching are on my local SSD drive, not on the NAS. Sometimes I don't make any changes at all and it still takes a long time. 

Following a suggestion from that thread, I made a Smart Collection identifying any images whose Metadata Status is "Has Been Changed" - it found zero images. I made another Smart Collection to identify all images whose Metadata Status is not "Up to date." It found about 7500 images, most appear to be status "Conflict detected" due to editing in an external app. I have just told it to overwrite the metadata for all those files, which took 15-20 minutes.

Within a couple of minutes after finishing that process, there were 831 images in the collection saying the metadata has been changed in Lightroom. If I save the metadata to the file for all these images, as soon as it finishes the collection repopulates immediately with the same 831 images (I made a regular collection with these images so I could compare to the smart collection - it's the same images). I've tried doing the same again several times, including after optimizing the catalog, and the same thing happens again. So it looks like there's a metadata issue with these files that may be causing it to write the metadata over and over and over again.  

[JBenfordPhoto]

Now it's added a few (28) more images that supposedly have metadata changes. Very weird!

[johnrellis · Accepted solution]

"Within a couple of minutes after finishing that process, there were 831 images in the collection saying the metadata has been changed in Lightroom. If I save the metadata to the file for all these images, as soon as it finishes the collection repopulates immediately with the same 831 images"

This is a known bug that's been occurring for perhaps a couple years, though I was first able to file a reproducible bug report a year ago:

https://community.adobe.com/t5/lightroom-classic-bugs/p-save-metadata-to-file-doesn-t-always-set-met...

Metadata Status and saving to XMP has been getting flakier the last several years, and it's self-evidently not a priority for Adobe to fix. Whether saving to XMP is important is a matter of religion in the forums -- I prefer it as my last line of defense for failing backups (after Time Machine and Backblaze), and it protects against accidental deletion of edited photos (something I do about once a year even though I religiously use the delete-rejected-photos workflow).

[JBenfordPhoto]

Thank you, John. Yes, this combined with Automatically write metadata changes to xmp files seems to be causing an infinite loop. I agree with your thinking about a last line of defense but it's not workable to keep that setting on until Adobe fixes the bug. Turning it off seems to solve my problem for now.  

[dj_paige]

Honestly, none of these symptoms sounds to me like a corrupted catalog. There have been threads about catalogs taking a long time to close (which I didn't read), perhaps you should search for them.

[JBenfordPhoto]

Hmm, ok, I will take a look. Question though: if it's not an issue with the catalog file, how do you explain the fact that, if I export all the images to a new catalog, the new catalog doesn't have these issues?

[JBenfordPhoto]

I didn't do an exhaustive search but the main thread I found about catalogs taking a long time to close described the same problem I am having.