P: Constantly "saving xmp for xx photos" slowing down my Lightroom Classic!

[Junction10Photo]

Did a job yesterday and got home to edit the photos.. 
Immediately after importing the images (.CR2 Raw files from Canon 5DS) Lightroom started struggling to perform ANY action..  even going to the next frame in the film strip took several seconds, with a few short appearances of the spinning beach ball.. Editing was practically impossible - trying to crop an image and move the crop rectangle was taking about 15 seconds to update the frame position..

I looked what 'activity' was happening behind the identity plate at the top left and it was "saving xmp for xx photos" (xx was a number between 25-80 something)..  but then if I moved to the next frame, it repeated - "saving xmp for xx photos"...

Everytime I perform ANY action, it's saving XMP to multiple photos - even though I only have one photo selected in the strip. If I switch from library to develop, it again saves XMP to multiple photos - without me changing ANYTHING...

Why is Lightroom constantly writing XMP to multiple photos that haven't changed?? It's hogging my system resources!

Here's a video - the only action I'm performing is pressing the right cursor key to go to the next frame in the folder.. ALL files have already had current metadata saved to them, so there is NOTHING to save XMP for!

Was working in Lightroom Classic 13.5.1 - first job in that, so reverted back to 13.5 but it continued to keep saving XMP info for multiple files.

 

[JohnVPhotos]

LR 13.5.1 Mac Sonoma 14.5

As the headline states:

"Save Metadata to File" & "Read Metadata from File" Updates whole catalog. Not just current Collection as expected.

Issue has been around for a long while for me. Most of the time when I choose either, LR will save/read all 140,000+ images in my catalog, when I only want it to scan my current Collection. It can take 15-25 minutes until it's complete.

I'm using Aftershoot to cull my images. Aftershoot updates the .xmp files after it's cull, that's when "Read Metadata from File" comes into play.

• So my workflow is import using LR Import window as a Collection.
• In LR general corrections are made to the current collection. 
• In Grid View I select "Save Metadata to File" ... that's when LR usaully scans my whole catalog not just the collection.
• Cull in Aftershoot and go back into LR. 
• In Grid View I select "Read Metadata from File"... that's when LR usaully scans my whole catalog not just the collection.

I also have not edited any images in other Collections prior to a new import.

Is there anyway to have LR just Save/Read Metadata from the Collection only? It's super time consuming waiting for LR scan the whole catalog. 

Many Thanks!

John V.

[GoldingD]

Can you [post a screenshot of your LrC /Catalog Settings/Metadata/

[John R Ellis]

A full-resolution screen recording of the entire LR window could also give more clues and understanding.

[daleducatte]

After watching Junction10photo's video showing excess XMP activity, I realized I was experiencing the same thing -- mainly because I had seen performance deterioration with LrC 13.5 and 13.5.1 that I hadn't seen previously. 

I wanted to see if I could reproduce the problem intentionally. Here's what I came up with, and now I can repeat it at will. 

I have "Automatically write changes into XMP" turned on. I'm not synching any collections to the cloud.

I have a collection with 25 images in it; the images are only in this one collection.

In the Develop module, I first right-click on one of the images and select "Show in Finder" then filter the Finder list by ".xmp" and sort it by descending Date Modified. This is so I can easily see what happens next.  

I select 15 of the 25 images, turn on Auto Sync and make a basic editing change. All the XMPs for the images I just edited get updated, with their Date Modified set to, for example, "Today at 10:55 AM." This is expected, of course; and I used Auto Sync simply to get the same Date Modified on all 15 XMPs.

Without doing anything else in Lightroom, I wait until the computer clock changes to 10:56 AM.

When the clock changes, I switch from Develop to Library. After a few seconds, the 15 XMPs get updated again, with their Date Modified changed to "Today at 10:56 AM" -- even though I didn't make any more changes to them after the Auto-Sync/edit.

I wait another minute (for the clock to change) while in Library view without doing anything, then switch back from Library to Develop. All the same XMPs are updated again, their Date Modified changing to "Today at 10:57 AM."

I keep doing this, and every time I wait a bit and switch between the two modules, the XMPs get updated. This isn't the only time it happens; it was just easier to explain this way with a repeatable test. It will also happen if I stay in Develop and edit images; select different ones; switch between collections; pass the mouse over the filmstrip; or leave Lightroom open, work in another app, then return to Lightroom. These last two are weirdly fascinating.

Here are some other observations on what else may be happening because of these extra XMP updates:

- Memory usage climbs substantially, when the only thing I do is switch between modules. Performance drops, and the memory isn't freed until I restart Lightroom.

- I often get the "Adobe Lightroom is not finished writing metadata changes..." message when exiting -- something I had never seen until I updated to 13.5 then 13.5.1.

- I had been syncing about 500 photos to the cloud until recently, but noticed (by watching Sync Activity in Preferences) that sync kept looping repeatedly through photos it had already synced (and I hadn't edited). I reset preferences and followed the reinstall procedure for Lightroom, but the problem persisted so I stopped synching. Now I wonder if repeated syncs were getting triggered because of the excess XMP updates.

In conclusion: Well, I can't really think of anything, but maybe this testing provides some insights into a number of problems people are reporting since the 13x releases. 

[JohanElzenga]

@Rikk Flohr: Photography That does seem like a serious bug to me! This might be the explanation for many of the compaints of slowness!

[John R Ellis]

@daleducatte, excellent detective work!

Here's a slightly simpler recipe for reproducing the misbehavior on LR 13.5.1 / Mac OS 14.6.1. I've attached a screen recording with the long pauses edited out:

1. Place 30 raws in a folder.

2. Open Finder on that folder, with View As List, sorting in descending order by Date Modified, searching by Name: .xmp.

3. In LR, set Catalog Settings > Metadata > Automatically Write Changes Into XMP.

4. Import the raws from the folder with option Add.

5. Select all the photos and go to Develop.

6. Change Exposure.

7. Do Sync with Check All. 

8. Wait until the clock ticks over to the next minute and go to Library.  Observe all the .xmp sidecars get modified.

9. Wait until the clock ticks over to the next minute and go to Develop. Observe all the .xmp sidecards get modified.

10. Repeat ad nauseum.

[John R Ellis]

And here's a recipe showing that simply hovering the mouse over a photo in the filmstrip causes its .xmp sidecar to be modified (see the attached recording):

1. Do steps 1-7 of the previous recipe I posted.

2. Still in Develop, hover the mouse over another photo in the filmstrip. Observe that its .xmp sidecar gets sorted to the top in Finder, indicating it was just modified.

3. Repeat for each of the other photos in the filmstrip, repeatedly. Observe the .xmp sidecars get modified over and over.

[John R Ellis]

@Rikk Flohr: Photography, please consider moving to Bugs. I was able to easily reproduce @daleducatte's steps (slightly simplified) and also discovered that simply hovering the mouse over the filmstrip causes the photo's .xmp sidecar to get modified.  See my two previous posts for recipes and screen recordings.

[daleducatte]

Thank you! It was "fun" to try and reproduce the problem. 

Just watched both your videos -- seeing the XMPs get updated when mouse-hovering is wild!

P.S. In step 9 above, did you mean to say "go to Develop" rather than "go to Library"? 

[Junction10Photo]

Thanks for everyone managing to find repeatable steps to reproduce the bug.. I've not had chance to do the full screen recording you suggested @John R Ellis - but just wondering - have you tried to see what changes are being written to the XMP file? Is it updating a particular parameter in the file or is it just rewriting the current settings and causing the timestamp only to change?

if I get chance later I'll try and compare the contents of an xmp file to see BEFORE/AFTER differences.

j

[Rikk Flohr: Photography]

updating status and adding bug number. 

[John R Ellis]

@daleducatte: "In step 9 above, did you mean to say "go to Develop" rather than "go to Library"?"

Thanks, I corrected it.

[John R Ellis]

@Junction10Photo: "I've not had chance to do the full screen recording you suggested"

I don't think that's necessary, now that @daleducatte identified a simple recipe for reproducing the problem.

"have you tried to see what changes are being written to the XMP file?"

I did -- the only fields that changed were MetadataDate, an industry standard field recording when the metadata was last written, and InstanceID, identifying the unique instance of that metadata. These get changed whenever the metadata is written.

I think the two recipes indicate the internal comparison method for determining when metadata has changed is completely broken. I believe the recipes may be triggered by a longstanding LR heuristic where it gives priority to re-rendering library previews of changed photos whose thumbnails are currently being displayed in the filmstrip and grid view (deferring other changed photos until later). As part of that re-rendering, it may also be checking if the metadata has changed and needs to be written to disk.

diff 1.xmp 1\ copy.xmp 
35c35
<    xmp:MetadataDate="2024-09-08T08:12:08-07:00"
---
>    xmp:MetadataDate="2024-09-08T08:11:18-07:00"
47c47
<    xmpMM:InstanceID="xmp.iid:a59c9467-74e9-4ca4-9a99-1aae1d0cc160"
---
>    xmpMM:InstanceID="xmp.iid:6f5b4f63-36bb-48c7-be49-06ad64986d6b"
164,165c164,165
<       stEvt:instanceID="xmp.iid:a59c9467-74e9-4ca4-9a99-1aae1d0cc160"
<       stEvt:when="2024-09-08T08:12:08-07:00"
---
>       stEvt:instanceID="xmp.iid:6f5b4f63-36bb-48c7-be49-06ad64986d6b"
>       stEvt:when="2024-09-08T08:11:18-07:00"

[Jao vdL]

This might also be related to the other bug that adds history steps for just going into develop while not making a single change. 

[daleducatte]

I wanted to add something that I confirmed by additional testing, since we now know how to trigger excess XMP updates. I originally wrote:

"I had been syncing about 500 photos to the cloud until recently, but noticed (by watching Sync Activity in Preferences) that sync kept looping repeatedly through photos it had already synced (and I hadn't edited).... Now I wonder if repeated syncs were getting triggered because of the excess XMP updates."

This morning, I turned "Automatically write changes to XMP" off and synced the collection I'd been using for testing. With this setting off, sync behaves normally -- each image gets synced as it's edited and there are no repeated syncs.

Then I turned "Automatically write changes to XMP" back on and edited three of the collection's photos. After that, the three photos kept re-syncing to the cloud every time I use one of the methods we found that unnecessarily updates XMPs, like moving between Develop and Library, selecting another image in the filmstrip, or just mousing around. This continued until I switched to a different collection (when Lightroom synced them one last time).

That this would happen may be obvious, but I wasn't sure if there was a relationship so wanted to see if I could make it loop both XMP updates and syncs -- and it did!

[JohnVPhotos]

Thanks everyone for troubleshooting this issue. The fix is turning "Automatically write changes to XMP" to off. I've shot a number of events since and have not had a slowdown. Though I still think it's a bug!

Thanks again!

John V.

[DdeGannes]

@JohnVPhotos , you stated "I'm using Aftershoot to cull my images. Aftershoot updates the .xmp files after it's cull, that's when "Read Metadata from File" comes into play."

Is Aftershoot a plug-in for Lightroom Classic?

[JohnVPhotos]

Aftershoot is a stand-alone app for culling and editing but works with LR.  It works with LR via its XMP files - hence the READ and WRITE of the XMP. Right now I'm using the culling side of the app. The app is super useful for me when shooting galas and events where repetitive frames are needed such as group shots. For instance I'll take 5 frames of a 10 person group. Aftershoot will scan all 5 frames and choose the best frame based on eye-blinks and focus. The app has saved me tons of culling time.

https://aftershoot.com

[DdeGannes]

@JohnVPhotos , Ok understand but why would it trigger LrC to Read Metadata from XMP?

[JohnVPhotos]

Aftershoot reads and then modifies LR's XMP files. User needs to initiate Read XMP Files in LR to complete the task.

[DdeGannes]

Correct!

[sej671693]

I'm on 13.5.1 on a newish windows machine running Win 10. It's a fast machine.

When I import a few hundred photos and do some work (or even not much work at all) and try to exit, I *always* get the message that lightroom isn't finished writing XMP files yet. (I'm set for always write sidecars...)

I can leave the program running an entire day and it won't finish. In fact, if I look in the folder, there may be no XMP files at all -- it's not even getting started.

However, if I close the program down and restart, then it will (usually) pick up where it left off and beging writing the sidecars, taking maybe 20 seconds to write a few hundred of them, then shut down as normal.

Now that I know the workaround, at least I can address this, but this clearly isn't working as intended.

[sej671693]

Should note that other times, all the xmp files may be there in the new folder with the new images, and a count shows they're all there and up to date, but LR is thinks they're not up to date for some reason (or thinks some other files I haven't touched, that have been up to date for days/weeks/months/years? are out of date? That sounds like the big reported earlier in 2024 that's supposedly been fixed.

[sej671693]

ok, after going throught the process to see if I have the prior bug (files reverting back to "has been changed" right after re-saving metadata) I see that somehow I do. I have about 2,500 jpegs that keep reverting back to "has been changed" despite multiple writes. I'm not sure why lightroom let me quit out w/o the "writing metadata" warning on exit if it was seeing these erroneous updates, but it did.

Looks like this bug remains.

[sej671693]

More screwing around... I could *not* get some of those files to stayed updated writing the metadata manually. So I thought heck with it, it'll quit without the nag, I know those files are fine, I'm done for now. So I switched auto XMP writing back on. Then lightroom went ahead and auto-updated the xmp for those files and it stuck, so now they're OK. Now my only outliers are in a small pile of jpegs that LR says are read only (they're not).