P: Performance Slow on Large Catalog (540k images). Crash (9690382)

[Adam Steenwyk]

I am at my wit's end. I have been a LR user since Beta (2006), and have devoted tens of thousands of hours to creating and managing my catalog. But, in the last 4-6 months, performance (and stability) has reached unusable levels.

Here are my system stats (also attached)

• Ryzen 9 79503D
• 192GB Ram
• 32TB NVME Raid0 (Catalog + Previews) (17 GB per second sequential read]
• 96TB Thunderbolt 4 attached RAID 6 (Original Photos) [1.6 GB per second sequential read]
• Radeon 7900XTX w/ 24GB V-Ram
• 4X 27 inch 4k displays
• Windows 10 Pro for Workstations 22H2 - 19045.4894

Here are my lightroom settings

• Lightroom 13.5.1
• Full GPU Acceleration
• No XMP Autowrite
• 6.06GB Catalog File Size
• Catalog Optimized at every close (daily)
• 3840/High Previews
• Never Discard 1:1 Previews
• Replace Embedded Previews during idle OFF
• Sync OFF
• 200GB Camera Raw Cache
• Generate Previews in Parallel ON
• Enable HDR in Library OFF

Here is what I have tried to fix performance

1. Create a brand new catalog and import old catalog into it - no effect
2. Delete all 8TB of Previews and Cache and regenerate for current catalog - no effect
3. Optimize Catalog - no effect
4. Turn Off / On all options I outlined above - no effect
5. Check all local disk / network attached disk settings, INCLUDING spending $3k on a new thunderbolt 4 NAS - no effect
6. Reduce Preview Size down from 3840px - unclear effect
7. Full OS Re-install of Windows

Here are my complaints:

1.  LIBRARY PERFORMANCE - Scrolling in library, even with no images selected, is choppy at best. Scrolling library folders and performing basic operations is sluggish and error prone.
2.  PREVIEW GENERATION - Generating previews does NOT utilize all processors. Re-generating 500k previews is going to take 3-4 weeks, when in the past this has only taken a couple days. Why?
3.  SLOW IMPORT - Related to 2, importing images is almost impossible because generating previews slows the process down to a complete crawl. I have no idea why this is the case with a system like mine. It took me 4 separate NIGHTS of trying to import 1TB of images recently because the act of creating previews made the import process so slow, and then the program would crash.
   4. LACK OF STABILITY - Since everything takes so long, the numerous crashes the program encounters during normal operation cause deep frustration. Here's a pastebin of one such crash that occurred 10 minutes into re-creating the previews I had deleted in an attempt to solve the terrible performance issues of late. I have experienced numerous others crashes when batch merging HDRs and Panoramas overnight that require me to then slog through the terrible browsing performance to re-select the files and re-batch them, only for the program to crash again in another 2 hours.
   5. SLOW UI OPERATION - Example: setting labels, ratings, keywords, all drag on. Moving between images in library while doing this is barely functional.

I rarely post anything on any forum ever, but the fact that my decades of dedication to the program are now being thrown to the winds of "shove AI into everything" rather than focusing on the QUALITY and USABILITY of the tool makes me deeply sad and frustrated.

In my view, the execs at Adobe need to take a step back from the AI craze and look at their products. People spend their lives learning your tools. Take some pride in them and make them WORK WELL. If this continues for another half a year, I am throwing it all away for C1 Pro and never looking back. I'll have lost thousands of hours of work, but at least I will have a functional tool once again.

Unfortunately, I have worked designing and building software for 2 decades and I understand how prioritization and decision-making works in a large organization. I know this "bug report" will get zero traction because I am complaining of a myriad of issues with no clear repro steps. But maybe someone will hear my voice and decide to do something about it. But probably not. I will hope, at least.

[AxelMatt]

In a first step try resetting the preferences of Lightroom Classic:  How to reset Lightroom Classic preferences (adobe.com)

It's recommended to backup your preferences before you reset the preferences to the default settings: 

https://helpx.adobe.com/lightroom-classic/kb/preference-file-and-other-file-locations.html

If this doesn't help try this:

• Close Lightroom 
• Take a backup of  this folder : C:\Users\xxxx\AppData\Roaming\Adobe\Lightroom  and rename it to LightroomOld
• Restart Lightroom

Try to deactivate all plugins and check if it's better. If yes, activate one plugin at a time and test whether there is a change in performance.

Another try is to create a new catalog, import several images and check it.

[dj_paige]

Also, temporarily try turning off the anti-virus to see if that helps.

[Rikk Flohr: Photography]

We’ve reviewed the email address you use with this forum and find no crash reports in our system.

Did you see a crash dialog?

Do you use a different email address when filling out the crash report?

Was the crash dialog Adobe’s? Microsoft’s? 

[Adam Steenwyk]

I did see a crash dialog. Unfortunately, when I clicked "send", the application hung for 5 minutes, so I gave up. It was not a Windows dialog. But a dump of that crash I mentioned is in the pastebin I referenced in my post.

https://pastebin.com/uMSye9Q9 

[Adam Steenwyk]

As I described, I attempted to create another catalog. Same issues. It's also not relevant to me to consider a split catalog with fewer images in each. I frequently go back and forth between images in the past, and having to remember where they are in a different catalog would make that impossible.

I will try the preferences / plugins reset. I have done this in the past with no luck, but will try anything at this point.

[Adam Steenwyk]

No change.

[Rikk Flohr: Photography]

If you repeat the crash and see a dialog, be sure to fill it out completely and screenshot it before submitting it. 

Post the screenshot in this thread. 

Crash reports pasted into the forum are of little value in tracking down causes. I reviewed the report and it revealed little without context. A report really needs to be transmitted. 

Thanks. 

[Adam Steenwyk]

I will keep that in mind and will submit one the next time it happens. However, do note that when the tool does not allow you to submit the report due to a hang, that is impossible. 

Putting that aside, the one thing that is somewhat concrete in my feedback is the preview generation performance. Can you comment on if this is something that can be worked around, or if a fix is planned, @Rikk Flohr: Photography?

[Adam Steenwyk]

I have reset all preferences and can confirm the issues still exist.

[Adam Steenwyk]

Hey @Rikk Flohr: Photography,

I have a few updates:

1. I have reset all preferences and deleted all plugins and reloaded the original catalog. No effect.
2. I have manually created a new catalog from the old one and loaded it again. No effect.
3. I have encountered several crashes along the way and was able to successfully submit them under ajamess@gmail.com. But none of these reproduce any sort of performance issue, they were just random crashes. But maybe they are helpful.
4. I have tried versions 13.5.1 and 13.4 and see no appreciable difference in performance.

At this point, my biggest issue is the extremely slow preview generation speed, as now I am focused on simply getting my catalog up and running. I did some math and calculate that I am getting around 800 standard sized previews generated per hour. I see this is consuming only 5-10% of my CPU.

In the past, this process would consume 100% of my CPU and went about 10x-20x faster. What can I do to get back to the appropriate speed here? I recall 13.1 - 13.3 being faster, but of course now that my catalog is upgraded to the new format, I can no longer downgrade to those versions.

Is this something you can have the adobe engineers investigate? I am more than happy to provide any and all information to facilitate this.

Thanks,

Adam.

[GoldingD]

• Radeon 7900XTX w/ 24GB V-Ram

Did you look at the possible AMD Adrenalin setting as game issue? Abut old, may have been fixed.

Their have been reported issues with AMD GPU drivers causing all sorts of issues. Appears problem relates to newer updates to the Adrenalin software. Problem is the software, the controll for the GPU in a bid to improve GPU performance, accisentaly trats LrC as a game, and some of the settings that get applied foul up the GPU for LrC.

• https://www.reddit.com/r/radeon/comments/1ad69lg/lightroom_classic_crashing_on_the_latest_driver/
• https://community.adobe.com/t5/lightroom-classic-discussions/lightroom-classic-crashing-when-gpu-is-...

[Adam Steenwyk]

Thanks for your suggestion. I had discovered that post in my search for a solution, but forgot to mention I already tried that. Here are the settings I have configured. I can confirm the issues persist despite checking and disabling all of those features (very few of which were enabled to begin with).

[HDelion]

Same issues for me since May. Tried several operations without succes. I think this is due to the conversion of catalog since LRC 13.2. I will be patient again 2 months but will go with C1 after to work properly. And Im sad that Adobe's support answered with same operations to realize. Why to not organize a better support with real tests ?

[Rikk Flohr: Photography]

@Adam Steenwyk I was able to locate your crash report and found it is attached to a bug that will be fixed in an upcoming release.  This thread will receive an update when the fix is released. 

Thank you for submitting the bug report. 

[richardcphotography]

Hi,

Could this be the same reason I'm finding import very slow? It seems to import in batches... then there's a big pause befroe it starts to convert to DNG and build previews... then this task seems to also be very slow. It's not crashing, so there's no report to send, but I left it on overnight and it still hadn't finished an import of around 2000 images...

[Adam Steenwyk]

Good questions - I am glad to hear other users are experiencing similar issues.

[Adam Steenwyk]

I appreciate you found the bug report and took action on it. Unfortunately, that bug is unlikely to resolve my performance issues.

Can you help with what other information I can provide to help resolve the slow preview generation and library performance, @Rikk Flohr: Photography ? As you can see, several other users are experiencing similar issues that I am patiently describing.

Thank you,
Adam.

[Rikk Flohr: Photography]

@Adam Steenwyk 

I want to set an expectation. I do not work in support - I work in engineering. 

That said, I currently run Lightroom on (9) Mac and Windows machines with catalogs ranging from a few images to over 400K images.  Looking at your five numbered complaints, I do not experience any of these issues on any of my systems. 

I partially addressed #4 via the crash report you sent and have commented on that directly.  Thank you for submitting the report. While I do not doubt that you are experiencing the issues you mention, I will say that you will get better results in this forum by doing the following:

1.  Distinctly different issues, e.g., Crash vs Slow Scroll, should not be enumerated in the same thread.  The way our internal systems work and integrate with this forum, combined with the structure of how things work in the form, means that laundry lists of grievances will never be handled gracefully.  Separate different issues. There may be more apt threads for your 1-3, 5 items that have already been addressed and will receive updated information in the future. They can't be separated and attached to various threads as a list.
   
   
2.  Use precise language.  From your OP: " Importing images is almost impossible because generating previews slows the process down to a complete crawl" is of practically no value to our engineering staff. Example: "100 images took 6 hours and 30 minutes to import when it formerly took less than 10 minutes" is good actionable data. 
   
   I appreciate the great system information you've provided, as well as your settings and the steps that you've already taken, listed so clearly. It is of great help, and I wish everyone could be this thorough when reporting. It saves a lot of time. 
   
   Specifically, regarding:
   
   

1.  I do not see this on Win 10 or Win 11.

2.  I did a preview regeneration (you don't mention your settings)  400K images took overnight

3. 1300 images import and preview generation take about 20 minutes here

4. Crash Reporting to an existing bug , which will be fixed in our next release. 

5. Slow UI - I do not see any issues with Mac or Win, but again, precise timings would be helpful here instead of "barely functional"

All that said, Lightroom Classic will release a major update in a few weeks. It will address your item 4 crasher and, coupled with a clean install of LrC, I would expect these issues to go away.  If they do not, I recommend your reposting those remaining in new threads with good information so we can get it to the devs as quickly as possible. 

One question I have for you regarding this from your OP:

• 32TB NVME Raid0 (Catalog + Previews) (17 GB per second sequential read]

Describe the location of this drive. Is this internal to your computer? If not, how is it attached? 

[Adam Steenwyk]

>> I want to set an expectation. I do not work in support - I work in engineering. 

Great! 

>> That said, I currently run Lightroom on (9) Mac and Windows machines with catalogs ranging from a few images to over 400K images.  Looking at your five numbered complaints, I do not experience any of these issues on any of my systems. 

I believe you! You must be one of the lucky ones, then. Because this forum is filled with people who have challenges. Just google "lightroom performance" and you will see hundreds and thousands of reports filling this and other forums. This cannot be treated like a "problem exists between keyboard and chair" issue. I understand and empathize that you have millions of variables to sift through. But, I can tell you that without changing ANY variables on my end that over the last year (somewhere around 13.1 or 13.2) it has become unusable. 

>>I partially addressed #4 via the crash report you sent and have commented on that directly.  Thank you for submitting the report. While I do not doubt that you are experiencing the issues you mention, I will say that you will get better results in this forum by doing the following:

Thanks for taking action on it, I really appreciate it!
 

>>Distinctly different issues, e.g., Crash vs Slow Scroll, should not be enumerated in the same thread.  The way our internal systems work and integrate with this forum, combined with the structure of how things work in the form, means that laundry lists of grievances will never be handled gracefully.  Separate different issues. There may be more apt threads for your 1-3, 5 items that have already been addressed and will receive updated information in the future. They can't be separated and attached to various threads as a list.

Okay, so then let's focus this one on preview generation performance alone and we can go from there with other posts. Would you like me to change the title and original post to focus on that? I can open more threads later once we resolve that one.

>> Use precise language.  From your OP: " Importing images is almost impossible because generating previews slows the process down to a complete crawl" is of practically no value to our engineering staff. Example: "100 images took 6 hours and 30 minutes to import when it formerly took less than 10 minutes" is good actionable data. 

Okay, here's some data from what I can remember.. I imported 520GB of images from a USB3 attached CF Express card that can read out at 1700MBps. I was reading this to a NAS connected via 2x 25GBE connections that can write at 1000MBPs. Import usually took ~1hour with preview generation. It took more than 20 hours to finish the import.

Another datapoint. I mentioned previously that I deleted all my previews and regerate them to try to fix this. I selected 20,000 65MP  Sony A75R images and clicked "generate standard previews". I got about 800 previews generated per hour.

>> I appreciate the great system information you've provided, as well as your settings and the steps that you've already taken, listed so clearly. It is of great help, and I wish everyone could be this thorough when reporting. It saves a lot of time. 

Thanks for mentioning this. It took quite a bit of my time.

Specifically, regarding:

1.  I do not see this on Win 10 or Win 11. Please read my original post where I give you a precise build number.

2.  I did a preview regeneration (you don't mention your settings)  400K images took overnight Please read my original post where it is th 6th bullet under settings

3. 1300 images import and preview generation take about 20 minutes here Not my experience.

4. Crash Reporting to an existing bug , which will be fixed in our next release. Thanks!

5. Slow UI - I do not see any issues with Mac or Win, but again, precise timings would be helpful here instead of "barely functional". Some off the top of my head. I right click one image and the menu takes 2 minutes to load. I scroll in library, and it stutters, making it hard to get to the image I desire. I select 5 HDR bracketed images and click "merge to HDR", the window takes ~1 minute to appear. I click on a folder on the side bar to expand it, it sits there with a spinning icon for 1 minute before expanding and showing image count. 

>> All that said, Lightroom Classic will release a major update in a few weeks. It will address your item 4 crasher and, coupled with a clean install of LrC, I would expect these issues to go away.  If they do not, I recommend your reposting those remaining in new threads with good information so we can get it to the devs as quickly as possible. 
That is wonderful. I am hopeful you are right. I will follow your instructions in the future.


One question I have for you regarding this from your OP:

• 32TB NVME Raid0 (Catalog + Previews) (17 GB per second sequential read]

>> Describe the location of this drive. Is this internal to your computer? If not, how is it attached? 

This is a BIOS controlled hardware raid that is directly attached to my motherboard. I can provide more details if you would like.

[Rikk Flohr: Photography]

"This is a BIOS controlled hardware raid that is directly attached to my motherboard. I can provide more details if you would like."

That should be fine if the system assigns a conventional drive letter to the internal drive. 

You don't mention details regarding your "Full OS Re-install of Windows". Was this a hard drive format - clean install or a reinstall over the top operation?

When Lightroom Classic next updates, I would recommend using a clean install procedure.

Clean Lightroom Install Procedure

• Close Lightroom
• Restart the computer
• Use the Adobe Creative Cloud App to uninstall Lightroom
• Restart the computer
• Install Lightroom via the Creative Cloud App without launching any other programs.
• Restart the computer
• Launch Lightroom
• Wait 5 minutes

 
I've forwarded information regarding your Preview Generation comments to the LrClassic team for review. 

[Adam Steenwyk]

>> That should be fine if the system assigns a conventional drive letter to the internal drive. 

I can confirm this is the case. It is my C Drive

>> You don't mention details regarding your "Full OS Re-install of Windows". Was this a hard drive format - clean install or a reinstall over the top operation?

This was a brand new clean reinstall of 22H2 Windows Pro.

>> When Lightroom Classic next updates, I would recommend using a clean install procedure.

I will do so. 

 
>> I've forwarded information regarding your Preview Generation comments to the LrClassic team for review. 

Thanks. Please let me know if there are any tips they might have to speed this up and or data I can gather for them.

[Rikk Flohr: Photography]

Please review the direct message I've sent you. 

[Adam Steenwyk]

Done, thanks, @Rikk Flohr: Photography 

[muzuzunga]

I have exact same situation with Lightroom Classic 13.5.1 on a Mac OS Sonoma.  I've been a user since Beta as well, 2007, and amassed about 750,000 DNG photos uploaded to the cloud.  After the upgrade from 13.4 from 13.2 things went downhill.  Reverted to 13.2, they were OK again.  Finally saw reports that 13.5 seemed to have fixed the sync bugs, upgraded again to 13.5.1.  The catalog size shot from 11GB to 20GB.  Every operation involving files is taking forever.  Import does not begin for a while.  A list of collections is not showing up.  Previews for a few hundred images take a day to build.

I've thoroughly investugated an alternative solution with Photo Mechanic Plus.  It works very fast and I plan to import images into Apple Photos for convenient export on the go.  

We need these issues fixed, they are killing the setups of the most veteran users here.

[HDelion]

Same behavior for me and Im on Windows 11. how to explain that we have issues on Mac and Microsoft environment ? I tried to check all drivers since several weeks  since the major upgrade in May. Since the catalog conversion, several operation can be frozen during several minutes, even hours ... I don't understand