P: Performance Slow on Large Catalog (540k images). Crash (9690382)

Done, thanks, @Rikk Flohr: Photography 

Please review the direct message I've sent you. 

>> That should be fine if the system assigns a conventional drive letter to the internal drive. 

I can confirm this is the case. It is my C Drive

>> You don't mention details regarding your "Full OS Re-install of Windows". Was this a hard drive format - clean install or a reinstall over the top operation?

This was a brand new clean reinstall of 22H2 Windows Pro.

>> When Lightroom Classic next updates, I would recommend using a clean install procedure.

I will do so. 

 
>> I've forwarded information regarding your Preview Generation comments to the LrClassic team for review. 

Thanks. Please let me know if there are any tips they might have to speed this up and or data I can gather for them.

"This is a BIOS controlled hardware raid that is directly attached to my motherboard. I can provide more details if you would like."

That should be fine if the system assigns a conventional drive letter to the internal drive. 

You don't mention details regarding your "Full OS Re-install of Windows". Was this a hard drive format - clean install or a reinstall over the top operation?

When Lightroom Classic next updates, I would recommend using a clean install procedure.

Clean Lightroom Install Procedure

• Close Lightroom
• Restart the computer
• Use the Adobe Creative Cloud App to uninstall Lightroom
• Restart the computer
• Install Lightroom via the Creative Cloud App without launching any other programs.
• Restart the computer
• Launch Lightroom
• Wait 5 minutes

 
I've forwarded information regarding your Preview Generation comments to the LrClassic team for review. 

>> I want to set an expectation. I do not work in support - I work in engineering. 

Great! 

>> That said, I currently run Lightroom on (9) Mac and Windows machines with catalogs ranging from a few images to over 400K images.  Looking at your five numbered complaints, I do not experience any of these issues on any of my systems. 

I believe you! You must be one of the lucky ones, then. Because this forum is filled with people who have challenges. Just google "lightroom performance" and you will see hundreds and thousands of reports filling this and other forums. This cannot be treated like a "problem exists between keyboard and chair" issue. I understand and empathize that you have millions of variables to sift through. But, I can tell you that without changing ANY variables on my end that over the last year (somewhere around 13.1 or 13.2) it has become unusable. 

>>I partially addressed #4 via the crash report you sent and have commented on that directly.  Thank you for submitting the report. While I do not doubt that you are experiencing the issues you mention, I will say that you will get better results in this forum by doing the following:

Thanks for taking action on it, I really appreciate it!
 

>>Distinctly different issues, e.g., Crash vs Slow Scroll, should not be enumerated in the same thread.  The way our internal systems work and integrate with this forum, combined with the structure of how things work in the form, means that laundry lists of grievances will never be handled gracefully.  Separate different issues. There may be more apt threads for your 1-3, 5 items that have already been addressed and will receive updated information in the future. They can't be separated and attached to various threads as a list.

Okay, so then let's focus this one on preview generation performance alone and we can go from there with other posts. Would you like me to change the title and original post to focus on that? I can open more threads later once we resolve that one.

>> Use precise language.  From your OP: " Importing images is almost impossible because generating previews slows the process down to a complete crawl" is of practically no value to our engineering staff. Example: "100 images took 6 hours and 30 minutes to import when it formerly took less than 10 minutes" is good actionable data. 

Okay, here's some data from what I can remember.. I imported 520GB of images from a USB3 attached CF Express card that can read out at 1700MBps. I was reading this to a NAS connected via 2x 25GBE connections that can write at 1000MBPs. Import usually took ~1hour with preview generation. It took more than 20 hours to finish the import.

Another datapoint. I mentioned previously that I deleted all my previews and regerate them to try to fix this. I selected 20,000 65MP  Sony A75R images and clicked "generate standard previews". I got about 800 previews generated per hour.

>> I appreciate the great system information you've provided, as well as your settings and the steps that you've already taken, listed so clearly. It is of great help, and I wish everyone could be this thorough when reporting. It saves a lot of time. 

Thanks for mentioning this. It took quite a bit of my time.

Specifically, regarding:

1.  I do not see this on Win 10 or Win 11. Please read my original post where I give you a precise build number.

2.  I did a preview regeneration (you don't mention your settings)  400K images took overnight Please read my original post where it is th 6th bullet under settings

3. 1300 images import and preview generation take about 20 minutes here Not my experience.

4. Crash Reporting to an existing bug , which will be fixed in our next release. Thanks!

5. Slow UI - I do not see any issues with Mac or Win, but again, precise timings would be helpful here instead of "barely functional". Some off the top of my head. I right click one image and the menu takes 2 minutes to load. I scroll in library, and it stutters, making it hard to get to the image I desire. I select 5 HDR bracketed images and click "merge to HDR", the window takes ~1 minute to appear. I click on a folder on the side bar to expand it, it sits there with a spinning icon for 1 minute before expanding and showing image count. 

>> All that said, Lightroom Classic will release a major update in a few weeks. It will address your item 4 crasher and, coupled with a clean install of LrC, I would expect these issues to go away.  If they do not, I recommend your reposting those remaining in new threads with good information so we can get it to the devs as quickly as possible. 
That is wonderful. I am hopeful you are right. I will follow your instructions in the future.


One question I have for you regarding this from your OP:

• 32TB NVME Raid0 (Catalog + Previews) (17 GB per second sequential read]

>> Describe the location of this drive. Is this internal to your computer? If not, how is it attached? 

This is a BIOS controlled hardware raid that is directly attached to my motherboard. I can provide more details if you would like.

@Adam Steenwyk 

I want to set an expectation. I do not work in support - I work in engineering. 

That said, I currently run Lightroom on (9) Mac and Windows machines with catalogs ranging from a few images to over 400K images.  Looking at your five numbered complaints, I do not experience any of these issues on any of my systems. 

I partially addressed #4 via the crash report you sent and have commented on that directly.  Thank you for submitting the report. While I do not doubt that you are experiencing the issues you mention, I will say that you will get better results in this forum by doing the following:

1.  Distinctly different issues, e.g., Crash vs Slow Scroll, should not be enumerated in the same thread.  The way our internal systems work and integrate with this forum, combined with the structure of how things work in the form, means that laundry lists of grievances will never be handled gracefully.  Separate different issues. There may be more apt threads for your 1-3, 5 items that have already been addressed and will receive updated information in the future. They can't be separated and attached to various threads as a list.
   
   
2.  Use precise language.  From your OP: " Importing images is almost impossible because generating previews slows the process down to a complete crawl" is of practically no value to our engineering staff. Example: "100 images took 6 hours and 30 minutes to import when it formerly took less than 10 minutes" is good actionable data. 
   
   I appreciate the great system information you've provided, as well as your settings and the steps that you've already taken, listed so clearly. It is of great help, and I wish everyone could be this thorough when reporting. It saves a lot of time. 
   
   Specifically, regarding:
   
   

1.  I do not see this on Win 10 or Win 11.

2.  I did a preview regeneration (you don't mention your settings)  400K images took overnight

3. 1300 images import and preview generation take about 20 minutes here

4. Crash Reporting to an existing bug , which will be fixed in our next release. 

5. Slow UI - I do not see any issues with Mac or Win, but again, precise timings would be helpful here instead of "barely functional"

All that said, Lightroom Classic will release a major update in a few weeks. It will address your item 4 crasher and, coupled with a clean install of LrC, I would expect these issues to go away.  If they do not, I recommend your reposting those remaining in new threads with good information so we can get it to the devs as quickly as possible. 

One question I have for you regarding this from your OP:

• 32TB NVME Raid0 (Catalog + Previews) (17 GB per second sequential read]

Describe the location of this drive. Is this internal to your computer? If not, how is it attached? 

I appreciate you found the bug report and took action on it. Unfortunately, that bug is unlikely to resolve my performance issues.

Can you help with what other information I can provide to help resolve the slow preview generation and library performance, @Rikk Flohr: Photography ? As you can see, several other users are experiencing similar issues that I am patiently describing.

Thank you,
Adam.

Good questions - I am glad to hear other users are experiencing similar issues.

Hi,

Could this be the same reason I'm finding import very slow? It seems to import in batches... then there's a big pause befroe it starts to convert to DNG and build previews... then this task seems to also be very slow. It's not crashing, so there's no report to send, but I left it on overnight and it still hadn't finished an import of around 2000 images...