P: Library Module slow performance: FIX found!

[FeedbackCommunityMember]

I've found a solution to a problem many others seem to have reported - but it does involve a work-around, suggesting the Adobe Photoshop Lightroom team need to fix a bug.

When a large number of Develop Presets are loaded into Lightroom (in my case, over 500), the Library Module, and strangely, to a lesser degree, the Develop Module run very slow.

Having performed some troubleshooting, I discovered the reason for the slowness was a CPU spike. On my iMac 3.4GHz with 32Gb RAM, NVIDIA GeForce GTX 680MX with 2GB, and 1TB Fusion Drive, the CPU would go over 100%, while the disk and memory allocations were low. I run Mavericks 10.9.2, so it's not a matter of an out of date OS. This occurred whether I had 10 images or 10,000 images in the catalog (I tested various scenarios).

I read on a forum that the number of Develop Presets could affect performance, so deleted the majority of them (backing them up first) to see if it would make a difference. I reduced the number of presets to less than about 100, and the performance increased INCREDIBLY and INSTANTLY.

So, it would seem to me there is a bug in the way the Develop Presets are being indexed in Lightroom. Hopefully, this is something the team can find and fix very easily, as it should be easy enough to identify the root cause and reproduce the problem.

Please, Adobe - release a fix for this in the next point release. It is disappointing not to be able to keep all my Develop Presets in a single session. My work-around is to exit Lightroom, copy in the Presets I need for a job, then start the program again. Obviously not ideal.

Many thanks,

Matthew

[DdeGannes]

I guess this might make a big difference the "Power / Super performance system users" who are having problems. Hope you get free upgrades of Lightroom for life.

[johnrellis]

I wonder if this is related to the Undo bug:

http://feedback.photoshop.com/photosh...

In that situation, an Adobe employee reported that a menu with a very large number of items (e.g. the develop preset menu) could cause Undo to fail. Perhaps another side effect is to cause Library performance to go south.

[Benjamin Warde]

Hi Matthew,

Thanks for the info. I haven't observed this problem myself. Can you provide a bit of information about what you mean specifically when you say "slow"? What are some specific actions that you performed? How long did they take with all the presets present? How long did they take after you removed the presets?

And to answer John's speculation, I doubt that this issue is related to the other problem, since Matthew is on Mac, and the other problem is Win-only. But hey, anything's possible. :-)

Thanks,
Ben

[FeedbackCommunityMember]

Hi Ben,

Thanks for the response here. Happy to help where I can.

Essentially, clicking anything brings up the BBOD (Beachball of Death / Spinning wheel). It doesn't matter whether it's an image with or without a 1:1 preview. In fact, even scrolling is very slow to respond.

With the Presets present, each click or scroll easily took 1-2 seconds. Without them, the response was almost instantaneous.

As I mentioned above, the CPU was spiking up over 100% in Activity Monitor when I had the Presets installed whenever I did almost anything in LR. Without the Presets, the CPU was hovering around 18%.

Have you tried loading up (say) 500 Develop Presets on a similar spec system? I've not got anything installed I'd consider particularly unusual - I tend to run a pretty vanilla system. Mavericks, with all the latest updates to both OS and LR.

The other apps I have installed are Google Drive (though I don't stall anything LR related there), 1Password (only recent, and the condition was present before I installed it), Creative Cloud, and Evernote.

The iMac is pretty highly spec'd and runs all of my other applications (including Photoshop) without problems. I also tried the other solutions, like increasing my cache, to no avail. In fact, the condition was present with both very small (10 images) catalogs and very large (10K+). Always worse in Library Module, but still very noticeable in Develop.

I'm convinced it's got to do with the way LR is handling / caching the Presets. As a former developer and long-time IT guy, my guess is there's an inefficient algo somewhere in the code that's iterating over the presets to refresh some kind of cache. Such a condition would not be a problem with a small number of Presets loaded, but as the cache gets bigger, it would spike.

Hope that helps,

M

[FeedbackCommunityMember]

I guess the other possibility here is that one or more of the Presets themselves might be corrupt. Although, I would expect LR to warn me if that were the case.

Some of them were designed for earlier versions of LR, so perhaps that has something to do with it?

Cheers
M

[Benjamin Warde]

"Essentially, clicking anything brings up the BBOD"

So you're in the Library grid view, you click on an unselected photo to select it, and Lightroom beachballs for a couple seconds before selecting the photo?

"I guess the other possibility here is that one or more of the Presets themselves might be corrupt."

That was my thinking, since I've tested with over 1,000 presets on a much wimpier system without any trouble. Do you mind zipping up your entire collection of presets, and sending it to me? You could put it in a public Dropbox folder, and send me the link, or if you prefer not to do that, I can contact you privately and give you instructions for FTPing it to me directly at Adobe.

Thanks,
Ben

[FeedbackCommunityMember]

Correct - clicking in the grid view causes the BBOD. LR is also slow in the Develop module.

I don't have a Dropbox account, so please email me the FTP details. I'm at work presently, and based in Australia, so it may take a few hours for me to get this across to you.

Out of curiosity, is there any easy way for me to tell if the presets are corrupt? Is there some way I can verify this?

Cheers
M

[Benjamin Warde]

Hi Matthew,

I've emailed you privately with the FTP info.

"Out of curiosity, is there any easy way for me to tell if the presets are corrupt?"

Not that I'm aware of, unfortunately.

Thanks,
Ben

[FeedbackCommunityMember]

Thanks Ben - I'll upload tonight.

Really appreciate your support on this.

Cheers
M

[Mike_Katz]

Tried removing presets, didn't help.

[FeedbackCommunityMember]

If you’re having this problem on OS X, check your Spotlight Privacy tab and see if your Lightroom folder is listed in there. Mine was, and removing it allowed me to keep the presets installed while preserving performance in the Library module. Really weird stuff.

[FeedbackCommunityMember]

Thanks @mark. I did check that - and in my case, it hadn't been added to the Spotlight dialog. I don't think it is by default?

Sorry to hear it, @mike. Care to share your configuration / platform / OS, Memory, etc?

Cheers
M

[Mike_Katz]

Hi Matthew
Intel i7 3.4GH 16GB RAM 240GB SSD working drive
Latest Win 7 Professional 64-bit

Mike

[FeedbackCommunityMember]

Thanks @mike. Seems like this problem is not isolated to Mac. I wonder if it's got to do with i7 processors?

[FeedbackCommunityMember]

Hello Matthew,

I have been living with this annoying problem for a long time now, thanks for finding somewhat of a solution!

I wish Adobe would address this issue, it is a huge burden on my day to day work, so much so I've even been temped to edit in Capture One or some other app.

Please Adobe, fix this one soon. It's really killing my workflow and speed, especially on shoots with thousands of images to edit down.

FYI I have this problem on 5 different Macs from 2009 Mac Pros to the New 2013 Mac Pro and on a i7 MacBook Pro.

[Mike_Katz]

Hi Erik
I'm in a similar position and also looking at C1. Editing is nightmarishly slow because of this issue.
Mike

[FeedbackCommunityMember]

I'm currently just living with removing the presets all the time. Let me know how it goes with C1, I'm not currently a big fan.

[FeedbackCommunityMember]

@Ben,

Haven't heard anything from you since I sent you my presets. Any further word?

Clearly, I'm not the only guy shouting about this - and it seems others have verified the solution works for them also.

Would be good to get an update from Adobe on this issue.

Thanks

[Benjamin Warde]

Hi Matthew,

I have done some testing and have confirmed that you do not (so far as I can tell) have a corrupted preset. I believe you are correct, it is simply the large number of presets that is causing Lightroom to perform poorly.

However, your statement that you have "over 500" presets is, while true, a bit of an understatement. You have 7476 presets which, I'll be honest, is quite a bit more than I had previously tested (I'd only tested up to about 1,000 presets).

How many presets you can have before performance drops off obviously depends on a lot of things, in particular how powerful your machine is. On my relatively low end test machine Lightroom was still able to perform reasonably well with 2,000 presets.

So, I will bring it to the team's attention that performance degrades with large numbers of Develop presets. But I want to be honest up front, I imagine that the team will probably consider having 7476 presets to be something of an edge case, and fixing this may not get top priority. So in the meantime, if you can limit yourself to your favorite 2,000, you should be in good shape. Heck, if you can go crazy and cut it down to 1,000, things should be really snappy. :-)

Thanks,
Ben

[Mike_Katz]

Hi Ben

I have the same speed issue, but presets are NOT my problem. I have around 1500, but when I remove them all (as well as camera profiles from packages such as VSCO), my import is still abysmally slow compared to 5.3.

Therefore, I'd ask you to please NOT think that this issue is solely related to the number of presets.

To clarify, I had the same number of presets etc. with 5.3, and there is no issue.

PS I have a high-end windows machine, quad processor, 16GB RAM, and I work off an SSD.

Thanks

Mike

[FeedbackCommunityMember]

Thanks @Ben,

For clarity, I do not need or use all of those presets together at any one time. I sent you my entire library to make sure none of them were corrupt.

I take your point regarding the number of presets loaded at any given time, and how 7K represents something of an edge case, however I have tried this with far fewer (less than 1K) installed and still had issues. It was only when I removed the vast majority of presets that the performance improved.

In that regard, Mike has highlighted there may be two independent issues here, which makes me suspect it's not as straightforward as first thought.

That said, the team should be able to identify the root cause of such issues using code profiling.

Thanks for looking into the issue - I do hope the team will continue looking at ways to improve product performance. Photographers are a hungry bunch!

Cheers
M

[Benjamin Warde]

@mike:

Unfortunately, performance is not a single issue, it is a vast array of issues, which must each be carefully defined and addressed separately. If you're experiencing a degradation in import performance in 5.4 vs. 5.3 (which is not at all what this thread is about), I would suggest you start a new thread on that topic, with as much detail as possible.

@ Matt:

On a much lower end machine than yours, I notice no degradation whatsoever when using 1,000 of your presets, and very little when using 2,000. If that is different than your experience, then there may be a separate issue here. Let me know how many presets you feel you can have before you start to notice a performance problem. There may be some diagnostics we could run on your machine that might shed some light on the matter.

Thanks,
Ben

[Mike_Katz]

Hi Ben
I assumed that there was some type of relationship.
The topic already exists:
http://feedback.photoshop.com/photosh...
Mike

[FeedbackCommunityMember]

Notably, Mike is on Win, I am on Mac.

[Benjamin Warde]

Hi Mike, thanks for pointing me at that thread.

-Ben