Experiencing performance related issues in Lightroom 3.x

[laurencec]

Hi

I just upgraded from lightroom 2.7 to lightroom 3. I then proceeded to import my old catalog. this all went fine but lightroom is so slow, the thumbnail previews take forever to load if I manage to have the patience to wait  for them.

is there a quick solution?? How can it be sped up?

thanks

Laurence

Message title was edited by: Brett N

[molwizard2]

Mikkasa wrote:

That's basically it. There aren't many clues here as to why two identical computers should perform so differently with the same software, but one we do have concerns data bottlenecks. As goodlux7 has a machine v similar to mine it's worth cutting out the obvious differences first.

While firewire is a fast interface for external devices, it's nowhere near as fast as internal drive access. I've heard of a couple of people who've upgraded to solid state eSATA drives (basically external version of the SATA drives used as internals) and reported vastly improved bitrates compared to FW drives. Might be an option if the internal access fixes things and the RAID has an eSATA socket..

I don't think that data transfer rate will currently be the limiting factor with LR3. At least on my system (Win7 64 bit, 16 cores; more details somewhere on this thread) LR3 reads only at a total rate of 10-20 MB/sec combined with a low CPU usage (even while reading several images in parallel). Even USB 2 attached drives enable transfer rates of 25-30 MB/sec. So firewire 800 with more than twice the maximum rate should be much more than LR3 currently will use.

So if the windows resource monitor isn't providing false rates it seems to be the very low intrinsic LR3 reading rate that is causing my "slowness" problem. (not CPU, as others reported here)

I havn't talked about export speed (or better called export slowness) on this thread yet. I just exported about 600 full size JPEGs from 5DMarkII raw files (only basic adjustments like exposure, white balance or crop). Even while all 16 cores were working on exporting 16 images in parallel only one image was finished every 4-5 seconds. This led to a total export time of about 40 minutes.

[Ian Lyons]

molwizard2 wrote:

I havn't talked about export speed (or better called export slowness) on this thread yet. I just exported about 600 full size JPEGs from 5DMarkII raw files (only basic adjustments like exposure, white balance or crop). Even while all 16 cores were working on exporting 16 images in parallel only one image was finished every 4-5 seconds. This led to a total export time of about 40 minutes.

I don't think it's officially documented (i.e. in the User Help document), but Lightroom is designed to perform best when doing multiple concurrent exports rather than a single large serial export. For example, selecting 100 images and exporting as one export is much slower than breaking the export down into two, three or even four smaller concurrent exports. There isn't a lot to be gained by breaking down into more than four exports.

There are some additional words of explantion at about halfway down this page http://macperformanceguide.com/Optimizing-Lightroom.html

[molwizard2]

Ian Lyons wrote:

molwizard2 wrote:

I havn't talked about export speed (or better called export slowness) on this thread yet. I just exported about 600 full size JPEGs from 5DMarkII raw files (only basic adjustments like exposure, white balance or crop). Even while all 16 cores were working on exporting 16 images in parallel only one image was finished every 4-5 seconds. This led to a total export time of about 40 minutes.

I don't think it's officially documented (i.e. in the User Help document), but Lightroom is designed to perform best when doing multiple concurrent exports rather than a single large serial export. For example, selecting 100 images and exporting as one export is much slower than breaking the export down into two, three or even four smaller concurrent exports. There isn't a lot to be gained by breaking down into more than four exports.

There are some additional words of explantion at about halfway down this page http://macperformanceguide.com/Optimizing-Lightroom.html

If it is really designed this way I would call it a poor design. Load balancing between several jobs is fine. But if there is only one job running it should get the full power.

I am aware that the LR jobs are supposed to be background jobs, allowing to work while they are running. But it should really be possible to override this and give a job full priority for maximum speed.

Thank you for the tip and the link. I will try it with my next export.

[DdeGannes]

I experience the same sort of timing for exporting jpeg from raw.

Win 7 64bit  on Pentium Core 2 Quad; at 2.6GHz; 4GB ram. Single file exported 4 seconds.

[Butch_M]

molwizard2 wrote:

Even while all 16 cores were working on exporting 16 images in parallel only one image was finished every 4-5 seconds.

I'm working with an iMac 2.16 GHz Core2Duo maxed out with 3GB RAM, and it has been my experience that it takes 4-5 seconds to export a jpeg as well ... though I also use the Mogrify plugin to add a custom watermark overlay in the export process.This was the case for LR2 as well. I shoot mostly with a D200 (10 MP) and D300 (12 MP) .... All my working image files are stored on an external FW800 drive.

I had also read in the past that spitting up an export could speed up the job, however, I think I recall that doing more than 2 processes at at time didn't offer a significant time saving ... never tested my self though.

[Guest]

Let's see if we can grow this thread to 1000 replies.

[Ian Lyons]

Butch_M wrote:

I had also read in the past that spitting up an export could speed up the job, however, I think I recall that doing more than 2 processes at at time didn't offer a significant time saving ... never tested my self though.

The number of available cores will be a limiting factor.

[arnelg]

In Lr 2, I always split exports, up to three at a time and I can still Develop [mostly brushing and applying graduated filters of exposure, brightness, saturation] in which I can immediately see the results as I apply. With Lr 3, when I split the jpeg exports, 2 or 3 splits, I can still brush and apply graduated filters, but it takes time to see the effect, or worse, the spinning beach ball appears. Considering that I created a new catalog to Lr 3, with fewer than 2,000  files of a mixed of 40D and 5D Mk II captures, while the LR 2 has more than 17,000 files of each catalog that I worked with.

I have tried almost all of the suggestions here, but I get the same sluggish response time. And to ask again like an earlier posts asked: Why would an "upgraded" version of the same software run slower compared to the older version in the same machine? And then I received those ad emails again about speed"....

Good thing, I got only the 30 day LR 3 trial and it has expired, so I have "upgraded" back to Lr 2.7. Just for info: iMac 2.4Ghz, 4 GB Ram, Image files in FW800 - Mercury Elite Pro 7200 rpm.

[Guest]

I think that what we have learned here is that the added features in LR 3.0 may have pushed Lightroom beyond the knee of the curve in terms of being forgiving of unbalanced or underpowered systems.   Much like there is not one type of cancer with one "magic bullet" cure, there may not be one type of configuration issue and one type of "magic bullet" fix.

For example:

    1.  Have you given LR 3.0 a chance to stabilize - that is all the background previews, etc conversions are over?

    2.  There is a performance penalty to be paid with external drives on USB and firewire.  No where near the speed of bus connected hardware. Possibly the reason that Lightroom catalogs cannot be placed on network drives.

    3.   RAM constrained systems pay a performance price.

    4.   Are speedy video cards becoming more important?

    5.   Finely tuned positioning of cache, preview, database, application, OS, and swap files may be more important than in the past.

    6.   Is LR 3.0 to the point with large files and multiple exports, etc that 64 bit is more important to power users?

    7.   Constraints in the above will increase issues with bottlenecks such as underpowered CPUs.

For seasoned "Computing pros", none of the above is a surprise and has been experienced for years, in particular as more muscular Operating Systems are installed on last year's hardware.  To the "Computing Pro", the above is self evident and "motherhood".

However, I believe that most in this forum are seasoned "Photography Pros"  who just want their computing technology to work, and there in is the rub - if it were only that simple, which of course it is not.

I am a "Computing Pro" who happens to use photgraphy to drive his computing habit, rather than the "Photography Pro" who uses computing as a tool.  My system is blazing fast with LR 3.0, but two weeks ago with an overheating CPU automatically thottled back tyo 1/2 speed, it was not.  Fixing computers, is my hobby now after a career in computing system development.

Maybe there is a "magic bullet" coming from Adobe in LR 3.1, I don't know.   It would seem, though, that if nothing else Adobe should provide more guidance on performance penalties associated with LR 3.0 and configuration options that are being employed by their users.

Anecdotal evidence presented in this forum is interesting, but unless apples are compared to apples, not particulary useful for debugging issues such as this.

I have seen many forums take polls and acquire quantifiable data, and  it helps tremendously.  If this forum cannot organize itself to acquire quantifiable info on user experiences in order to try to detect underlying patterns, then all we can really do is wait for LR 3.1 and hope that Adobe does it for us.

As I have said, in prior posts - I am not volunteering to do it, but someone should.

[JayS_In_CT]

csaager,

Why are you posting blank posts?

[MikeLeone]

csaager,

>

Why are you posting blank posts?

When you reply via email, this forum software will delete any text after

the word "w r o t e". And since any decent email program starts a reply

with that word, the actual reply gets cut off. Extremely stupid

programming, but that's the way it is.

[csaager]

Sorry for the blank posts, didn't know that Adobe is so enterprisey that

they pay for a completely crappy forum software... but it explains why this

thread rarely produces any exchange of ideas...

csaager,

>

Why are you posting blank posts?

>

When you reply via email, this forum software will delete any text after

the word "w r o t e". And since any decent email program starts a reply

with that word, the actual reply gets cut off. Extremely stupid

programming, but that's the way it is.

>

Here my latest post (hopefully there isn't the dirty word somewhere in the

middle:-(

from:sherlocc

I think that what we have learned here is that the added features in LR 3.0

may have pushed Lightroom beyond the knee of the curve in terms of being

forgiving of unbalanced or underpowered systems.

Not the case it reportedly runs better than Lr 2.7 und rather basic systems

...

1. Have you given LR 3.0 a chance to stabilize - that is all the

background previews, etc conversions are over?

>

Yes, wait 30' minutes to gain speed for 30" - my bet is cache thrashing -

where can I place my money?

2. There is a performance penalty to be paid with external drives on

USB and firewire. No where near the speed of bus connected hardware.

Possibly the reason that Lightroom catalogs cannot be placed on network

drives.

>

Why the heck it uses then 4+ GB of memory? - unless you scroll infull speed

through a catalog in full screen the chache should give a reasonable

lookahead. Even Picasa does better the Lr3 (it actually does a very good

you, perhaps because it had been designed by performance aware engineers)

3. RAM constrained systems pay a performance price.

4. Are speedy video cards becoming more important?

>

Yes and yes, but I wonder why my nVidia can cope with the latest games at

60fps, but not with still-shots...

5. Finely tuned positioning of cache, preview, database, application,

OS, and swap files may be more important than in the past.

>

Just the opposite shall be true! This low-level tuning stuff is a thing of

the past. Modern systems auto-tune to the hardware and the usage profile.

Besides, for a consumer product (it costs less than $1000, thus it is a

consumer product) it is a product defect. Period.

6. Is LR 3.0 to the point with large files and multiple exports, etc

that 64 bit is more important to power users?

>

I can't think of a file editable by Lr that exceeds the limits of a 32-bit

platform - the only benefit can come from caching to ahve a smoother flow,

but there it fails miserably

7. Constraints in the above will increase issues with bottlenecks

such as underpowered CPUs.

>

You mentions that you have an overclocked 4-core - nice to have, but almost

unseen in professional environments. If a $300 product requires a

super-computing cluster to run we are back to the old IBM mindset: Give-away

the software so that they'll buy the iron (AFAIK Adobe hasn't a stake in

Intel, thus this business model will lead to their demise)

>

For seasoned Computing pros, none of the above is a surprise and has been

experienced for years, in particular as more muscular Operating Systems are

installed on last year's hardware. To the "Computing Pro", the above is

self evident and "motherhood".

>

However, I believe that most in this forum are seasoned "Photography Pros"

who just want their computing technology to work, and there in is the rub -

if it were only that simple, which of course it is not.

>

>

I am younger than you, but still long enough in the business to have

"unlearned" many of the "performance" tweaking, because it is overall

inefficient to get a job done. I don't want to waste my time to figure out

how many packages I have to make to have an efficient export - the software

should figure out itself, same for cache-size and placement (look at ZFS to

see what is possible in that area)

I am a "Computing Pro" who happens to use photgraphy to drive his computing

habit, rather than the "Photography Pro" who uses computing as a tool. My

system is blazing fast with LR 3.0, but two weeks ago with an overheating

CPU automatically thottled back tyo 1/2 speed, it was not. Fixing

computers, is my hobby now after a career in computing system development.

>

Same here, but I am still active in CS and it annoys me to no end that a

commercial software is written so carelessly, poorly documented and

virtually unsupported (where is the "Report a problem" menu?). I hadn't had

the highest regard of the quality my company delivers, but this experience

proves a former boss of mine who said "elsewhere is where it is only worse".

>

Maybe there is a "magic bullet" coming from Adobe in LR 3.1, I don't

know. It would seem, though, that if nothing else Adobe should provide

more guidance on performance penalties associated with LR 3.0 and

configuration options that are being employed by their users.

>

Configure Lr3 like Oracle 7.0.16? No thanks, again this is a $300 piece of

software, if serious users have to invest an order of magnitude of that to

make it usable it is almost fraudish.

>

I have seen many forums take polls and acquire quantifiable data, and it

helps tremendously. If this forum cannot organize itself to acquire

quantifiable info on user experiences in order to try to detect underlying

patterns, then all we can really do is wait for LR 3.1 and hope that Adobe

does it for us.

>

Good luck with that: Intel has a dozen of CPUs on the market, several

chipsets, motherboards, disks and their arrangements - thus hundreds of

possible configurations and you don't have a clue which parameter is

relevant. What Adobe should (have) do(ne) is to include performance logging

and a feedback agent, but that only costs money... I remember an early post

from Melissa who recommended optimizing the catalog, but making a backup

before so that they have something to analyze... sad.

[DanTull]

> There is a performance penalty to be paid with external drives on USB and firewire.  No where near the speed of bus connected hardware.

Yes. Often LR is CPU bound and not I/O bound, but there are certain aspects of what it does that are gated by how fast it can slurp bytes off the disk.

> Possibly the reason that Lightroom catalogs cannot be placed on network drives.

Actually, the main reason is that the file locking can't be trusted (read: corruption risk), but yes the latency of a network drive would be deadly for the database's patterns of disk access. Under certain loads with a really fast network, it might actually be a win since there'd be no contention with accessing images on local storage, but that's just a theory. In any case, that's a whole separate thread of discussion.

DT

[goodlux7]

ok, so having the catalog on the firewire drive was the issue here.

Thanks Jay and Mikkasa for pointing me in the right direction there. I think

this is the blog post that recommended storing on an external drive:

http://thelightroomlab.com/2009/05/how-to-get-your-lightroom-catalog-onto-an-external-hard-drive/

Bad advice!

I've moved the catalogs to the main local drive, and since I am using file

vault on os x, I don't put them in my user directory...I store them in an

unencrypted folder off of the root drive. My source files are still on the

firewire drive. I've also moved the ACR Cache to an unencrypted folder from

the default location.

This is much quicker that what I was just experiencing with the catalog on

the firewire drive.

However...I'm back to square one; when I move though the previews, I still

get the half-second pause when the image first comes up...first jaggy jpeg

image, then smoother image. Sometimes the loading... icon comes on for a

split second, sometimes longer, and sometimes it just hangs and I can't

switch images...I'm going to say 10 seconds.

I thought the whole point of having previews is so that this wouldn't

happen. Is there any fix for this?

Thanks again!

[JayS_In_CT]

goodlux7 wrote:

ok, so having the catalog on the firewire drive was the issue here.

Thanks Jay and Mikkasa for pointing me in the right direction there. I think

this is the blog post that recommended storing on an external drive:

http://thelightroomlab.com/2009/05/how-to-get-your-lightroom-catalog-onto-an-ext ernal-hard-drive/

Bad advice!

I've moved the catalogs to the main local drive, and since I am using file

vault on os x, I don't put them in my user directory...I store them in an

unencrypted folder off of the root drive. My source files are still on the

firewire drive. I've also moved the ACR Cache to an unencrypted folder from

the default location.

This is much quicker that what I was just experiencing with the catalog on

the firewire drive.

However...I'm back to square one; when I move though the previews, I still

get the half-second pause when the image first comes up...first jaggy jpeg

image, then smoother image. Sometimes the loading... icon comes on for a

split second, sometimes longer, and sometimes it just hangs and I can't

switch images...I'm going to say 10 seconds.

I thought the whole point of having previews is so that this wouldn't

happen. Is there any fix for this?

Thanks again!

Goodlux7,

Cache size and location?  I have my cache set at 60GB on an external but eSATA connected drive.  Clear the cache, select about a 100 or so images, create standard size previews and see if that helps on those 100..  take it from there on the rest perhaps.

Glad the other move helped....

Jay S.

[goodlux7]

Unfortunately ESata is not an option...the fastest port on the iMac is

fw800. The only option is to have the cache on the local drive, or the

firewire drive, and my guess is neither would be faster than the internal

drive.

This is an apple defect...I don't know why they don't include an Esata port

in this day and age. In my mind's eye I see Steve Jobs having a hissy fit

about esata.

If I could backtrack and do it over, I would definitely go for a PC with

Windows 7 and Esata external drives for photo doing editing.

When you say "clear the cache" do you mean: "Purge Cache" button in the

settings? Or actually go in and delete the .lrdata file?

I just tried "Purge Cache" and when I'm only looking at 100 photos (I've

selected one folder) it does seem faster....not so many jaggies, and the

loading icon overlay doesn't come up as often...it seems to only come up

after I've moved over a bunch of images, then stop on a particular image.

Again though this is much quicker.

Not sure why it suddenly seems to be working better, but thank you!

[hpgisler]

Hi

I experienced the same. My config: Windows XP, 2GHz, 2GB RAM

1. I have disabled indexing in XP a) all individual harddrives b) disabled the indexing service

2. I let the 1:1 previews render on (after) import

Currently, this seems to have done the trick (only testing it now for a couple of minutes) - don't sure however, whether it was 1. oder 2. ...

Regards

[pikeas]

Windows 7 x64, i7 920, 6 GB RAM, 2x GTX 260 cards for 2 24" monitors.

I'd like to add another data point to the discussion. LR3 is significantly slower for me than LR 2.6. I notice it primarily in the Develop module. Changes made to a photo (WB, Blacks, recropping, etc) take a second to update on my primary display, and several seconds to update on my secondary display (set to Full Screen, Loupe Normal).

My toolbars are on my primary screen, so the secondary screen is the one I watch while editing. This slowness makes the application much harder to use. I'm very close to switching back to LR2.

Also, here's a fun bug: I can no longer paste Develop settings onto multiple photos selected in the filmstrip. Pasting picture by picture -> productivity plummets.

[WJMooney]

The following info may have already been provided, but I don't have time to review the list--sorry. Like most people checking this thread, LR3 is running much slower than LR2 (for me anyway). On vacation, I took my laptop (Lenovo T61 Window 7 Enterprise Core Duo 2.5 GHz, 4 GM RAM, 64 bit OS) and after a few days downloaded some pics. Then started playing with them and totally forgot about the performance issues. Life was good and I actually forgot about the performance issue--until today. Today, upon my return home, I connected my laptop to my dock station, which is connected to a Dell 24 inch display (Dell 2408WFP) via a DVI cable, and I'm back to being frustrated as I work with LR3. So, I believe the performance issue is not related to the computer per se, but rather somewhere between the computer and display monitor. Hope this helps. Sorry if reduntant.

Cheers,

Bill

[MikeLeone]

Some people have reported that as monitor resolution goes down,

performance goes up. You may want to try that - try setting your display

resolution to the same resolution as your laptop. Maybe it will help.

[Lee_Jay-ZyZk56]

MikeLeone wrote:

Some people have reported that as monitor resolution goes down,

performance goes up. You may want to try that - try setting your display

resolution to the same resolution as your laptop. Maybe it will help.

You could just make LR a window, and shrink it.

[csaager]

Perhaps shrinking down to 3"? I doubt that it will reach the speed of the

in-camera preview...

Has anyone noticed this as well (when importing):

- Go to loupe view

- exclude from image (X)

- move to next image (cursor-right)

1. Almost immediately "Include in Import" becomes check

2. only after a small delay "Loading data" is displayed

3. "Loading data" disappears but the image is unchanged

4. after a while the next image is displayed

Very annoying (and simply poor programming, even performance left aside) and

can lead to mistakes where you import the wrong image:-(

Expected behaviour

1. Disable "Include in Import"

2. Display "Loading data"

3. Display Image

4. Update value "Include in Import"

5. Enable "Include in Import"

Maybe it takes some time to display an image, but try hitting cursor-right

two times quickly, you wait and the first image displays, then it goes to

the next one - what a waste of time! Wouldn't be a problem if performance

was OK, but it isn't and again poor programming.

Take your chance and hit cursor-left, should be fast, because the last image

should be already in cache - forget it. Hit cursor-right again - perhaps the

geniuses of Adobe think you don't have to go back if you are a professional

(I'm not) - again no luck, "Loading Data"... What Lr3 is doing with 4+GB of

mem if not for caching is beyond me - allegedly it runs on 32-bit.

Adobe, please fix at least this, it can't be too difficult - hire some

programmers who know multi-threading (take the budget from QA, they didn't

find these blunders)

-Carsten

[goodlux7]

clearly there is no budget for QA. They just job that out to us.

[JayS_In_CT]

WJMooney wrote:

The following info may have already been provided, but I don't have time to review the list--sorry. Like most people checking this thread, LR3 is running much slower than LR2 (for me anyway). On vacation, I took my laptop (Lenovo T61 Window 7 Enterprise Core Duo 2.5 GHz, 4 GM RAM, 64 bit OS) and after a few days downloaded some pics. Then started playing with them and totally forgot about the performance issues. Life was good and I actually forgot about the performance issue--until today. Today, upon my return home, I connected my laptop to my dock station, which is connected to a Dell 24 inch display (Dell 2408WFP) via a DVI cable, and I'm back to being frustrated as I work with LR3. So, I believe the performance issue is not related to the computer per se, but rather somewhere between the computer and display monitor. Hope this helps. Sorry if reduntant.

Cheers,

Bill

Bill,

This is one of the more confusing issues.  I have to go back and compare the on board graphics for the T61 vs. my MBP 2.1 (2.33 Core 2 Duo, 3GB RAM).  I have the Dell 2407WFP-HC, hooked up the same way and see no difference running LR3 on the MBP stand-alone or in Dual Monitor mode (not mirrored).  The Dell is running 1920x1200.  LR takes up the full real estate of the screen, but I do have the side panels turned on full time so there is some reduction in the image size there.  I still tend to think that these display issues are not as much global but perhaps more specific to various chipsets..  Wish I had a fix, but seeing as how it doesn't seem to matter one way or the other here, perhaps there is one that they can get to.

Jay S.