Lightroom 4.2 very poor CPU usage

[Sebasvideo]

Lightroom 4.2 seems reasonably fast when I work with it, whether it's browsing photos or adjusting sliders, although it takes several seconds to go into develop mode after launching it for the first time.

But now I'm exporting 1498 photos that are 5184 by 3456 and it's taking quite a while, I would say about an hour or more. This is on a brand new system I just assembled consisting of an i7 3930k with 32 GB of RAM that flies with every other program. While exporting this batch I opened Task Manager and I noticed that CPU usage never goes to 100%, not even close. There are peaks of 50%, but on average it must be in the 20s:

This is very disappointing on a CPU that has 6 physical cores and 12 logical cores with hyperthreading at 3.2 with turbo at 3.8 Ghz. The batch is exporting these photos from one SATA 6 drive to another SATA 6 drive, and the HD LED barely lights up, so I know the hard drives are not the bottleneck. So I'm wondering, is Lightroom 4.2 really that bad when it comes to taking advantage of the CPU cores? Is there anything I can do to make it use the CPU more?

Thanks,

Sebastian

[web-weaver]

BillAnderson1,

I think you nailed the issue of slow performance of Lr.

I'm not a computer engineer but "intuitively" I understood that data transfer and data storage times are the root cause of sluggishness in today's systems.

Lr constantly writes to and reads from the catalog. When working in the Develop Module, every step is instantly stored in the catalog and immediately it and all previous steps are read back. That necessitates a lot of data transfer. At the same time Lr uses the ACR cache and (possibly) the preview cache. In the background Windows is using the page file.

If all these data packets have to be written to and read from the same drive the queue builds up while the CPU sits idle.

SSDs help because the read and write times are shorter, but I think distribution of the data transfer to as many physical drives as possible does more to improve Lr performance.

Naturally, having 3 SSDs is best.

I don't have the fastest system (Quad Xeon 3550. 12 GB RAM) but I use 3 internal drives (with OS and Lr on drive 1, Win page file and Lr ACR on drive 2, Lr catalog on drive 3), and photos on external drives. My ACR cache is set to 75 GB.

I don't experience the slow performance that many complain about and I attribute it to the drive configuration.

So it makes eminent sense to distribute the data transfer to as many drives as reasonably possible as Bob Frost (post # 44) suggests.

[areohbee]

web-weaver wrote:

I think you nailed the issue of slow performance of Lr.

I think that is a an over-statement, to say the least, and put it mildly...

Theory without validation is just, well, theory. And as we know, in theory, there is no difference between practice and theory, but in practice: there is...

Why not:

* test these theories

* post results

For example, I had a theory once, that it would be a lot faster if I put catalog by itself (as much as possible) on a separate SSD instead of my main data drive (hard disk).

Result: not much faster - oh well, so much for that theory...

Rob

[bob frost]

From: "web-weaver

Lr constantly writes to and reads from the catalog. When working in the

Develop Module, every step is instantly stored in the catalog and

immediately it and all previous steps are read back. That necessitates a

lot of data transfer. At the same time Lr uses the ACR cache and

(possibly) the preview cache. In the background Windows is using the page

file.

In addition LR seems to spend a lot of time writing to temporary files on

your system drive, updating your xmp sidecar files (if you use them). Even

when doing nothing, LR still spends a lot of time reading and writing

between your previews, previews.db, and catalog, of whatever folder of

images it is currently on. If you put it on All Photographs and you have a

lot, it can spend a couple of hours doing that, as a background task.

Bob Frost

[areohbee]

Along those lines: Dreamweaver consumes 25% of my CPU whenever there is a file open in it (even when app is minimized and no activity...). So, I run it at lower priority and close all files when not editing...

[Todd Shaner]

BillAnderson1 wrote:

Putting high access file in a new partition will do nothing for performance. A partition is just another location on a disk that must share the same and single access path as all other partions on the drive.

My response and others here are to your post #33 concerning disk fragmentation slowing down LR. I suggested putting the LR Catalog and Camera Raw cache on separate partitions to reduce disk fragmentation, which will improve performance by your own admission.

BillAnderson1 wrote:

Plus it will do nothing whatsoever to avoid fragmentation or make its effects any less.

A partition is a contiguous area of reserved disk space and putting the LR catalog and/or Camera Cache on a separate partition greatly reduces the possibility of fragmentation. I'm not saying it will "automatically" increase performance.

BillAnderson1 wrote:

Even SSDs can have performance problems if too many file are accessed at the same time as is the case in a highly multithreaded program like lightroom.

It's seems you are recommending disk fragmentation as the only solution to disk I/O bound performance issues. There's no question that a small percentage of LR users are experiencing LR performance issues even when using multiple SSDs and very fast processors and the actual root cause has yet to be identified.

I've been using LR on a single 1TB hard drive based Windows 7 system for several years now with a separate partition for the LR catalog and Camera Raw cache. I've experienced no major performance issues processing 21mp 5d MKII raw files. I recently added a 2nd internal hard drive (WD Caviar Black) and moved all my image files to the new drive. I also tried moving the LR catalog and Camera Raw Cache to different drives to see the affect on performance. From my tests there appeared to be no "measurable" performance advantage to using two hard drives with LR, so I more or less agree with you.

I have no experience using SSDs with LR so can't comment on their performance improvement.

[BillAnderson1]

I'm sorry to be a contrarian and I mean no disrespect, but fragmentation occurs when the file system cannot or will not allocate enough contiguous space to store a complete file as a unit, but instead puts parts of it in gaps between other files. This frequently happens in LR when the file size changes and becomes larger during an edit. Partitions are simply locations on a disk and and offer no protection from fragmentation.

I am working on a little demonstration that I will publish later today that I think will demonstrate the value of multiple physical drive.