I was exporting 50 photos on my newly built Core I7-3770 machine, which is a CPU has 4 cores 8 threads. From the task manager, I noticed that only 50~60 percent of CPU was used.
If exporting 1 photo only takes half of the CPU power, why not Lightroom process 2 or more photos at one time for those systems have extra power?
I suspect the export process is memory and (to some extent) i/o bound, not CPU bound. The assumption based on looking at task manager that 100% utilization would mean more ability to export is not based on real observations.
Not only is the task manager a very rough and not very accurate picture of the work your computer is doing, it is not in the least predictive. And, like I said, it depends if the problem at hand (i.e., exporting renditions) is CPU bound or not.
However, I have Lightroom 4.1 running with 16GB ram on a Windows 8 64bit system. So I don't think memory is a bound here for just 50 photos. And my catalog, raw cache files and original raw are stored on SSD drive and I guess there is no i/o bound here as well. But just try to eliminate these cause, I will try to use the Resource Monitor the monitor the CPU/Memory/Disk usage during export next time when I doing the export.
BTW, I have tried to turn off HT for my CPU and I got almost exactly same amount of time used for exporting 50 photos. So Lightroom is actually not using the extra threads provided by the CPU at all.
Scott Kelby offered a tip in a video some time back that you could save time and invoke more resources by exporting in simultaneous batches ...
Say you need to export 300 images to jpeg to upload to the web ... select 150, then export ... immediately select the remaining images and export ... the long hand method to multitasking export ...
I have just tried the manual parallel processing method provided by Butch_M.
By exporting 56 photos in one go, it took 125 seconds
By manually select 30 photos export first and immediately select the remaining 26 export next, it finished in 100 seconds.
That's about 25% speed increase and that is a lot!
I believe if this can be done within the software it could be even faster.
Yeah, they've left it not maxed out, so that you can continue working while the export runs in the background, but a 'turbo export' checkbox in the Export dialog would certainly get my vote.
_______________________________________________ Victoria - The Lightroom Queen - Author of the Lightroom Missing FAQ & Edit on the Go books.
There is a fine line in resource management ... Not all users have the same desires and/or requirements ...
Sometimes I return from an event and need to post 4,000 to 6,000 images to my online shopping cart ASAP as time is money in getting the the pictures in front of potential customers ... other times I only need to export a few dozen images ... while I move on and perform other tasks ...
I agree with Victoria that allowing the option, based upon the current workflow circumstance would be the most appropriate method ... it would be the best of both worlds ... mainly because it would allow the user to be in control, not an arbitrary decision of the underlying code ...
Coincidentally, today I was playing with controlling an export through code . Breaking a 267-file export into 3 roughly-simultaneous exports saved 60% of the start-to-finish export time. I may develop it a little further - for example letting the user choose how many simultaneous batches to run - but the existing code does bypass the standard export dialog so I'm not sure it would be compatible with other plugins.
I think it's best to use all available horse power, and let process priority dictate how it gets shared among apps. Apps shouldn't have to artificially restrict CPU usage - that's what the OS/priority system is for. On my machine, all cores are sometimes utilized at 100% during export, just not for as much of the time as I would expect (goes up to 100% for a while, which is good, then drops down, for longer than I would expect, which is bad, then back up...). I run Lightroom at below-normal priority - long exports have no noticeable impact on other normal priority apps (I started doing that so I could watch HD video without any stuttering, while exporting...). With Lightroom at normal priority, there is occasional stuttering during HD video playback... Other apps doing CPU-intensive tasks (e.g. video/audio compression, use CPU at 100% almost the whole time, without interfering with other higher priority apps (same priority app performance may be degraded, lower priority app performance will get no cpu while higher priority apps are contending for it).
It's long been a mystery what Lightroom is doing sometimes - CPU not maxed out, but ultra-fast disks don't help much - hmmm.... what's the bottle-neck then?
It could be that Lightroom's got some serious problems in the multi-tasking department which is accounting for some of it's ongoing performance woes - even when functioning normally.
If Lightroom is truly so "memory-bound" (I'm not convinced yet), I'd really like to understand that better...
Definitely agree, resource allocation should be a task of OS but not Lightroom itself. It should use all resource available to speed up its processing.
Here's what Adobe should do---> Take a look at Final Cut X and how they tackle real-time rendering.
If Adobe says that they bind the CPU load for exports to keep some overhead CPU for running the UI, I say they simply look at what's going on in the interface. If the mouse is idle, boost the CPU usage to execute export (or other job) as fast as possible. But, as soon as I start moving in the module, clicking on photos and doing other tasks within LR, scale the CPU back on that export. Final Cut X has a user-set timeout period before the CPU kicks into rendering mode. Same could apply to LR. Get up from the computer and 5 seconds after no action, LR floors the CPU.
It's patently silly that we have to divide up our exports into 3 groups to get the software to use 100% of my cpu power. As a pro, when I export, 99% of the time I want it now. I'm not moving onto other jobs. This has been an issue since the beginning.
[If I were wanting to move onto other jobs, I'd want to have some sort of job queue to run at midnight, but that's for another thread.]
I agree - If Adobe has down-throttled export CPU usage so UI remains responsive, they really need to optimize... - that would *not* be considered "best practice", programming-wise.
On the other hand, you may find Lr4.3RC1 is better in this regard. In previous versions anyway, parallel exports greatly increase export throughput on some systems, but on others: not so much. - They fixed some processor utilization issues in .3 candidate.
Dunno what to say: I think Lr has been "CPU allocation challenged" since day one. I also have apps that can utilize all cores at 100% (for extended periods of time) when doing cpu intensive computation - dunno why Lr can't too, or doesn't. I get better CPU allocation than some, but even on my machine the average CPU consumption during a multi-second export is way under 100% (not sure exact number...). I would expect that to be primarily CPU bound, and hence very close to 100% most of the time. Maybe I don't understand the nuances/details, or maybe Adobe really hasn't programmed Lr "correctly" - just dunno... If down-throttling for UI responsiveness is the rationale, then something is not being done optimally, or so I speculate from the comfort of my padded armchair...
I have a 6-core system. With one export job, all 6 cores are in use but at 30%. When I stack up 3 export jobs, I can peg the CPU at a full 100%, all 12 threads.
Yep - I think it's high time Adobe took a good hard look at their software from a CPU allocation perspective. - you shouldn't have to stack up export jobs like that...
But since you do, consider using publish services to help facilitate simultaneous exports, or ExportManager if your exports aren't via publish services.
The same thing that keeps Lr from being as fast as possible during exports (only partial CPU utilization) is also keeping it from being as fast as possible when rendering for develop...
That last bit there... since all of the dev stuff is serial [photo 1, then 2, then 3, then 4] doesn't it make sense to be CPU aware and prerender 2,3 and 4 in the same time it takes to render the current image in the dev panel? In other words, yes, I can confirm that selecting a new image uses 6 cores and 6 threads at about 25%. That means my CPU (1/2 of the threads are idle) and hella powerful GPU are sitting around while I wait as well. Seems like there's a ton of room for optimization here, but I'll admit as you, I'm mostly armchair.
You're preaching to the choir: I don't see any reason why the CPU couldn't be far more utilized when rendering a single image for develop. And regardless, pre-caching (aka look-ahead caching) for develop would be a worthwhile optimization, in my (armchair) opinion.
PS - I have no idea whether the GPU could be tapped for rendering tasks - I would guess: not really.
When converting RAW images, process one image per core. Similar to SETI@home. It may not be efficient to split the conversion of an image across multiple cores but running 4-16 conversions at a time would speed up the overall batch. I don't know if this means that ACR would need to spawn as many instances of itself as there are cores or not. Of course this is memory and disk bandwidth limited, but that can be addressed by the user.
My typical workflow after importing is to keyword the imported photos. Even with previews pre-rendered, scrolling down in grid view is a bit painful. As I'm working away on my current screen full of thumbnails, LR could be doing whatever it needs to do to make it faster to render the thumbnails further down.
Another way to describe it would be to take better advantage of idle CPU and memory.
Better multi-threadedness would be appreciated too on imports and exports.
Lightroom doesn't seems to fully utilizing the CPU/memory. For example on export on my quad-core CPU, utilization often drops to 40-60%. It seems like the writing of the resulting files needs to be done asynchronously with reading/writing files
57
Replies
AdChoices