Rendering for a plugin does not multithread?

[PaulWaldo]

Hi all, I'm writing a plugin that exports selected images, runs an external program to do some calculations, then writes metadata back to the original image in Lightroom.  I've notice some things that are odd:

• Lightroom seems to be taking its sweet time rendering the images.  I've got an 8 core Mac and it looks like images are being rendered serially.  I would have expected that LR would render 8 images simultaneously.
• Metadata values are not updated in the UI  until the entire task is finished, even on the first image rendered.

I've tried inserting LrTasks.yield() in various places in my plugin, but that seems to have made it even slower!  Does anyone know how the underlying threading mechanism works?  I want to hear my machine's fans screaming! 😉  Thanks in advance...

[areohbee · Accepted solution]

Regarding point #1:

Not sure I can tell you anything you don't already know:

Exports within a single session are serial.

If you want parallel exports you need to create multiple export sessions.

Some performance improvement can be had that way, but don't get your hopes up too high.

Regarding point #2 - Lr does seem to update stuff on it's own schedule, I've tried various things to convince it to update stuff, but with only marginal success (e.g. deselect, then reselect...). I *don't* think it necessarily waits until exporting is finished, - probably just seems that way, since it's kept pretty busy by exporting (?).

PS - I have a feature request for a "goosing" function, so plugin can force refreshing of externally changed photos:

http://feedback.photoshop.com/photoshop_family/topics/lightroom_sdk_function_to_refresh_thumbnail_an...

Sorry I couldn't be more help,

Rob.

[PaulWaldo]

Thanks for the reply, Rob.  I tested and multiple export sessions do allow me to fry eggs on my machine.  Thanks!

[areohbee]

In theory, you shouldn't see much improvement by having multiple export sessions, because, in theory, a single export session should consume 100% of all cores most of the time. If you do see a lot of improvement, then it could be due to the "less-than-optimal" processor utilization that seems to plague some machines.

I get only marginal improvement in throughput via parallel exports.

I've heard here and there that Lightroom downthrottles cpu consumption during rendering so UI stays responsive, but it shouldn't have to do that, and it doesn't do that on my machine.

Cheers,

Rob

[PaulWaldo]

Hi Rob,

I see just the opposite.  As single export session uses 25% CPU at best.  I had to get 4 simultaneous sessions going to get things humming.  Even with 4 sessions, I still see 25% idle quite a bit.  I'm running MacBook Pro, 2.2 GHz i7, 16 GB memory, Mountain Lion, LR 4.2

[PaulWaldo]

Also, the majority of time is spent here:

local success, pathOrMessage = sourceRendition:waitForRender()

according to my logging

[areohbee]

You've got Lr4-itis I think.

(non-optimal CPU allocation syndrome)

[PaulWaldo]

OK Rob, I'll bite.  What is "non-optimal CPU allocation"?  I've googled and searched the forums with no hits.  Is this a symptom of my particular system configuration or a specific version of Lightroom?

[areohbee]

Lightroom should use CPU to it's fullest during exports. If it doesn't, then there is something wrong (non-optimal).

All 4 cores of my CPU are utilized 100% most of the time during exports (single session). (which generally take 3 or 4 seconds for D300 files on a modest AMD system - single export session; more if lots of dust-spots and brush-strokes...).

Parallel exports (multiple export sessions) doesn't make it much faster.

There has been a lot of discussion about CPU allocation, hyperthreading, and performance issues in general which have haunted & plagued Lr4 - more so than any other version so far, or so it seems to me.

It sounds to me like your CPU is not being optimally utilized when exporting single-session.

Rob