P: SDK: LrTasks.yield is *extremely* slow on Windows

[areohbee]

Lightroom SDK: LrTasks.yield is extremely slow on Windows.
(its over 100 times slower on Windows than Mac - something is really amiss...)

I sure hope this gets fixed soon - any plugins calling it repeatedly in a loop run like molasses in winter (on Windows).

[johnrellis]

Also, it appears that some SDK API methods call LrTasks.yield() internally and are thus much slower on Windows than Mac. See these threads for more discussions with Rob on this issue:

http://forums.adobe.com/message/3264134
http://forums.adobe.com/message/33902...

[areohbee]

Is this problem acknowledged by Adobe/Chet?

[dtull]

Yeah, I've run across this. The underlying implementation of event loop dispatch on Windows is based on a timer primitive with much lower resolution than what was used on the Mac.

The best answer at the moment for this specific manifestation of that is don't yield so often (the timer on Windows has roughly 10 ms refsolution, if you yield more often than that in a tight loop, you'll take a walltime hit). There's actually a routine in the internal event loop called yieldAfterElapsed time that is used to make it easier to avoid this kind of high-frequency yielding.

It's not something we'd fix in a dot release (too risky), but if I can find a case where it's causing a significant performance deficit (that isn't resolved by reducing yield frequency a bit, which is probably better anyway), I think it'd be possible to implement a higher precision approach for the next major rev.

[dtull]

Interesting. That might give me my excuse push for getting this plumbing issue fixed. I'll take a look.

[areohbee]

Thanks Dan. Yeah, plugin authors who know about the problem can avoid it... I have assumed the Lightroom programmers know about it too and avoid it similarly, although there are some SDK functions that take so much longer to run on Windows than Mac its gotta make a guy wonder...

It helps just knowing its "acknowledged"...

[dtull]

That's probably the best argument for fixing it. It's a portability issue. It's just too easy to forget it and create performance problems.

[areohbee]

Well said.

[johnrellis]

Checking my notes from last fall, LrKeyword.getName() takes about 6 msecs on Windows, while it takes only 1 msec on Mac. I measured other SDK methods but I can't find my notes.

[dtull]

Hmm. That seems about right as a confirmation of my hunch of where else this timer would slow things down. Any data request is going to involve some amount of inter-system messaging. When those messages are coarse grained (where the underlying operation takes significant time) the 10 ms won't be noticed. Where the operation is very fast, it'll show up as a significant (several times slower) overhead.

[johnrellis]

And if the dispatch loop has a 10 ms timer on Windows, then a one-way trip through the loop will take on average 5 ms, which is the difference between the Windows and Macs LrKeyword.getName(). Maybe the difference is actually caused by something else (e.g. the differential cost of file-system locking invoked by SQLite), but the internal yield sure looks suspicious.

[areohbee]

Just did some profiling:

LrTasks.yield 12.4 msec average.
LrTasks.sleep(1) 102 msec average (right - no decimal point)

Yikes!

[areohbee]

Did some profiling:

LrTasks.yield() 12.4 msec average (too long)
LrTasks.sleep(.00001) 15.6 msec average (expected)

I thought maybe yield was doing same as minimal sleep, but that's not it...

I assume sleep burns one tick then waits for the next (expect 15msec & change)

If yield were just waiting for 1st tick it would take average of 5ms & change - but that's not what's happening.

[dtull]

Some experiments with a plugin that uses lots of individual metadata requests (as opposed to using the batch routines) that shows the same behavior and Windows specific slowness. I've got some buy-in to get a fix for this, so there is an internal bug for it now. I suspect that change is still going to be a little too risky for a 3.x dot, but it does at least give me a good benchmark to use for confirming a fix.

[FeedbackCommunityMember]

Excellent, Dan. It is very good that you are looking into this. I've seen LR to be quite a bit snappier on Macs than on PCs (different Macs and different PCs). While it is likely that hardware differences played a role too, it would be great to eliminate a potential source for unnecessary slowdowns.

[FeedbackCommunityMember]

Dan, you wrote "if I can find a case where it's causing a significant performance deficit (that isn't resolved by reducing yield frequency a bit, which is probably better anyway), I think it'd be possible to implement a higher precision approach for the next major rev.".

Sorry if this is a complete miss, but is there any chance that the mediocre grid scrolling performance in the Library module" (on Windows only?) is caused by the yield timer resolution issue?

It appears that (at least partially) the display of image decorations (badges, etc.) causes the slow down and if these are applied in a loop which contains a "yield" this may have a negative impact?

I feel that the lack of a smooth, fluid scrolling experience in the grid mode is a significant problem.

Sorry, again, if this is unrelated to the "yield" issue.

[dtull]

Funny you should mention that just now. I was just discussing today with somebody that very possibility. I find the grid hitchy on the Mac as well, but since grid scrolling and thumbnails involve a lot of inter-thread communication it could definitely be impacted by the timer. It turns out that fixing the event loop timer issue didn't require as much nudging as I originally expected anyway (there's already a fix in the works).

[areohbee]

So, fixing the event loop timer *did* fix the grid scrolling? Or, does that remain to be seen?

[dtull]

Nope, that's still speculation. The timer change isn't in the builds yet, just in experimentation, so I haven't subjected it to stress testing/benchmarking, etc.

[FeedbackCommunityMember]

Dan, that sounds very good! Thanks!

[johnrellis]

Just did a quickie timing in LR 4 beta on Windows:

- LrTasks.yield() took 20 nanoseconds, as compared to 10 - 15 msecs in LR 3.

- Any Tag takes about 1 second to access 600 keywords, compared to 30 seconds in LR 3.6. An "access" consists of multiple calls to LrKeyword methods.

[areohbee]

I've also noticed some marked improvements in plugin performance, e.g. DevAdjust UI comes up *much* faster :-)

Thanks for informing us John.

And thanks Adobe for attending to this issue, if you're listening...

Rob

[dtull]

You're welcome. The disparity between the two platforms' event loop and inter thread dispatching performance was something that was a really portability pain point and this thread was just the nudge we needed to fix that up.

[FeedbackCommunityMember]

Which Adobe product are you using or asking about?

[FeedbackCommunityMember]

Hi Chris,It is the revel app for I-pad

[Charlie.D]

Hi Nigel

Sorry for the lag--I just caught your post here.

Once your photos are uploaded completely and safely stored online, you can free iOS disk space by purging said photos from your camera roll. Revel will display high resolution thumbs that take less space on your device, and can download original images or full resolution edited versions as needed from the Revel Cloud back to you device, your desktop, etc.

It sounds like you've already installed and tried Revel, and see it consuming iOS disk space. This is because Revel copies original images into its app cache to ensure a smooth and safe upload. This prevents iOS from deleting images before Revel uploads them, but also decreases available storage.

There are a couple things you can do about this:
1) in your Library settings in Revel, tap the 'Clear Cache' button and give Revel a few minutes to process files. This should purge originals that are already safely stored in the cloud. You'll find library settings by tapping the button with 3 lines, then tap the 'i' info button to the right of your library name.

2) if you device storage is already significantly limited (eg., you have 4 GB of images to upload, but only 1 GB of free space), you might consider transferring your photos from the iPad to your desktop and using www.adoberevel.com in a browser to upload your files.

For the future, we are considering ways we might improve Revel iOS cache management, making it's auto-purging more aggressive, so users with limited storage space do not find the app awkward to use.

Thanks,
Charlie