Copy link to clipboard
Copied
[This technical note was originally published in the old feedback forum, June 4, 2016, but didn't get migrated to the new forum platform:
A number of SDK methods do a yield, allowing other tasks to run while the method is being called. This makes it extremely difficult (practically impossible) to implement robust plugins.
The basic advantage of cooperative, non-preemptive tasks is that the programmer doesn't need to worry about locking. It is trivial (or should be trivial) to write blocks of code that are guaranteed run atomically. But that guarantee goes out the window when many SDK methods can yield to other tasks.
As an example, consider a floating dialog that wishes to obtain the crop information of the current photo in the Develop module. It could do these calls:
photo:getRawMetadata ("width")
photo:getRawMetadata ("height")
LrDevelopController.getValue ("CropLeft")
LrDevelopController.getValue ("CropRight")
LrDevelopController.getValue ("CropTop")
LrDevelopController.getValue ("CropBottom")
But getRawMetadata() can yield, allowing the user to switch to Library, and the calls to getValue() would then return nil, creating a nasty bug. (* See footnote)
Yes, it's theoretically possible to check for every situation like this and think of clever ways to handle each one. But putting aside that there's no documentation on which methods yield, it's very likely that even the best programmer would find it difficult to defend correctly against unexpected yields.
You might think of using LrFunctionContext.callWithContext_noyield()to execute its block of code atomically. But at least with some methods such as getRawMetadata(), wrapping them with callWithContext_noyield() fails with the error "Yielding is not allowed within a C or metamethod call".
Here is a simple script that tests whether a call might yield:
local catalog = import 'LrApplication'.activeCatalog ()
local LrDevelopController = import 'LrDevelopController'
local LrDialogs = import 'LrDialogs'
local LrTasks = import 'LrTasks'
local photo = catalog:getTargetPhoto ()
local n = 0
LrTasks.startAsyncTask (function ()
for i = 1, 10 do
n = 1
for j = 1, 10 do
--catalog:getTargetPhoto ()
--LrDevelopController.getValue ("CropLeft")
--photo:getDevelopSettings ()
photo:getRawMetadata ("width")
end
n = 0
end
end)
LrTasks.startAsyncTask (function ()
for i = 1, 10 do
LrTasks.yield ()
if n == 1 then
LrDialogs.message ("n = 1!")
break
end
end
end)
If the message "n = 1!" appears, then the SDK method has yielded.
----------
* Footnote: There are two solutions for this example. First, the calls to getRawMetadata() (which yield) could be placed after the calls to getValue() (which don't yield). This would ensure that getValue() is called while still in Develop. Second, the calls to getValue() could be replaced by photo:getDevelopSettings(), but that's less satisfactory, since getDevelopSettings() is extremely slow (50 ms per call), making it unsuitable for use in a polling loop (necessitated because an LrDevelopController change observer executes on the main task and can't call most SDK methods).
But the subtlety of both solutions merely illustrates the main point: How is the plugin programmer supposed to anticipate all of these situations and possible workarounds?
Copy link to clipboard
Copied
A number of SDK methods do a yield, allowing other tasks to run while the method is being called. This makes it extremely difficult (practically impossible) to implement robust plugins.
The basic advantage of cooperative, non-preemptive tasks is that the programmer doesn't need to worry about locking. It is trivial (or should be trivial) to write blocks of code that are guaranteed run atomically. But that guarantee goes out the window when many SDK methods can yield to other tasks.
See my post in the feedback forum for more details: Lightroom SDK: Many methods yield, preventing robust plugins | Photoshop Family Customer Community
Copy link to clipboard
Copied
This seems to be a statement rather than a question, but I agree that this could at least be a bit better documented. I ran into the same "issue" very recently when I had to call an "expensive" keyword-list-collecting function that has to start from a call within LrTasks.startAsyncTask()
After endlessly debugging my tables and getting varied results, I realized that a simple period of LrTasks.sleep()"fixed" things. My solution (as it currently stands) was to assign the return values of the function I was calling to global variables and ran it before some other "heavy work" was started (from another .lua file, hence my use of a global variable) and some time before the plugin would be trying to access the values of the new global variables. Since those globals wouldn't even exist until the return finally happened, I could then just use a while loop, inserted at the point (in another file) where the values of those variables were needed, and wait for them to exist (i.e. not be nil... or time out). It was a simple solution that got around the issue of trying to access a table that didn't yet exist (worse, before, I was trying to access elements in large tables that were still being written to, with wildly random results and lots of errors coming from logic assumptions). There should be a (simpler) way to indicate that one SDK process beginning depends on the completion of a particular previous process.
To illustrate a bit better, my context was contributing to a Lightroom plugin on Github that uses an export process to send files to Clarifai (a "computer vision" keywording service). Just before the files are exported to the service, I call on a function to collect all keywords (along with their "ancestry paths" in a hierarchical keyword set) into a lookup table. Each "suggestedTagName" can correspond to multiple keyword objects (though that is something a user should try to avoid, where possible, by cleaning up their keyword list):
LrTasks.startAsyncTask(function()
local catalog = import 'LrApplication'.activeCatalog();
_G.AllKeys, _G.AllKeyPaths = require 'KwUtils'.getAllKeywords(catalog)
end)
When it comes time to build the tagging dialog with the results of the request, we want to ideally add existing keywords to photos. Assuming you have a controlled vocabulary (with a hierarchical structure), you need to find the keywords by the names returned. Iterating the keywords to find all keywords by a given name and create a checkbox for each (on a larger group of photos) was too "expensive", so this function collects all keywords into the table (indexed by keyword names). I won't go into further details (there are many) here, but this is the way I currently pause operations to be sure the keywords table is ready for access (from the lua file that generates the tagging dialog):
-- Before we trim down our lookup table for keywords and keyword paths, we should
-- be sure the process of populating our global variables for these has completed.
local sleepTimer = 0;
local timeout = 30; -- If it doesn't complete within 30s more, something is wrong.
while ((_G.AllKeys == nil) and (sleepTimer < timeout)) do
LrTasks.sleep(1);
sleepTimer = sleepTimer + 1;
end
Now that I think about it a bit more, I'll probably move both of those blocks into my KwUtils.lua file (helper functions for keyword-related tasks), the first block into my "getter" function and the second into a waitForVariable helper function (probably in the KwUtils-dependent LUTILS.lua set of utility functions that I found myself inserting into every project in a more haphazard way). My intention is to share my set of utility functions (inspired by Jeffrey Friedl's JSON.lua package, which the aforementioned project also makes use of) via CC attribution license. I've included them in a few of my own projects, now, and they really help simplify some things.
Okay, I've just re-written that second block into a helper function, so all we have to do, now is call:
local timeout = 30;
local timeWaited = LUTILS.waitForGlobal('AllKeys', timeout);
And the nice thing about writing it into a helper function like that is that it can be easily used for any global ('_G'-namespace) variable that might take a while to be ready for use. This is the helper function:
--Wait for a global variable that may not have been initialized yet. If it is nil, wait.
-- Times out after 'timeout' seconds (or 30s, if only one argument is passed)
-- Returns the number of seconds which were waited (for debug/monitoring purposes) or false
-- if the timeout was reached and the variable name still didn't exist.
function LUTILS.waitForGlobal(globalName, timeout)
local sleepTimer = 0;
local LrTasks = import 'LrTasks';
local timeout = (timeout ~= nil) and timeout or 30;
while (_G[globalName] == nil) and (sleepTimer < timeout) do
LrTasks.sleep(1);
sleepTimer = sleepTimer + 1;
end
return _G[globalName] ~= nil and sleepTimer or false
end
Copy link to clipboard
Copied
@LoweMo_photo This is brilliant, it helped me put an end to three days of searching how to return a value from an asynctask. Thanks!
Copy link to clipboard
Copied
Moderators, @Rikk Flohr: Photography, please merge this thread with the original tech note to which it refers, now copied from the old feedback forum: