Having a problem using catalog:withWriteAccessDo

[drtonyb]

I'm having a problem using catalog:withWriteAccessDo() in a function attached to an adjustment change observer.

Here is a code fragment from my plugin

local function resetMykey( observer )
  local writeAccessStatus
  local catalog = LrApplication.activeCatalog()
  local photo = catalog:getTargetPhoto()

  if nil ~= photo and not catalog.hasWriteAccess then
    -- gets here
    writeAccessStatus = catalog:withWriteAccessDo( 'Reset mykey', function( context )
      -- doesn't do this
      photo:setPropertyForPlugin( _PLUGIN, 'mykey', nil )
    end, { timeout = 15 } )
    -- and never gets here
  end
end

...

LrDevelopController.addAdjustmentChangeObserver( context, Observer, resetMykey )

When a setting is changed in the Develop modue, the function resetMykey executes as expected. It gets to the line with catalog:withWriteAccessDo, indicated by my comment gets here, but it doesn't execute the withWriteAccessDo function and therefore, doesn't execute the photo:setPropertyForPlugin function, indicated by my comment doesn't do this. The withWriteAccessDo doesn't timeout either, indicated by my comment and never gets here; it seems that the withWriteAccessDo function just dies a silent death, but doesn't kill off the plugin.

Since the withWriteAccessDo never returns, I can't inspect the returned writeAccessStatus, so I'm completely in the dark after spending many hours trying to resolve the problem.

Any insights from SDK gurus would be most appreciated.

[johnrellis]

I think you're tripping over two issues.

1. When an error occurs, the SDK often silently terminates the task without displaying error dialog. It's gotten a little better about this over the years, but errors still get silently discarded in many contexts, including many/most/all? callback functions passed to SDK API methods.

Often the errors get recorded in the file "lrc_console.log" in the Lightroom settings folder (Preferences > Presets > Show All Other Lightroom Presets), which started appearing some versions ago, not sure when. This usually/often records the Lua call stack whenever an error is thrown, regardless of whether it is caught by pcall().

To ensure that you get notified by an error dialog whenever an unhandled error occurs, you can use LrTasks.pcall() (not the built-in pcall) at the top level of any function passed as a callback to the SDK, including the top-level functions passed to LrTasks.startAsyncTask() and related methods. 

Or you can use LrFunctionContext.callWithContext() and LrDialogs.attachErrorDialogToFunctionContext(), which is quite wordy to do with every callback function. (It uses pcall() in is implementation.)

Or you can use my Debugging Toolkit, a lightweight debugger tailored to LR, and wrap every callback function with the toolkit's "showErrors(func)".  It's a little tedious to set up and get comfortable with, but once you get it set up and work with it for a couple hours, it's the most productive way I know of for developing plugins.  The Toolkit has other conveniences, like a pretty printer for arbitrary Lua values that other tools seemingly lack, and read-eval-print loop for executing code snippets interactively (a great aid for reverse engineering undocumented behavior).

The Zerobrane IDE supports Lightroom debugging, but it requires you to similarly annotate every callback and task you want to debug (due to LR's task architecture).  I've seen references to using a plugin for VSCode to debug Lightroom plugins but haven't investigated. Almost certainly it has the same requirements for annotating your LR code.

I'm a minimalist and use Sublime and my Toolkit. Nearly all of my development time is spent reverse-engineering LR's undocumented behaviors and tripping over LR bugs and oddities.

2. Now for the error you're tripping over, many of the SDK's methods must be called from what it calls an "asynchronous task" (a task created by LrTasks).  The "main task" is the one that handles the user interface and on which all plugins start executing.  One undocumented behavior is that callback functions for LrView UI controls and LrDevelopController are executed on the main task, even if they are passed in from calls in asynchronous tasks.

If you try to call one of these SDK methods from the main task, you'll get an error, often "Yielding is not allowed within a C or metamethod call". 

The documentation is pretty good (but not perfect) about calling out these methods.  I see that the documentation for catalog:withWriteAccessDo() doesn't do so.

So many methods require execution on asynchronous tasks that I simply start each dialog-based plugin with this idiom:

LrFunctionContext.postAsyncTaskWithContext ("", showErrors (main))

* * *

In your example, you want to write to the catalog inside a callback from LrDevelopController.adjustmentChangeObserver().  The callback will need to create an async task from which to actually call catalog:withWriteAccessDo(). But that creates the possibility that these tasks will execute out of order. Whether that happens in practice in your case, I have no idea. 

Some of my interactive plugins, like Any Crop, have encountered such problems.  Sometimes I've programmed these tasks to execute serially using a ticketing system with spin/sleep loops (sleeping as little as 0.001 seconds). Other times I've designed the code to avoid the need by recording actions in variables and then persisting the variables in the catalog (or files) outside of the callbacks from another async task.  (The LrPrefs preferences object essentially uses this approach.) And other times, I don't use change observers at all, instead having an async task poll for changes.

This is an area where the LR SDK's architecture is weakest.  Adobe abandoned most development of the SDK around LR 4, with its senior designer/engineers moving on to other things (or companies), leaving a number of such dark corners in the SDK design and implementation.

[drtonyb]

@johnrellis 

Thanks John for your detailed reply.

I did work it out eventually with this modification

local function resetMykey( observer )
  local writeAccessStatus
  local catalog = LrApplication.activeCatalog()

  LrTasks.startAsyncTask( function()  -- needed this
    local photo = catalog:getTargetPhoto()

    if nil ~= photo and not catalog.hasWriteAccess then
      -- gets here
      writeAccessStatus = catalog:withWriteAccessDo( 'Reset mykey', function( context )
        -- does do this
        photo:setPropertyForPlugin( _PLUGIN, 'mykey', nil )
      end, { timeout = 15 } )
      -- and now gets here
    end
  end )
end

Although the main task was called with LrTasks.startAsyncTask, the observer function resetMykey above needed to have its own LrTasks.startAsyncTask. I'm guessing that when the observer function resetMykey is called, it's outside the main AsyncTask. I probably don't need the not catalog.hasWriteAccess condition, but it's not doing any harm.

[johnrellis]

"Although the main task was called with LrTasks.startAsyncTask, the observer function resetMykey above needed to have its own LrTasks.startAsyncTask."

Right, as I wrote above, callback functions for LrView UI controls and LrDevelopController are executed on the main task, even if they are passed in from calls in asynchronous tasks.  That's why the call to catalog:withWriteAccessDo() was failing -- it can only be called from asynchronous tasks.

But as detailed above, beware that if the change observer is called rapidly, there is a possibility that the async tasks containing the call to catalog:withWriteAccessDo() could be executed out of order.

[drtonyb]

@johnrellis 

"But as detailed above, beware that if the change observer is called rapidly, there is a possibility that the async tasks containing the call to catalog:withWriteAccessDo() could be executed out of order."

The observer function does get blasted every time an adjustment is made; even creating a VC triggers the observer function four times. In my case, this rapid succession of triggers shouldn't be a concern for my observer because it is always doing only one thing: setting my custom metadata field to nil. A small rearrangement using not catalog.hasWriteAccess might prevent multiple LrTasks.startAsyncTask() while a catalog:withWriteAccessDo() is active. I will check.

The one thing I actually wanted from the LrDevelopController.addAdjustmentChangeObserver() was to be notified when the Profile settings is changed in Develop, but that is about the only adjustment that isn't observed. The Profile cannot be read or set from the getValue() and setValue() functions, and there aren't any specialised functions for managing the Profile either. Is that an oversight or is there something hidden from us?

Correction: changing the Profile does trigger the observer function, the value just can't be read (or set)

It would also be nice if a change observer could be registered for only selected adjustments.

I should also add that the code I've been developing in my plugin has been derived from studying the Lightroom Controller SDK Guide.

The example plugin code given in this guide will only work if the plugin is loaded when LrC is in the Develop module, otherwise it won't work. A key part of this plugin example is

LrDevelopController.addAdjustmentChangeObserver( context, senderObserver, updateDevelopParameters )

The SDK documentation clearly states that addAdjustmentChangeObserver() must be called while the Develop module is active; if it is not, as in the case when LrC is launched in the Library module for example, the function fails to register the change observer and nothing will happen when adjustments are made in Develop. Another trap for plugin developers trying to learn from the poor SDK documentation.

This example plugin also uses the LrTableUtils class which isn't documented at all in the LrC SDK API Reference.

I am also aware of many other omissions from the LrC SDK API Reference which you John have discovered and documented.

Some years ago I had an Adobe developed plugin, which I can't find now, that used a similar set of SDK classes that weren't the same as those we see in the LrC SDK API Reference; once again, no documentation on these.

Bad documention is such a time waster.

[johnrellis]

"the LrTableUtils class which isn't documented at all in the LrC SDK API Reference."

It first appeared (undocumented) in LR 4 and/or 5:

https://community.adobe.com/t5/lightroom-classic-discussions/lr-5-functions-methods-and-properties-n...

[johnrellis]

[This post contains formatting and embedded images that don't appear in email. View the post in your Web browser.]

"The Profile cannot be read or set from the getValue() and setValue() functions, and there aren't any specialised functions for managing the Profile either. Is that an oversight or is there something hidden from us?"

It's undocumented, as much of the develop settings are. 

If the profile is a DNG camera profile (stored in a .dcp file), it is represented with this key/value pair in photo:getDevelopSettings():

    CameraProfile = "Adobe Standard", 

If the profile is an enhanced profile, stored in a .xmp file, it is represented with two keys:

    CameraProfile = "Adobe Standard", 
    Look = {
        Amount = 1, 
        Copyright = "© 2018 Adobe Systems, Inc.", 
        Group = {
            ["x-default"] = "Profiles"}, 
        Name = "Adobe Color", 
        Parameters = {
            CameraProfile = "Adobe Standard", 
        ...}

All or nearly all such keys can be provided to LrDevelopController.get/setValue(), e.g.

When the API first started exposing developing settings in LR 3, the reference for photo:getDevelopSettings() added this warning (still present):

Adobe has mostly kept the list of setting keys up-to-date but made no attempt to document their values -- you have to reverse-engineer those. Most are obvious but some, especially those with masking, remove, and the other AI commands, are more obscure.  

You might find my utility Show Metadata plugin useful for examining all the metadata and develop settings exposed by the SDK API and Exiftool:

https://www.dropbox.com/scl/fi/hagpmldch9q60ns4u4btq/showmetadata.1.9.zip?rlkey=06u2fautb3azdpgutw7m...

[johnrellis]

"Some years ago I had an Adobe developed plugin, which I can't find now, that used a similar set of SDK classes that weren't the same as those we see in the LrC SDK API Reference"

The SDK has several sample plugins in the Sample Plugins subfolder that are referenced by the Lightroom Controller SDK Guide (in the Manual subfolder). They went missing from the downloaded SDK for a few years but then were restored in LR 13.3:

https://community.adobe.com/t5/lightroom-classic-discussions/changes-to-the-lr-13-3-sdk/m-p/14636965

[drtonyb]

@johnrellis 

I found the script that uses a different set of SDK classes. It was a script rather than a plugin. It is available here:

https://helpx.adobe.com/au/lightroom-classic/kb/extract-previews-for-lost-images-lightroom.html 

but in compiled code, and in two versions: an original and an updated version.

I'm not sure which version I have, but when I downloaded it in June 2024 m(so it could the be updated one), it wasn't compiled. In addition to the Lr prefixed classes, the script uses Ag prefixed classes, such as "AgBlob", "AgLibraryCustodian" and "AgPreviewUtils". Some are similar to their Lr prefixed counterpart, such as "AgFileUtils", but the difference between them is unknown to me.

It is sad, and quiet limiting, that plugin developers don't have the full SDK documentation.

[johnrellis]

"the script uses Ag prefixed classes, such as "AgBlob", "AgLibraryCustodian" and "AgPreviewUtils". "

Scripts run from the Scripts folder can import internal LR modules, whose names usually start with "Ag". Of course, Adobe doesn't disclose the documentation for these modules.

In years past, Adobe has very infrequently relied on this ability of scripts to deliver various kinds of functionality and fixes outside of the normal LR app.

The "restart" samizdat script is my most used script/plugin by far:

import "AgApplication".relaunch ()