Lightroom SDK, Async task issue

[#1]

I have a single diagnostic image, and am trying to export it with each of my many presets applied to it for the purpose of creating LUTs.

I'm trying to loop over all presets in all presets folders, and for each, reset any development settings on the current image, apply the current development preset, and then export the image with the preset name.

I have two issues with the code below. First, it seems to lag an image behind. For example, filename B.png will actually have preset A baked into it. Secondly, I get a stack overflow at some point during execution and am not sure why: I get "An internal error has occurred. ?:0: C stack overflow"

Really, I wish I didn't even have to do any of this with async tasks. This is a one time thing and I'd be happy to just have each preset export block. I added the absurdly long timeout because I was finding the main loop would run super fast and spawn all those async tasks, then most would fail since the first one seemed to be in the withWriteAccessDo call.

Any tips? Happy to do this another way...

local LrDialogs = import 'LrDialogs'
local LrLogger = import 'LrLogger'
local LrApplication = import 'LrApplication'
local LrTasks = import 'LrTasks'
local LrDevelopController = import 'LrDevelopController'
local LrExportSession = import 'LrExportSession'
local LrExportSettings = import 'LrExportSettings'
local LrFileUtils = import 'LrFileUtils'

local MyHWExportItem = {}

function MyHWExportItem.ExportForPreset( i_folder, i_preset )

    local cat = LrApplication.activeCatalog()

    LrTasks.startAsyncTask(function( )

        cat:withWriteAccessDo("Custom plugin", function ( )

            local presetname = i_preset:getName()
            local photo = cat:getTargetPhoto()

            -- Get the settings for the current develop preset
            local presetSettings = i_preset:getSetting()

            -- Turn off sharpening
            presetSettings["Sharpness"] = 0

            local tempLocalPreset = LrApplication.addDevelopPresetForPlugin(_PLUGIN, "", presetSettings)

            -- Reset any prior adjustments
            LrDevelopController.resetAllDevelopAdjustments()

            -- Apply the new preset
            photo:applyDevelopPreset( tempLocalPreset, _PLUGIN )

            -- Configure the export
            local settings = {}
            settings["LR_export_destinationPathPrefix"] = "D:\\mtf.SynSync\\LUTS\\BakedNeutrals"
            settings["LR_export_destinationPathSuffix"] = i_folder:getName()
            settings["LR_tokenCustomString"] = i_preset:getName()
            settings["LR_tokens"] = "{{custom_token}}"
            -- Eliding some of these settings for the forum post. All just k/v 

            local pparams = {}
            pparams["photosToExport"] = { photo }
            pparams["exportSettings"] = settings

            local export_session = LrExportSession( pparams )
            
            -- Export to disk
            export_session:doExportOnCurrentTask()

        -- Before I added this long timeout, the loop would run
        -- and almost instantly spawn all these async tasks for 
        -- all the presets. Then, they'd all run as fast as they
        -- could, and fail b/c they couldn't get write access 
        -- to the catalog, I think...
        end, { timeout = 100000 } )

    end)

end

presetFolders = LrApplication.developPresetFolders()

for fi, i_folder in ipairs (LrApplication.developPresetFolders ()) do 
    for pi, i_preset in ipairs (i_folder:getDevelopPresets ()) do
        MyHWExportItem.ExportForPreset(i_folder, i_preset)
    end
end

[#2]

I have a single diagnostic image, and am trying to export it with each of my many presets applied to it for the purpose of creating LUTs.

I'm trying to loop over all presets in all presets folders, and for each, reset any development settings on the current image, apply the current development preset, and then export the image with the preset name.

I have two issues with the code below. First, it seems to lag an image behind. For example, filename B.png will actually have preset A baked into it. Secondly, I get a stack overflow at some point during execution and am not sure why: I get "An internal error has occurred. ?:0: C stack overflow"

Really, I wish I didn't even have to do any of this with async tasks. This is a one time thing and I'd be happy to just have each preset export block. I added the absurdly long timeout because I was finding the main loop would run super fast and spawn all those async tasks, then most would fail since the first one seemed to be in the withWriteAccessDo call.

Any tips? Happy to do this another way...

local LrDialogs = import 'LrDialogs'
local LrLogger = import 'LrLogger'
local LrApplication = import 'LrApplication'
local LrTasks = import 'LrTasks'
local LrDevelopController = import 'LrDevelopController'
local LrExportSession = import 'LrExportSession'
local LrExportSettings = import 'LrExportSettings'
local LrFileUtils = import 'LrFileUtils'

local MyHWExportItem = {}

function MyHWExportItem.ExportForPreset( i_folder, i_preset )

    local cat = LrApplication.activeCatalog()

    LrTasks.startAsyncTask(function( )

        cat:withWriteAccessDo("Custom plugin", function ( )

            local presetname = i_preset:getName()
            local photo = cat:getTargetPhoto()

            -- Get the settings for the current develop preset
            local presetSettings = i_preset:getSetting()

            -- Turn off sharpening
            presetSettings["Sharpness"] = 0

            local tempLocalPreset = LrApplication.addDevelopPresetForPlugin(_PLUGIN, "", presetSettings)

            -- Reset any prior adjustments
            LrDevelopController.resetAllDevelopAdjustments()

            -- Apply the new preset
            photo:applyDevelopPreset( tempLocalPreset, _PLUGIN )

            -- Configure the export
            local settings = {}
            settings["LR_export_destinationPathPrefix"] = "D:\\mtf.SynSync\\LUTS\\BakedNeutrals"
            settings["LR_export_destinationPathSuffix"] = i_folder:getName()
            settings["LR_tokenCustomString"] = i_preset:getName()
            settings["LR_tokens"] = "{{custom_token}}"
            -- Eliding some of these settings for the forum post. All just k/v 

            local pparams = {}
            pparams["photosToExport"] = { photo }
            pparams["exportSettings"] = settings

            local export_session = LrExportSession( pparams )
            
            -- Export to disk
            export_session:doExportOnCurrentTask()

        -- Before I added this long timeout, the loop would run
        -- and almost instantly spawn all these async tasks for 
        -- all the presets. Then, they'd all run as fast as they
        -- could, and fail b/c they couldn't get write access 
        -- to the catalog, I think...
        end, { timeout = 100000 } )

    end)

end

presetFolders = LrApplication.developPresetFolders()

for fi, i_folder in ipairs (LrApplication.developPresetFolders ()) do 
    for pi, i_preset in ipairs (i_folder:getDevelopPresets ()) do
        MyHWExportItem.ExportForPreset(i_folder, i_preset)
    end
end

[#3]

"I'm trying to loop over all presets in all presets folders, and for each, reset any development settings on the current image, apply the current development preset, and then export the image with the preset name."

(I have no experience with LUA coding or writing scripts)

This can been done by a plugin- 'Excessor'

https://www.capturemonkey.com/excessor/

An option allows you to enter the Preset name for each Virtual Copy.

You can Export each Virtual Copy as you want.

[#4 · Accepted solution]

Rather than starting a new task inside each call to ExportForPreset(), use a single task that wraps the loop over the preset folders:

LrTasks.startAsyncTask(function()
    presetFolders = LrApplication.developPresetFolders()
    for fi, i_folder in ipairs (LrApplication.developPresetFolders ()) do 
        for pi, i_preset in ipairs (i_folder:getDevelopPresets ()) do
            MyHWExportItem.ExportForPreset(i_folder, i_preset)
        end
    end)

In addition, do as little work as possible inside catalog:withWriteAccessDo(), only those calls that must be called from inside it. withWriteAccessDo() is essentially executing a database transaction, and a good rule of thumb (with exceptions) is to avoid long-running database transactions. Scanning your code, it looks like only photo:applyDevelopReset() needs to be called from within withWriteAccessDo().

Finally, you can use the undocumented photo:applyDevelopSettings() to avoid the need for creating a temporary plugin preset. Plugins have been using it for years, and it's just a typical oversight that Adobe hasn't documented it.

[Use the blue reply button under the first post to ensure replies sort properly.]

[#5]

"I have a single diagnostic image, and am trying to export it with each of my many presets applied to it for the purpose of creating LUTs."

The Export LUT plugin can create LUTs directly from multiple presets or photo settings.

[Use the blue reply button under the first post to ensure replies sort properly.]

[#6]

Wow. Thank you *so* much for this incredibly thorough answer, and taking the time to reply!

By the time you'd written back, actually, I'd already purchased your plugin and gotten the job done! Thank you for that, it's wonderful and exactly what I needed. 🙂

Anyway, I decided to give a shot to making the changes you recommended. All seems to be going according to plan now. The only small thing is that, when running this on my two thousand some presets, I noticed the folders on disk with the exported images were created very quickly for, say, the first 8 folders of export, but the remaining ones took much longer to show up.

Specifically:

The first 10 folders were all created and exported to at 9:06PM

Then, 3 more folders 1 minute later

three more 6 minutes later, all at once

roughly this pattern continued until all 35 folders were done at 9:24PM, an 18 minute export.

It's just a bit confusing what was going on in that first 6 minute gap, for example. The UI was furiously updating the histogram of the selected image the whole time as presets were applied, so I expected to see the export writing out as the process ran. Perhaps they all got queued up somehow, or the export slowed as it went on due to some memory issue. Not a big deal at all since eventually the job completed as expected, but more so beacuse I used your plugin to get the job done, just curious and wanted to post my experience here for posterity.

If anyone comes across this thread down the line, here are the edits:

local LrDialogs = import 'LrDialogs'
local LrLogger = import 'LrLogger'
local LrApplication = import 'LrApplication'
local LrTasks = import 'LrTasks'
local LrDevelopController = import 'LrDevelopController'
local LrExportSession = import 'LrExportSession'
local LrExportSettings = import 'LrExportSettings'
local LrFileUtils = import 'LrFileUtils'

local MyHWExportItem = {}

function MyHWExportItem.ExportForPreset( i_folder, i_preset )

    local cat = LrApplication.activeCatalog()

    local presetname = i_preset:getName()
    local photo = cat:getTargetPhoto()

    -- Get the settings for the current develop preset
    local presetSettings = i_preset:getSetting()

    -- Turn off sharpening, vignetting, noise - things that don't work with LUTs
    presetSettings["Sharpness"] = 0
    presetSettings["PostCropVignetteAmount"] = 0
    presetSettings["VignetteAmount"] = 0
    presetSettings["GrainAmount"] = 0
    presetSettings["EnableLensCorrections"] = false
    presetSettings["Clarity2012"] = 0
    presetSettings["Dehaze"] = 0
    presetSettings["Texture"] = 0
    presetSettings["LensManualDistortionAmount"] = 0
    --presetSettings["Shadows2012"] = 0
    --presetSettings["Highlights2012"] = 0

    -- With write access block
    cat:withWriteAccessDo("Custom plugin", function ( )

        -- Reset any prior adjustments
        LrDevelopController.resetAllDevelopAdjustments()

        -- Apply the new preset
        photo:applyDevelopSettings( presetSettings )

    end, { timeout = 100000 } )

    -- Configure the export
    local settings = {}
    settings["LR_export_destinationPathPrefix"] = "D:\\mtf.SynSync\\shared_dev\\lut\\BakedNeutrals"
    settings["LR_export_destinationPathSuffix"] = i_folder:getName()
    settings["LR_tokenCustomString"] = i_preset:getName()
    settings["LR_tokens"] = "{{custom_token}}"
    settings["LR_collisionHandling"] = "overwrite"
    settings["LR_export_bitDepth"] = 16
    settings["LR_export_colorSpace"] = "sRGB"
    settings["LR_export_destinationType"] = "specificFolder"
    settings["LR_export_postProcessing"] = "doNothing"
    settings["LR_export_useParentFolder"] = false
    settings["LR_export_useSubfolder"] = true
    settings["LR_extensionCase"] = "lowercase"
    settings["LR_format"] = "PNG"
    settings["LR_outputSharpeningOn"] = false
    settings["LR_png_interlaced"] = false
    settings["LR_renamingTokensOn"] = true
    settings["LR_size_percentage"] = 100
    settings["LR_size_resolution"] = 240
    settings["LR_size_resolutionUnits"] = "inch"
    settings["LR_useWatermark"] = false

    local pparams = {}
    pparams["photosToExport"] = { photo }
    pparams["exportSettings"] = settings

    local export_session = LrExportSession( pparams )
    
    -- Export to disk
    export_session:doExportOnCurrentTask()

end

LrTasks.startAsyncTask(function()
    presetFolders = LrApplication.developPresetFolders()
    for fi, i_folder in ipairs (LrApplication.developPresetFolders ()) do 
        for pi, i_preset in ipairs (i_folder:getDevelopPresets ()) do
            MyHWExportItem.ExportForPreset(i_folder, i_preset)
        end
    end
end )

[#7]

"The only small thing is that, when running this on my two thousand some presets, I noticed the folders on disk with the exported images were created very quickly for, say, the first 8 folders of export, but the remaining ones took much longer to show up."

The slowth could be due to a long-standing, trivial performance bug in the SDK that Adobe has chosen not to fix:

https://feedback.photoshop.com/photoshop_family/topics/lightroom-sdk-severe-performance-bug-in-devel...

Enumerating all n develop presets takes O(n^2) time, whereas it should take O(n) time. It's a rookie programming mistake, sadly.

Please add your constructive opinion to the bug report, and be sure to click Me Too and Follow in the upper-right corner. That will make it a little more likely that Adobe will prioritize a fix, and you'll be notified when the bug's status changes. (Safari users, uncheck the option Safari > Preferences > Privacy > Prevent Cross-Site Tracking or use Chrome or Firefox.)

[#8]

Thanks again for your thorough answer John. I suspect you're right about that O(n^2) issue. I've read through the bug page, and agree all signs do seem to point to a flat list being scanned over and over again. I've added a comment and clicked Me Too and Follow as well.