Plugin slow to reimport with large catalog

[Laurence Norah]

Running the latest version of Lightroom classic. I have a large catalog file, around 400,000 images.

When I send an image file with the export dialog to an external editor, edit it, and then send it back to Lightroom, it takes Lightroom about 4 minutes to actually trigger the import that the plugin initialises, presumably via the API.

Example - export image file to DxO Photolab. Edit in DxO Photolab. Photolab closes. A few minutes later (around four in general), Lightroom pops up the import dialog and the file appears in the collections under DxO Photolab. During that time, LR starts using the CPU but not a huge amount.

If I do the same process with a test catalog with 1 image in, the process is almost instant. So something about the large catalog seems to be causing it to be very slow. I have optimised LR as far as I am able to, the catalog has been optimised, and not sure what could be causing this long delay. My system is pretty good (Ryzen 5800x, LR catalog on PCIe SSD, photos on SATA SSD, 64GB RAM, RTX 3060TI GPU), so I don't think there's a bottleneck there.

In addition, if whilst I'm waiting for the import to happen I try sending another file to the external application, I get the attached error popup. It seems LR is getting itself hung up somewhere, but I'm not sure what the fix is. Any ideas?

[dj_paige]

When I send an image file with the export dialog to an external editor, edit it, ...

What happens if you use the "Edit In..." command in Lightroom Classic, instead of the export command?

[Laurence Norah]

Unfortunately, the Edit In command doesn't have an option for DxO Photolab. It's because I'm sending the image as an unedited RAW file to PhotoLab for noise reduction, which then sends a DNG back to Lightroom. As far as I know the "Edit In.." command doesn't generally support the sending of RAW files in the way that the Plugin Export does.

I've been looking at the Lightroom plugin code for DxO against the Lightroom SDK, and it seems that as part of the re-import back into Lightroom the plugin runs a catalog:getAllPhotos() command as part of a check to see if the photo already exists in the catalog. I can see how iterating through 400,000 photos would take time. I figured a duplicate check would be handled as part of the catalog:addPhoto command, but I can't see anything in the documentation to suggest it. I'd love to go back to DxO with some tips for improving their plugin, but I'm getting to the limits of my understanding of the SDK.

[Ian Lyons]

Have a look at below linked post on DxO forum that seems to describe the same symptoms as yourself, and also provides guidance on how it may be resolved.

https://feedback.dxo.com/t/performance-problem-and-fix/13474

[Laurence Norah]

Thanks Ian. Unfortunately, the issue isn't with DxO, it's with the reimport back into Lightroom once DxO is done processing.