cr_sdk_00554910.tmp grows very large (User/AppData/Local/Temp)

[daehxxiD]

I've observed a very strange behaviour with LrC when using the new Enhance Noise Reduction feature.

For some reason, when I run the noise reduction on a large selection of photos (200+) spanning over several albums (photos filtered by HighISO metadata i.e. anything above 3200) at some point Lightroom seems to start writing to my boot drive like crazy:

This never really stops and the file slowly grows so large, that it fills my boot drive completely. The cr_sdk_....tmp file remains that large until I close LR, then it is immediately deleted and the up to 60GB are freed instantly.

This makes it near impossible to use the incredible NR in Batch tasks, which is a common use case for me. Any idea what might be going on here?

I have a hunch, that it has something to do with the Stacking of Images after the creation of Enhanced DNG, but I have not tried this without stacking yet (because the stacking is essential for my workflow).

Any idea what might be going on here? Or has anyone else observed this before? Any viable workaround?

My System:

Ryzen 9 3900X

32GB Ram

12GB RTX 3060 GPU

Win 10

Lightroom Classic 12.3

[daehxxiD]

Quick update: Lightroom just crashed while create the enhanced DNGs with me checking the version number, so there certainly seems to be something that is causing a memory leak of some sort.

[D Fosse]

That's a lot of data to move around. It has to go somewhere. Remember that this is creating new files.

What's the acual size of this temp file (your screenshot doesn't say), and how much free space do you have on your system drive? Unlike Photoshop, Lightroom doesn't have its own memory management and scratch disk. Perhaps it should have. But as it is, it has to use the operating system's pagefile.

[daehxxiD]

The funny thing is, while the drive space kept shrinking, the file still showed 0KB, although it was continuously written to. Yesterday I saw the file grow to 40GB when it completely filled my boot drive an lightroom threw an error, because the catalog was on the same drive that was filled to the brim. 

I've since cleaned up some of the clutter on my system drive and moved the lightroom catalog to another ssd in the system. On C: I now have around 65GB of free storage.

[D Fosse]

Well, 65 GB free space is really not a lot. If you used Photoshop, that would be absolutely inadequate.

Parametric editing in Lightroom has traditionally not used a lot of temporary disk storage, it's been more compute-intensive than I/O-intensive. But that may be about to change with all these new AI powered functions. It generates a lot of temporary data that has to be put somewhere. Especially if you're going to run them in high volume batches like this.

[daehxxiD]

Yeah I agree, which is why i have the Caches for Photoshop (and CameraRAW) on a separate drive. However, the fashion by which this file grows seems to be erratic. Also, it does not happen if I don't span over too many albums (or perhaps albums with a specific name?), even if I "Enhance" 300+ Photos. It only happens in very select circumstances, which makes it feel all the more like a bug.

[D Fosse]

I tried a Denoise batch of 35 files (60 MP so pretty big) - not to exactly duplicate your situation, but just to get a feel for resource management during such a process.

I couldn't see any footprint at all from this. GPU memory usage was steady at 8 GB throughout, system memory usage also a straight line well below max, no buildup.

No disk activity that I could see. Free space on my system drive remained entirely unchanged.

So I have no idea what you're seeing and what causes it. I see no sign of anything similar here.

[daehxxiD]

Yes, that's the thing. It usualy does not do anything on the system drive... Unless the bug hits. System Memory and Video RAM are available in large quantities when this happens, so it shouldn't be virtual memory either.

I've completed the Noise Reduction on a rather large project, which was shot mostly in the dark, for over 900 frames now. So it does work mostly. It would just be nice to figure out why sometimes this memory leak would occur. (filling the OS partition to the brim is generally not the most well-behaved action an app can take)

[daehxxiD]

Just to clear it up. I enhanced the 900 frames in batches of around 200 frames each, without the issue occuring this way (marking up to 3-4 Collections at once).

[daehxxiD]

It just happened again:

I shot a dance recital in rather dim light with a telephoto and wanted to let all shots over ISO 3200 be AI Denoised over night (Lightroom estimated around 180min. processing time for the ~800 photos). As you can see above, the infamous cr_sdk-File has reappeared. Both RAM and VRAM hat plenty of space to spare, so it is really not clear to me, what is happening here.