DNG Converter is taking over all available ram and processor (64G system with 32 threads)

[dwterry]

I walked away after loading tonight's session and after about a half hour came back to discover the DNG Converter still had not finished converting about 900 files. That is crazy long on my system. I noticed that moving my mouse was jerky, and that the conversion itself appeared frozen. So I aborted it. Moved all of the finished files and restarted with only 211 files left to go.

After it had finished about 100 of the images, the system was again extremely slow and the conversions were taking forever. So I brought up Task Manager. You can see what it looked like in the screen capture down below (100% cpu utilization and 87% memory).  This is on a Windows 11 machine with 64GB of memory and an AMD 3950 with 16 cores (32 threads) all of them pegged at full utilization.

I suspect the problem is a memory leak causing the DNG Converter to usurp all available memory and then the system just drives itself into the ground when there's very little memory left.

This occured using version 16.0.1.1683 with the following settings:
JPEG Preview: Full Size
Embed Fast Load Data

Don't use lossy compression
Preserve Pixel Count
Don't embed original

And this happened while processing files from my fairly new Sony a7iv  (not 'R'). 

I have had a Sony a7Riv (different from above - this one is a 60mp camera) for a couple of years now and have experienced "slow-ish" conversions, but nothing approaching this level of sluggishness nor taking over the system CPU and memory the way this one is.  So the problem might be a combination of the new camera (a7iv) and the settings I'm using (see above).

Here is a screen shot of what Task Manager looked like after converting about 100 of the a7iv images:

[PECourtejoie]

Moving to Camera Raw forum, so that @Rikk Flohr: Photography can notice it.

[dwterry]

Just reporting back... I noticed there was a newer version today so I downloaded it.  Version 16.1.0.1728.

Same problem. It very quickly zooms up to 100% memory and cpu utilization.  I let it convert a couple hundred, cancel the conversion. Kill it via Task Manager (because it takes FOREVER to exit even though the UI has closed). Then restart it on the rest of the files.  It's tedious restarting multiple times, but it's faster than letting it get stuck forever at full cpu/memory utilization.

[Lumigraphics]

Does Lightroom, Adobe Camera RAW, or Bridge (using the Export feature) have the same problem?

[dwterry]

I can't answer that.  I use the DNG Converter to convert all of my raw files to dng *before* importing to Lightroom, I've been doing it this way since 2018 when I switched to Sony cameras.  Maybe someone who uses one of the other workflows can try it out? 

Because it looks like a memory leak, I do believe the quickest solution would be to get one of the developers to look into ways that memory might not get freed after an image is converted.

[Lumigraphics]

So do a test?

[Patrick Baldwin]

I am having the same problems but with a M3 Max Macbook pro and Nikon Z8 files. It is just ridiculously slow. I am trying to convert 6000 at a time some days and it has become unworkable. Like over 12 hours to convert them. Its almost as if the app has to be in front on screen to work properly. When left unattended and my screens go to sleep it seems to just slow right down. The whole system is set for max performance at all times. It is baffling and extremely frustrating. It was always a time consuming thing but this is crazy. My conversion settings are the same as yours. My version is 15.3.1.1455 but given what you said the version doesn't really seem to matter at the moment.

[Lumigraphics]

Maybe do the conversion in Bridge or Lightroom instead?

[Patrick Baldwin]

I did a test with 200 50mb images using conversion from Bridge and the standalone converter. From Bridge it took 4 minutes 23secs and the standalone converter too 7 mins 30 secs so over 6000 odd files that would be a 90 minute saving which is well worth having. It will be interesting to see if the Bridge conversionslows down over time because the standalone is. It took way more than 4 hours to do 6000 from my last job. More like 14 hours. I confess I didn't actually know you could convert straight from Bridge. I have been using the dng converter since it was first launched. Thanks for pointing me in that direction.

[Patrick Baldwin]

To update my last entry. It seems that the standalone DNG converter slows right down when my screens go to sleep. It's odd beccause everything is set not to sleep or reduce power etc. I watched it convert 1500 files in about 90 minutes, left it running and went to bed with about 3500 still to conert. I got up about 6 hours later, woke the screen and it still had 200 to convert! I ran the same test using the converter in Bridge on the exact same files and all 5000 file took about  4 or so hours. So the dng standalone is seemingly affected by the screens going to sleep which seems very weird. It also quietens right down once the screens go to sleep so if the fans ere running during conversion they would fairly quickly shut off once the screens went to sleep. All really odd.

[Patrick Baldwin]

Update. My previous test that suggested that the DNG converter in Bridge was faster is actually not the case. I had the preview set to medium which makes the whole thing faster. I need full size previews so it actually is no better than the standalone converter. What I cannot understand is that if the converter is not the foreground app it appears to slow right right down. I set it to convert 1700 50MB files at 3am. By 8.30 am there were still 600 to convert. My computer is not set to sleep or reduce power when not in use but waking the screen and making the converter "front" app meant that the remaining 600 took less than hour to convert. This is so frustrating because I cannot seem to find a way to stop this slowdown. It seems to be a memory leak but when I am sitting in front of it and I make the app come to the front it all speeds up again!!

[mackers]

Bump! This issue seriously needs to be addressed. I redownloaded v14.4 and it is x4 faster as I measured it. The current version v16 seems to reduce file size more compared to the older versions which is nice, but the program frequently crashes now too!