Known Participant

Question

Large import from NAS Drives consumes excess memory

Forum|Forum|3 years ago
August 24, 2022
5 replies
573 views

I just hooked in an external RAID drive and am importing about 500k images. Most are jpegs from jobs but a good % are raw files. I find it confusing that the LR app seems to be importing them one at a time using a single core (out of 20) yet the whole UI is locked up and beachballing while the import works in the background. Even working off my Mac Studio's internal 2TB is unworkably slow. LR is also consuming about 110/128 GB of system ram. Does LR have a memory leak in the import?

macOS

This topic has been closed for replies.

Ian Lyons

Community Expert

How much memory does your Mac Studio have and what is the connection speed (1Gb/s, 2.5Gb/s or 10Gb/s) between the NAS on the Mac Studio?

Sean H [Seattle branch]Author

Known Participant

My Mac Studio has 128GB of LPDDR5. The storage device I've just imported from (second source, same results) is a fusion drive I created with a 1TB WD Black NVME front end and 40Gb/s interface. The main spinner is a 12TB WD enterprice over USB 3.2 Gen 1 10Gb/s (I think Studio defaults to 3.1). This second import was a 140GB of ram footprint.

[ ◉"]

Rikk Flohr_Photography

Community Manager

"ADMIN: I meant this to be a bug report. Can you move to that category?"

When confirmed as a bug, it will be moved back to the Bugs category. It was moved to discussions for confirmation and collection of additional information.

Thanks

Please review https://community.adobe.com/t5/photoshop-ecosystem-bugs/how-do-i-write-a-bug-report/idi-p/12373403 for additional information.

Rikk Flohr: Adobe Photography Org

Sean H [Seattle branch]Author

Known Participant

do you want the spindump and process sample?

spindump

sample process

[ ◉"]

Sean H [Seattle branch]Author

Known Participant

Import is done and app now has 136GB ram allocated.

[ ◉"]

Bob Somrak

Legend

"Simple" Import? 500,000 photos at 100/minute is about 3.5 continuous days. I agree LrC should be able to do this without error but it might be a good idea to import in smaller batches and see what happens. Try 10,000 and work your way up to larger ones.

M4 Pro Mac Mini. 48GB

Sean H [Seattle branch]Author

Known Participant

Oh the 100/min is what I see on an SD card over USB3 (~70MBps). I think the RAID enclosure is closer to 200MBps read. Anyhow, it's not really the speed I'm concerned with. It's the RAM and UI responsiveness. Memory leak.

[ ◉"]

GoldingD

Legend

In the import, what mrthod?

Copy
Copy as DNG
Add

And 500,000 files, heck 100 RAW 26 Mp files can take hours if via Copy from an SD to a hard drive.

Sean H [Seattle branch]Author

Known Participant

They are being added. The external volume is 18TB. Render is 'sidecar'. 100 NEF 24MP files from my SD reader to my WD12TB drives usually takes under a minute including a dupe backup to internal SSD which would mask a leak if there is one.

[ ◉"]

DdeGannes

Community Expert

Just some thoughts.
You stated "Render is 'sidecar'." Please explain.
 Did you previously work with the images in Lightroom or Adobe Camera Raw and you are reimporting to a new Catalog so they have xmp sidecar files with metadata and edits to be applied at import?
What size previews are you building?

Regards, Denis: iMac 27” mid-2015, macOS 11.7.10 Big Sur; 2TB SSD, 24 GB Ram, GPU 2 GB; LrC 12.5,; Lr 6.5, PS 24.7,; ACR 15.5,; (also Laptop Win 11, ver 24H2, LrC 15.0.1, PS 27.0; ) Camera Oly OM-D E-M1.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded