P: (Win) Consumes RAM during import and fails to release afterwards

[perfect_goal1587]

I notice that LR eats all memory when you start importing images.

When opened, it takes about 1.5-3GB ram. Then import 100 images, and RAM shoots up to +17GB

Iv been doing some tests with smaller raw files, same thing, always goes up to 17GB ram usage. The folder i imported as test contains only 5GB of raws, and LR uses more then twice that amount to import those images? When you delete the images, 17GB ram is still in use. Then tested less images, like 30, and memory still goes up, but less, around 5GB ram extra, for images that on disk only take around 1GB. Importing more images, like +300 will not make memory to increase more then 17GB.

Solution? Import images, restart LR to clear memory, edit... and ram usage will stay "normal" around 6GB during edit after some testing. Thats 10GB less then just doing an import. Whats up with that?

Im on 13.5 btw

[Moved from ‘Bugs’ to ‘Discussions’ by moderator, according to forum rules.]

[David24583147fm9k]

I have learned that the slowdown does not appear to be the result of cumulative work. It happens only with specific pictures. I will be working without incident, then one takes forever to respond and the GPU is maxed out. If I go to another picture, everything is fine. All the pictures, which are jpegs, are initially the same size. The number of masks doesn't appear to be an issue.

[steveisa054]

This what you describe is different than what I experience. I am working with large files, as RAW they are about 35 MB, and become 100 to 200 MB once converted to a PSD or TIF format. For me, the huge jump in utiized memory (and it seems to be the motherboard RAM and not the GPU RAM, but maybe this is just something the Task Manager and my older BIOS cannot figure out correctly) up to as much as 18 GB occurs just about every time I do an edit in Lightroom to an image which would require information to be stored on a pixel level. If I work with masks which can be described only by a few parameters (linear gradients, radial gradients) I do not seem to have this problem. But as soon as I start working with masks based on Subject, Background, Brush where the selection information for the mask necessarily needs to be on a pixel level, that is when Lightroom seems to suddenly grab a lot of memory and not release it. At the moment this is happening very often, and I have gotten used to a new workflow: I do some edits in Lightroom and if I need to edit in Photoshop (from where I do any Topaz or NIK work), I shut down Lightroom completely as soon as I have the image opened in Photoshop. After I am done in Photoshop and have saved the flattened image, I then open Lightroom again, and then I synchronize the folder to bring my newly edited image into my catalog. It is cumbersome, and I am hoping my new machine will not have this problem. Of course I also need to restart Lightroom every time it does freeze my PC while working with masks.

And while I have been writing this reply, my machine proved again how screwed up it is. I edited an image in Lightroom, without needing to use masks, but I did do a denoise enhancement. Lightroom was only using 6 GB of RAM at this point. I edited in Photoshop without closing Lightroom down this time. I then tried to edit in Topaz Sharpen from a layer, calling Topaz as a filter. And Topaz could not give me the sharpened preview, it failed with an error message. I get this, often, if I try to work in Topaz using the GPU for calculations while Lightroom is open, even if Lightroom is not really using much memory. What I need to do here is to close down Topaz, close down Lightroom, then open Topaz once again from Photoshop, and then it will work.

[Mitkó]

The same thing happens to me... Do you think a 2022/3 version of LR will run smoother?
I had an i7 13700k (no dedicated gpu) 16 GB 4800mhz of ram and I could do wonders in LR, but now I have AMD 9 7950x3d with 32GB G.Skill Trident Neo CL30 6000mhz and my pc is performing like a teenager, meaning you never know when it's gonna freeze or stop working completely. So LR crashes my system and I dare say my system is top-notch, the motherboard is an Asrock x670E Taichi... Nevertheless, I literally can't do 1/3 of what I could 5 months ago with my old PC..

[Mitkó]

Forgot to say that I have 1 image opened in LR... 1 image! And when I try to brush out smth from a mask everything just freezes after a few seconds.

[D Fosse]

(no dedicated gpu)

By @Mitkó

And that's the problem. You really need that nowadays. Some years ago you could get away without it.

[Mitkó]

I understand, but I am describing a situation from 4-5 months ago, did you go through my post? My 32GB RAM gets eaten by 3 masks and a few general corrections? I don't think that's because of the integrated GPU, BUT, I must admit that, it seems the integrated GPU eats another 15gigs of ram from the "shared" portion.

[steveisa054]

Sounds like a similar situation to what I experience. I do not know if it is due to my older chipset and not updating the BIOS so that Lightroom is allocated more and more memory without releasing it, or if this is just an unsolved bug in Lightroom affecting select user hardware configurations. But it must require a huge amount of memory to store information about masks on a pixel rather than parametric basis (e.g. brushes, subjects, background, people). My Lightroom can suddenly surge to 18 GB RAM usage and the only reason it cannot go higher is that there simply is no more to grab. My new PC will have 64 GB RAM and also a newer BIOS and chipset so I hope that solves the problem.

By the way, just noticed right now. On my PC, Photoshop is not acting so angelic either with memory. I had Photoshop opened to work on an image, which had a total size of under 1 GB. I left Photoshop open, but closed all images, and the memory allocated by Photoshop did not decrease at all by this. It is still taking over 6 GB of memory even though no image is opened. I do not know if my situation is getting worse, but with my workflow of Lightroom to Photoshop to Topaz or NIK products, back to Photoshop and back to Lightroom, I now find I have to constantly shut down Lightroom and Photoshop whenever I am just finished with them. That means that when I do an "Edit in Photoshop" from Lightroom, I then have to close down Lightroom or I cannot get the Topaz softwares to work without error from Photoshop.

In fact, the situation is more bizarre if I open up the 8 processes associated with Photoshop, see the image below. The 6.4 GB being used by Photoshop seem to be of my last edited image (which like I said only had a size of under 1 GB) even though I have closed this image down. This is really totally messed up with regards to memory allocations.

[D Fosse]

@Mitkó Yes, an integrated GPU uses system RAM, and lots of it. That can sometimes be a problem. A discrete GPU doesn't do that, it uses its own VRAM and stops there.

[D Fosse]

@steveisa054  I'll just answer regarding Photoshop.

Photoshop's memory rquirements go way beyond the nominal starting file size. Every history state for every open document can potentially add the full file size. Smart objects have additional overhead. All that data has to go somewhere.

In practice, there is no such thing as "enough RAM" no matter how much you have. It will very quickly get saturated. That's why Photoshop has a scratch disk, and that's also why you set a limit to memory allocation in Preferences. That limit should normally not be higher than 70% to avoid choking the whole system.

The point is, it's not Photoshop. It's the sheer amount of data that needs to go somewhere.

Photoshop will not release memory when you close a file. This is by design, because constantly requesting memory from the operating system is slow and inefficient. The memory is recycled and reused.

In short, to work with Photoshop, you need to have enough memory and disk space, and set a sensible allocation in Preferences.

[steveisa054]

Thanks, I can follow along what you are saying. And I think I will solve some of these issues when I get my new PC up and running (I am so close now, this coming weekend maybe ...) as I am starting off with 64 GB motherboard RAM using 2 of the 4 available slots. I can easily expand to 128 GB.

[Mitkó]

@D Fosse , in this case, what is your opinion as to Radeon RX 7600XT 16GB, is it a good choice for working with LR/PS/PP?

Thanks!

[Callie253560770b53]

I just wanted to add that I'm having that exact same issue as you. LR starts seriosly stuttering when I manipulate masks. I have a pretty modern PC: AMD Ryzen 9 3900x, RX 5700 XT 8gb, 62gb of DDR4 at 3600 MHz, 1tb nvme ssd and 5tb raid drive for photos. OS: Win 10 Pro (fully updated). Lightroom Classic: v.13.1

I would love to find out if other people haven't been experiencing this issue. I believe that the problems started when the new AI features were added.

I think that I have further narrowed down the issue to the GPU memory (not entirely possitive though). Although Lightroom is reserving a ton of my system memory 16gb+; looking at the 'system info' in Lighroom, I can see that my Lightroom is running in to a 'Real Memory' issue; however, I"m using 40gb of virtual memory which seems high but I wouldn't expect these very specific issues unless there is a coding issue. However, my Graphics card has 8gb of on-board storage and it is hitting crazy high percentages 155% (13+gb) according to 'system info'. It even sits high when I'm not doing anything (~130%).  

[Mitkó]

Hi @Callie253560770b53 , I resolved my Ram issue with Lightroomm by bying myself the 16GB  Radeon RX 7600XT, it's smooth like butter now, feels like it just flows. So, the point being is that the dedicated gpu's memory turns out to be the solution to the newest version of LR.

[Callie253560770b53]

@Mitkó I'm glad to hear that you solved the issue. Unfortunately that nice of a graphics card is at least a couple of months out for me.

Also, I wanted to correct my original post to say "I can see that my Lightroom ISN'T running in to a 'Real Memory' issue."

Also, I wanted to add after playing with it further that I think that there may still be a memory leak. This problem definitely gets better for a time when I restart the program then gets worse and worse until it isn't usable. It also starts to get bad when applying the AI masks and becomes basically unusable when trying to edit those masks. Also, I feel pretty confident based on what you said that when you have a dedicated GPU, this is a GPU memory issue. 

[Mitkó]

You, in my opinion, are totally right. The issue, essentialy is in LR as memory is obviously leaking big time, it feels almost like LR is made for people to comply with its leaks and bugs instead of the opposite. Now, this being said, just 5 months back, I was able to do all that I do now with an iGPU instead of a dGPU, with 0 issues or ram leaks.

I believe you got to the bottom of it tho, good job for those observations. I hope you'll manage soon to get yourself what's needed for your pc to run LR's issues faster (slight irony here). 😌

[steveisa054]

Interesting and good to read that the Radeon RX 7600XT is working well for you, Mitkó. Did you get one with 2 or with 3 fans? I am looking at the price, and it is a bit higher than what I bought (RTX 3060 12 GB) but maybe I should have gone for that one for the 4 extra GB RAM. With my graphic card, Lightroom denoise is quick, but even if I only have Photoshop calling Topaz Sharpen (with Lightroom shut down), Topaz does not work really as fast as I could hope for.

[Mitkó]

Thanks mate, appreciate it! Finally, after 3 months and a half of headbanging and buying 3 completely new PCs... I don't even wanna talk about it... Back to your question - I bought the 3-fan one, tho I gotta be honest with you - most of the time I see that the dGPU ram is fully used and if I stop using LR for a few mins, but I leave it open, the ram utilization of the dGPU drops with the incredible 2gigs.

[D Fosse]

I can't help but think there are some misunderstandings here. High memory usage in itself is normal and desirable.  Free memory is wasted memory. This is caching; it's how memory should be used.

It gets reallocated constantly. Note the dip here; this is after going crazy with AI masks - and then just switching to a different frame in Develop.

A memory leak is when memory usage grows without any activity, until it hits the ceiling and runs out. I've never seen that. It maxes at around 80-85% and never goes above that.

There are no performance differences with high or low memory usage.

I do suspect, however, that this can be an issue with integrated GPUs, because the GPU and system compete for the same memory. In that case timing matters, if, say, an application requests memory already occupied by the GPU or vice versa. That's not a consideration with dedicated GPUs.

[steveisa054]

Regarding the comments on memory usage, I cannot really agree that what is happening, with my older PC, should be considered normal. At any rate, it is not desireable. I am not using an integrated GPU for the record. It is not ok that Lightroom is having a memory allocation of 12 to 18 GB when all I have in my machine is 32 GB RAM and this Lightroom allocation brings my total memory usage to 100%. When this happens, nothing more happens. The mouse does not move, I cannot even use Alt-Tab to cycle between opened windows. All I can do is wait patiently until Windows pages enough RAM memory onto the hard drive so that there is enough RAM memory available for the usual Windows functions to start responding again. And when this happens I use the opportunity to shut Lightroom down so that all that 12 to 18 GB of RAM becomes freed up again. I then open Lightroom once more, and continue editing where I had left off, but now that Lightroom has been shut down and reopened, the Lightroom RAM usage is down to between 2 to 6 GB RAM. Maybe this means that to use Lightroom and Photoshop, 32 GB RAM no longer is sufficient?

I suppose there are two things being discussed here now, the memory being used by the GPU and the memory that Lightroom seems to be grabbing presumeably due to the high memory needs of the new masking possibilities. In my case, I know Lightroom is using the graphic card RAM, but I do not know how much of the reported RAM usage is from the motherboard or from the graphic card. Perhaps my BIOS cannot figure it out, and that may be the problem. I just know that when things freeze, and I can get Task Manager opened, the memory utilization is around 100%.

[D Fosse]

Well, the principle is the same whether its system RAM or GPU VRAM.

There has to be some other problem here. I also have 32 GB system RAM, and I never see any of these problems. Everything is snappy at all times. I work with big files (60MP), and I usually have Lightroom, Photoshop, Bridge and several other applications open at the same time.

So the question is - where are these problems rooted. There's nothing inherent in LrC that causes this.

[steveisa054]

I know. I am guessing it is the mix of a 12+ year old MSI motherboard and a never upgraded BIOS with new software and Windows 10 and a new graphic card having newer internal technology. Therefore I am not trying to solve it really, as hopefully this weekend I will get my new PC up and running. I am a bit slow with it as I am moving my graphic card over to it. I had a conversation with a Topaz supporter about the weird situation that GigaPixel 6.3.3 would not do any facial reconstruction on my blurry images while two versions earlier (6.1.0 maybe) worked ok. Strange. But the supporter said something about maybe the older instruction set of my aging motherboard and BIOS could be to blame. This is beyond my knowledge and understanding. But maybe there is an incompatibility somewhere with memory handling.

[steveisa054]

ust another observation. I edited a photo in Lightroom just now, with a number of people based masks. I had to restart Lightroom underway due to freeze of my PC. I then did an "Edit in Photoshop" to do more work. Before I shut down Lightroom, I had a look at the GPU memory usage in Task Manager on the Performance tab. I was still in the Lightroom Develop module pointing at my image, but not doing anything there. I saw that almost all of my GPU RAM was taken up in the "Dedicated GPU Memory Usage", and this I imagine is what gives me problems when trying to use Topaz Sharpen or Topaz Studio 2 from Photoshop. I shut down Lightroom, and saw that my GPU memory usage dropped pretty much to 0. I then opened Lightroom again, which opened with the same image loaded in the Development module, and my "Dedicated GPU Memory Usage" crept up slowly again but plateaued at a lesser amount, see attached image. I am curious why Task Manager is reporting 7.4/12.0 GB is used but the Utilization is only at 7%. Perhaps "Utilization" refers to the GPU activity and not memory? Anyway, what is upsetting here is the hold that Lightroom seems to have on the "Dedicated GPU Memory" even when I am not doing something that should be done in the GPU, meaning using Lightroom Denoise. This is why, I believe, I need to shut down Lightroom before I can use Topaz apps. Again, perhaps this memory allocation is specific to my mix of older motherboard/chipset and newer softwares and graphic card.

And yet another interesting observation. Based on the above, seeing that Lightroom is using up a lot of my GPU RAM and keeping a hold on it, I changed my Edit->Preferences Performance tab to not use the GPU. After restart of Lightroom I can see that my GPU memory utilization remains low. I also notice Lightroom taking a bit more time in getting my previews up on the screen. But then I did a denoise on an image just now, with Task Manager open on the Performance -> GPU monitor, and it did not look like Lightroom was really using GPU memory. However, the denoise task was done quickly, just as if I had set the Edit->Preferences Performance on "Auto". It seems that Lightroom will always use the GPU for doing the denoise work, no matter what the performance setting is set to. I am now wondering if I now no longer will need to shut Lightroom down when I want to use a Topaz app, since now Lightroom is not hogging the GPU RAM.

[D Fosse]

Well, again, high memory usage (system or GPU) is not a problem in itself. It should be high! That's efficient use of installed memory. 80% is perfectly normal.

I repeat: free memory is wasted memory.

It becomes a problem if the application or operating system doesn't re-allocate the memory to where it's needed. That's when you can talk of memory leaks. I'm afraid diagnosing that is above my paygrade. But possible causes include plugins/extensions, integrated GPUs, and generally buggy drivers.

What I can say is that on my two machines, Lightroom Classic does not behave like you describe. In other words - I cannot reproduce. Take that for what it's worth, but at least it's a data point in the equation.

[Callie253560770b53]

@D Fosse I just wanted to clarify because I was hastily typing my messages last night before I left dinner. You are correct and I agree. The system memory which is called "Real Memory" in Lightroom's 'system info' panel is showing that the program is reserving much more ram than the program is actively using.

The main issue as I see it is the program is calling for more vram that I have available: 12 gb+ in use vs. 8 gb on board. When the program does this the entire computer becomes unusable. The VRAM usage is usually around 5 gb at launch but once it climbs above 8 gb it will not come back down unless I restart the program. In that way it feels like a memory leak (vram) because I can't get it to drop by doing anything within the program.   

I think that this wouldn't necessarily be an issue with a igpu because you would have shared system memory and it could possibly manage. However, Lightroom leaves my GPU pinned at 8 gb and the computer becomes very laggy even when not locked down editing an AI mask. I think @steveisa054's suggestion to turn off hardware acceleration is my next option. However, I'm starting to wonder if this isn't somewhat of a windows problem. I believe that the 'Shared GPU Memory' is system memory that's made available to the GPU. So, I can image that Lightroom is seeing my GPU ram as 16 gb (or possibly 24gb = 8gb on card and 16gb shared?). I can imagine that the extreme stuttering that I'm seeing could easily caused by windows passing data between the vram and the ram.
(I included this image to show that the VRAM stays elevated even after switching off the photo that gives me issues, i.e. - it stays pinned, however it does drop down closer down to 10 gb with enough time).

[steveisa054]

What I noticed yesterday is that if I have Lightroom preferences configured to "Auto" for using the GPU for various tasks, then the dedicated GPU memory is being taken up to quite a large extent by Lightroom, as long as Lightroom is opened, even if I am not doing anything. To not have the GPU memory allocated by Lightroom, I need to configure Lightroom for the "Use Graphic Processer" to "off". I did this last night for the remaining of my editing session, and noticed that Lightroom was a bit slower with various tasks such as preparing previews when I navigate to other images, but the denoise AI feature of Lightroom was not slowed down by this at all. And for me this was a much better situation, because I no longer needed to shut down Lightroom if I wanted to use any of the Topaz apps. For the Topaz apps to be tolerably quick enough, I must configure them to use the GPU, but they will fail if Lightroom is holding on to so much of my GPU RAM. I don't know if this is a problem due to my older chipset, or if it is just a poor situation with Lightroom. It is great for Lightroom to use the GPU to make certain actions run quicker, but not if it is going to claim most of the graphic card RAM and hold onto it.