Lightroom LR4 - Silly RAM usage!!

[Guest]

Lightroom LR4 - Silly RAM usage!!

Edit Subject

Lightroom LR4 - Silly RAM usage!!

Hi there

I'm stunned at the amount of RAM usage by LIghtroom. I have 6Gb of RAM using LR4 in 64 bit win7.

This is what happened when I opened lightroom to edit a shoot of jpgs, all of which were only about 3mb each ( I wasn't even editing 30Mb RAW files from my DSLRs )

figures from task manager/performance pane:
**************************
No Lightroom

0.99 GB

Launch Lightroom and cat

1.53 GB

select next pic

2.00 Gb

select next pic (no editing yet )

2.20 GB

after editing two pics

3.13 GB

editing the 2nd pic in Nik Software Silver Efex pro 2 ( edit copy with lightroom adjustments ) and returning to lightroom

2.85 GB

but ....

Very soon, after only a few pics, the RAM usage is up to 5.5GB

*********************************************************************

This seems very disproportionate. Why for example would simply selecting another 3mb jpg eat up another 200Mb of RAM? And why after only a bit of editing on this 3mb pic in the develop module eat up another 900Mb of RAM?

The other highly annoying issue is that once Lightroom has decided to eat up nearly all my available RAM after editing only a few 3mb JPGs, things slow down rapidly. selecting another pic means everything locks up while their is vicious disk activity ( presumably swapping to swap file because Lightroom needs another 200Mb of ram for just one 3mb JPG )

it's not much fun having to quit LR every few images just to 'reset' thje amount of RAM.

[Lee Jay]

John Spacey wrote:

Why for example would simply selecting another 3mb jpg eat up another 200Mb of RAM?

The size of the compressed JPEG is irrelevant.  What matters is the pixel count.

LR should use 8 bytes per pixel just for the image you are processing.

[Brett N]

Is LR completing any processes (export, rendering, etc)? Progress bars will appear in the top right corner.

Is LR still building 1:1 thumbnails for all of your images? Just because you have just imported only 30 images doesn't mean that LR isn't doing other work with images imported previously. If you've setup LR to create 1:1 previews, if the process wasn't completed previously, then it will startup up again where it left off the last time you used it.

As a proof of concept, try creating a new catalog, don't import any images. See if you still see the same kinds of RAM usage. Then add a couple images, see how it performs.

[Guest]

Is LR completing any processes (export, rendering, etc)? Progress bars will appear in the top right corner.

Is LR still building 1:1 thumbnails for all of your images? Just because you have just imported only 30 images doesn't mean that LR isn't doing other work with images imported previously. If you've setup LR to create 1:1 previews, if the process wasn't completed previously, then it will startup up again where it left off the last time you used it.

Hi brett, no there aren't any outstanding processes. I always wait for 1:1 previews to be completed before attempting any editing. (  progress bars are at the top left btw. )

As a proof of concept, try creating a new catalog, don't import any images. See if you still see the same kinds of RAM usage. Then add a couple images, see how it performs.

yes, good call will try this.

[Guest]

Brett N wrote:

As a proof of concept, try creating a new catalog, don't import any images. See if you still see the same kinds of RAM usage. Then add a couple images, see how it performs.

Ok, done this. Created a new catalogue, imported just 5 images ( they are 10mp btw )

Same silly ram usage. up to 3.64Gb of usage after a a modicum of editing of those 5 images.     size of catalogue obviously makes no difference

[Guest]

Lee Jay wrote:

John Spacey wrote:

Why for example would simply selecting another 3mb jpg eat up another 200Mb of RAM?

The size of the compressed JPEG is irrelevant.  What matters is the pixel count.

LR should use 8 bytes per pixel just for the image you are processing.

Yes good point. 🙂

[LRuserXY]

Lee Jay wrote:

LR should use 8 bytes per pixel just for the image you are processing.

I would say 6 Bytes ... 16 bit processing (= 2 bytes) times 3 (RGB). But that does not really matter. The important point is that in either case it is a lot more than the original compressed file size, even for raws (that have a size of approx. 1...2 bytes per pixel). Furthermore, one has to take into account internal buffers when ACR passes image data through its pipeline, generates intermediate data for the ACR cache, database buffers,  etc., etc. (of course we do not know the internals here, but some buffers are certainly necessary).

[hamish niven]

I have a similar problem on my macbookpro. As soon as I start LR I see that my inactive memory starts rising.

Even if LR is not being used, ie at the moment, I'm watching  inactive slowly climb. At the moment its sitting at 2.26 Gb. Using PURGE in the terminal clears this inactive memory, but it slowly begins to climb again as soon as I start using LR again.

[borisporosin]

Well, LR4 does not manage memory properly. An example I faced recently: remove 5000 photos from catalog - the memory will raise from 200MB to 1600MB (Win7). Wait an hour or 24 (does not matter), then remove another 5000 photos from catalog - the memory will raise from 1600MB to almost 3GB of RAM... etc...

[bob frost]

Are you looking at memory specifically listed as being used by Lightroom, or are you looking at the total memory being used by Windows and all your programs? Windows will tend to fill all available memory with things that it thinks you might want in future (based on your past usage), but it frees it up as soon as a program wants it. I think Lightroom does some of the same, keeping things in memory in case you want to use 'undo'.

Bob Frost

[Guest]

Hi Bob.  In terms of RAM usage I'm looking at a real world situation of what happens when I launch lightroom in comparison to when it's not running. In regards to 'undo' remember that Lightroom is a database based application and uses non destructive editing. There aren't or shouldn't be copies of previous images in RAM in an undo history, as you might find in a regular graphics editing program like photopaint, etc  the images I was editing were 10mp, so lets say at a max thje RAM needed for the image would be 40Mb. Yet even after editing a couple of pics, LR has eaten up 1.5GBs of RAM.    but... even if for some strange and bizarre reason LR does Require 750Mb of RAM for ONE IMAGE, surely it should free up that RAM when you go onto loading the next image into the develop module.  there is clearly a problem here

[hhansard]

From my experiences, memory usage can vary from one configuration to another for the exact catalog/image combo -- HW and OS config do play a role here...  One guy can be crusing along with LR consuming 1.5G and another similar situation gobbles up 5.6G...  Now the 5.6G would really only be an issue if LR did not release the memory when not needed (garbage/heap management) --- this was a well documented issue with previous LR releases...  There was a fairly serious memory leak in LR3 (and I think in LR2) that took a while to sort out (only expressed itself in a particular set of circumstances).  My hardware configuration (SLI w/ dual 30” HD monitors) seems to expose boundary condition problems in the code base.

Haven’t played with LR4 yet – learned my lesson from previous LR releases – I now wait until the .1 release is gold prior to updating. 

[jsakhtar]

Windows 7 will on its own fill all available RAM with cached disk data. You can learn more here:  http://arstechnica.com/microsoft/news/2010/02/behind-the-windows-7-memory-usage-scaremongering.ars and http://www.tomshardware.com/reviews/windows-vista-superfetch-and-readyboostanalyzed,1532.html

What you really want to watch for is PAGING and . That is all that really matters. You would need to enable the column in the task manager called "Page Faults", and see where those are occuring.

Also, you should disable your virus scanner (like MS Security Essentials) for all your raw file types, the directories that contain your raw files, and the lightroom process itself.

You can get actual data from Lightroom itself Help|System Info, which will give you more useful information:

Lightroom version: 4.1 RC [820174]

Operating system: Windows 7 Home Premium Edition

Version: 6.1 [7601]

Application architecture: x64

System architecture: x64

Physical processor count: 8

Processor speed: 2.6 GHz

Built-in memory: 6135.1 MB

Real memory available to Lightroom: 6135.1 MB

Real memory used by Lightroom: 270.3 MB (4.4%)

Virtual memory used by Lightroom: 261.1 MB

Memory cache size: 138.1 MB

System DPI setting: 96 DPI

Desktop composition enabled: Yes

Displays: 1) 1920x1200, 2) 1920x1200

[Guest]

jsakhtar wrote:

Windows 7 will on its own fill all available RAM with cached disk data. You can learn more here:  http://arstechnica.com/microsoft/news/2010/02/behind-the-windows-7-mem ory-usage-scaremongering.ars and http://www.tomshardware.com/reviews/windows-vista-superfetch-and-ready boostanalyzed,1532.html

What you really want to watch for is PAGING and . That is all that really matters. You would need to enable the column in the task manager called "Page Faults", and see where those are occuring.

Also, you should disable your virus scanner (like MS Security Essentials) for all your raw file types, the directories that contain your raw files, and the lightroom process itself.

You can get actual data from Lightroom itself Help|System Info, which will give you more useful information:

Lightroom version: 4.1 RC [820174]

Operating system: Windows 7 Home Premium Edition

Version: 6.1 [7601]

Application architecture: x64

System architecture: x64

Physical processor count: 8

Processor speed: 2.6 GHz

Built-in memory: 6135.1 MB

Real memory available to Lightroom: 6135.1 MB

Real memory used by Lightroom: 270.3 MB (4.4%)

Virtual memory used by Lightroom: 261.1 MB

Memory cache size: 138.1 MB

System DPI setting: 96 DPI

Desktop composition enabled: Yes

Displays: 1) 1920x1200, 2) 1920x1200

Hi there jsakhtar

Indeed Lightroom Help|System Info is in fact  confirming what Task manager says about RAM usage. And yes becuase Lightroom is not releasing memory from previously edited images, once LRs RAM usage reaches close to the limit of physical RAM it certainly does go int opaging the swap file and things slow down to glacial speeds,  paging isn't all that really matters in htis case as the problem that means paging is reached when it shouldn't be is what matters the most.

regarding disabling virus scanner for RAW files, I'm not usign a virus scanner and indeed I'm reporting this usign jogs, not RAW files. 

As much as we can keep looking at our systems and blaming them, this is really a Lightroom problem and that's the where the fix should be applied. A program like lightroom shouldn't have to rely upon users being IT experts for it to run properly.

[bob frost]

I don't think you have said how many drives you are using? I hope you haven't got your system, pagefile, catalog, previews, acr cache, and images all on one drive!

LR does rely on your hardware/software working properly - we/you don't know that it is, do we/you?

Bob Frost

[Guest]

bob frost wrote:

I don't think you have said how many drives you are using? I hope you haven't got your system, pagefile, catalog, previews, acr cache, and images all on one drive!

LR does rely on your hardware/software working properly - we/you don't know that it is, do we/you?

Bob Frost

Hi there Bob.

OS ( system ) drive is SSD

I have 4 other HDDs, 2 x SATA2 and 2x SATA3

Catalogues on one SATA3 drive, images on another. ACR cache on another, although the AXCR cache isn't sued with JPGs IIRC.

could you explain how hard drive setup would affect RAM usage before swapfile is invoked? I can't see how Hard Drives would cause lightroom to not release RAM?

You ask "LR does rely on your hardware/software working properly - you don't know that it is, do you?"

Well although there may be some specific issue with my system that is causing Lightroom to eat up silly amounts of RAM and not release it, all other software I run on the machine does not have any poblems. Other Lightroom users have reported a similar issue above and Lightroom 3 was reknowned for memory leaks. 

I will cotninue to see if there is anything regarding mys system that can be done to lessen this ram usage issue though. I'll certainlytry the Open GL thing jsakhtarmentioned above.

Cheers

[bob frost]

From: "John Spacey

I will cotninue to see if there is anything regarding mys system that can

be done to lessen this ram usage issue though. I'll certainlytry the Open

GL thing http://forums.adobe.com/people/jsakhtar jsakhtarmentioned above.

How big is your pagefile?

Bob Frost

[Guest]

bob frost wrote:

From: "John Spacey

I will cotninue to see if there is anything regarding mys system that can

be done to lessen this ram usage issue though. I'll certainlytry the Open

GL thing  http://forums.adobe.com/people/jsakhtar jsakhtarmentioned above.

How big is your pagefile?

Bob Frost

Hi Bob. i'ts 6.2Gb - Could you explain how hard drive setup would affect RAM usage before swapfile is invoked? I can't see how Hard Drives would cause lightroom to not release RAM?

[bob frost]

From: "John Spacey

Hi Bob. i'ts 6.2Gb - Could you explain how hard drive setup would affect

RAM usage before swapfile is invoked? I can't see how Hard Drives would

cause lightroom to not release RAM?

Lightroom works fine on my Win7x64 with multiple drives, so I'm wondering

what is different about your computer that produces your problem.

I do have more RAM than you (16GB) but the system (including LR) only

usually uses about 2.5-3.5 GB RAM, while Windows uses various amounts for

cacheing - just over 4GB at present, making 6.5GB in total use (2,5GB 'in

use', and 4 GB 'on standby' as Windows reports in Resource Monitor), but I

have seen Windows fill nearly all the 'free' memory. You talked about 'disk

thrashing' when the pagefile was in use; an SSD can't thrash, can it? Mine

works totally silently!

Bob Frost

[Guest]

bob frost wrote:

From: "John Spacey

Hi Bob. i'ts 6.2Gb - Could you explain how hard drive setup would affect

RAM usage before swapfile is invoked? I can't see how Hard Drives would

cause lightroom to not release RAM?

Lightroom works fine on my Win7x64 with multiple drives, so I'm wondering

what is different about your computer that produces your problem.

I do have more RAM than you (16GB) but the system (including LR) only

usually uses about 2.5-3.5 GB RAM, while Windows uses various amounts for

cacheing - just over 4GB at present, making 6.5GB in total use (2,5GB 'in

use', and 4 GB 'on standby' as Windows reports in Resource Monitor),  but I

have seen Windows fill nearly all the 'free' memory. You talked about 'disk

thrashing' when the pagefile was in use; an SSD can't thrash, can it? Mine

works totally silently!

Bob Frost

Hi Bob,

so we haven't any explanation as to why a hard drive set up would affect Ram usage, before the swap file is invoked.

On 'disk thrashing', no I hvaen't talked about that. I mentioned vicious disk activity.  And when that starts everything in LR slows down. I can't even adjust the size of the adjustment brush for example

[coffeefrog]

Bob,

I can't see where "thrashing" got introduced into the thread or by who, but some background on the term might be useful.  It refers to lots and lots of disk activity, not to any noise.  Referring to memory, it usually means that the computer is writing virtual memory out to disk (the page file) and reading it back in over and over because it has run out of free RAM and the active part of the memory has gotten much bigger than the RAM can hold.  When this happens the machine can slow down to a complete crawl, screen redraws get very very slow (in the worst case it can be like a blind being pulled down slowly) and the machine appears to lock up because it is spending all of its time reading and writing memory to and from the disk.

Greg

[jsakhtar]

Hi John,

I work with RAW though one would think that would be more taxing, I don't

have memory issues. Have you disabled OpenGL?. You should really try that

if you haven't.

I have no memory leaks in 4 or 4.1. I have run VMMap for hours to be

certain.

4.0 was slow with multiple monitors but they seem to have fixed that in 4.1.

Sorry if I couldn't help.

[LRuserXY]

jsakhtar wrote:

(...)

Have you disabled OpenGL?

I thought LR does not use OpenGL like Photoshop. At least there seems to be no option to disable it. Or did you mean to disable OpenGL in the video card driver? And are you really sure that OpenGL is connected to RAM usage?

[jsakhtar]

I think in LR 4.1 they added some verbiage saying that it was only used for video. There is an option in the preferences panel to enable or disable it.

[LRuserXY]

Thanks for the pointer to that setting! I didn't really notice it before