Poor RAM management in Photoshop CS4 - need help!

[Overfocused]

I have done numerous hours of research over the years only to give up and come back to this subject later on. I will try again!

I must ask: is there ANY way to get Photoshop CS4 to automatically flush used RAM after closing image files? I'm tired and annoyed of CS4 eating more and more RAM without freeing anything after an image has been closed. An image shouldn't still eat up memory when it isn't even open anymore. It seems to be terrible RAM management of the program itself and how Adobe has forgotten to write that little snippit of code to clear the RAM after an image has been closed.

I know this is not because of:

Cache levels

Setting too large of a saved history state; purging history cache or any kind of cache still never frees RAM

A lack of updates; I have been up to date with CS4 very punctually as they were released

The computer I am on; this poor RAM management has been around since CS2 and it's the same on the last 4 computers I have used

Does CS5 still do this too?

[Chris Cox]

I must ask: is there ANY way to get Photoshop CS4 to automatically flush used RAM after closing image files?

No, because it shouldn't. This is very good memory management - otherwise Photoshop would be significantly slower.  Unfortunately this is more advanced than the techniques typically taught in universities.

Photoshop is not leaking RAM, and reuses the RAM it has allocated.

Unused memory will be paged out, and if the OS really needs more RAM available Photoshop will free up some of it's RAM.

Photoshop's RAM usage should have little to no impact on other applications.

So, why do you think a non-problem needs to be "fixed"?

[Overfocused]

Oh great now I'm going to get a B.S. response like that as if I know nothing, and from Adobe. You're getting needlessly defensive since apparently you're answering questions I haven't even asked.

In response to your statements:

1. It shouldn't clear RAM its not using anymore? What? You clean your garage workbench after a project so you can work on the next project right? I hope. If you don't you'll never have the space for your next project. RAM runs at gigabytes per SECOND and couldn't possibly slow anything down by changing allocation. Photoshop is leaving the workbench full after closing images, and remains with next to no space left to work after its built up to %100 usage.

2. I said nothing about Photoshop leaking RAM. You're right. It doesn't leak.

3. Photoshop does NOT reuse the ram from the space it used after closing an image. If it did I wouldn't have any problem here. Once it hits %100 usage and starts grinding at my page file, I can close all the images open but Photostop still remains to use %100 of all available RAM. It also churns at a snails pace with any simple operation like cropping after that since it never frees or reuses any of the RAM until the whole program is closed and restarted.

3. I am not talking about outside application performance, I am talking only about Photoshop and the way it operates within itself. You're right, it doesn't affect outside performance since the OS is left with enough to perform outside of CS4. If it affected programs outside CS4 my system would have locked up or become painfully slow every time it hit %100

4. I think it needs to be fixed because its a problem. Its disrespectful talking to me talking to me as if I know nothing, and treating my request as if there isn't a problem to address. Most people can't expect performance from photoshop since they typically don't have a lot of RAM, but I do on a regular basis fill 7GB allocated to just CS4 with panoramas and mass raw processing. I should have plenty freed after closing a 3GB panorama so I can process my next image or batch of images, shouldn't I? Well, I don't. I don't always want to uload all my images at the same time, and I repeatedly run into the same roadblock. I know its not just my imagination, because I can recreate the problem at any time.

Anyone else have any input on this that runs into this problem as well?

[Chris Cox]

Sorry, but that was the honest truth, and correct.

Yes, allocations (and freeing) are still relatively slow on modern OSes.  We measure the impact every release to see if we need to change the memory allocation parameters (and sometimes we do).  But overall it is much, much faster to reuse memory than constantly free and reallocate.

Yes, Photoshop does reuse the space freed after closing an image (or layer, or channel, or history state, etc.).  After you close an image, the memory that was used for that image goes into a pool of available memory that gets used for the next image (or preset, channel, history state, whatever).  That is central to our memory management.

I'm not being disrespectful:  it is simply clear that I know more about the topic than you do, and have spent a great deal more time exploring the options and their impact than you have, so I am trying to inform you about the things you do not seem to know.  Is a professor disrespectful for teaching you a topic you don't understand?

Again, the memory is always reused -- there is nothing slowing you down, no roadblocks, no need to exit and restart, and no problem except in your understanding of the situation.

[Overfocused]

And the best professors still learn from their students from time to time...  this isn't my understanding of the situation, its whats happening.

If Photoshop reused the space, it wouldn't still be performing as slow as molasses after I close 4GB in images. There is a slow down and it stays that way, until I restart CS.

Could this happen:

If photoshop is always reserving the RAM and never reallocating it, once those reserve borders cross into pagefile country, its permanently crossed the border and slowed down. Its doing exactly what its supposed to do and it leaves the reserve boundaries within the page, which is limiting it to the hard disk. So, until you restart CS, you're going to be slow since those borders techncially have never been moved and your reserve is still linked to the page file.  An "Only as strong as its weakest link" situation.

[Chris Cox]

I don't know why your system is performing slowly - but Photoshop's memory management is not the issue.

Try opening and closing the same image several times and check the memory usage after each open -- if the memory was not reused, the application memory usage would go up a fixed amount each time. But it will stay the same (in my quick test it went down after the third time and stayed the same for the next 20 cycles, though it could go up a tiny amount due to OS allocations for drawing and event handling).  That shows that the memory allocated the first time the image was opened is being reused on each subsequent open.

Have you tried disabling third party plugins (to rule out leaks or other bugs in them)?

Have you tried logging on under another account and running Photoshop (to rule out system settings)?

Have you tried disabling OpenGL drawing (to rule out leaks in the GPU driver)?

[Overfocused]

I've got no third party plugins, disabled openGL and my system settings meaning pagefile? My pagefile is set to min/max 2.4/4.8GB. I know recommended is 1.5x your current system RAM, but the point of having 8GB is to avoid the pagefile alltogether. The page size is just to keep it from locking up, not to use it.

An example situation in where it happens is this:

I  open and work on lots of images or made some big panoramas and have  done a lot of things to them. Then, once the 7200MB I've allocated to PS  is full, it really starts to stutter cause its bumping into the  scratchdisk or whatever. So, I save and close 4GB of images. When I try  to work on the still open images, it continues to stutter even with very  non-intensive actions like exposure.

I can open the  same file as many times as I want, yeah, but if you open different files and close them, that reserve grows and does not shrink. I do see in task manager that if you open a new file, PS does reallocate some RAM for opening that file, but its still keeping that container at a maximum possible size and choking the system somewhere.

[Chris Cox]

but the point of having 8GB is to avoid the pagefile alltogether.

The point is to avoid hitting the pagefile very often.  The pagefile will still get used (which is why we laugh hard at people who try to disable the pagefile on Windows).

but if you open different files and close them, that reserve grows and does not shrink.

Nope, it's the same as opening one file many times:  the memory gets reused.

Ok, Photoshop should not be stuttering like that -- something on your system is causing a conflict with Photoshop.

Did you copy any plugins from a previous version of Photoshop?

What else on your system might be interfering?  Corrupt fonts?  Bad anti-virus software?  Bad disk utilities? Bad external disk?

[Overfocused]

@Noel - That's what I'm talking about. I edit on large images, or a ton of RAW images at one time for batch procesing, and then it starts reaching some slump point where its not quite as it should be and it gets iffy.

@Chris

The stuttering only happens when I reach the maximum RAM photoshop is permitted to use, and then trying to close images to free up RAM to continue working on more images. The fluidity of working on anything just bogs down and seems to stay in a slump till I restart PS.

If I was aware that something was bad on my system, I'd fix it ASAP. My AV software is Avast, not sure how I'd even know if a font was corrupt I've never had any issues with fonts, no disk utilities run in the background, and the external hard drive I use is only for system image backups not scratch space. It's off %95 of the time I'm on the computer.

Opening one file over and over vs. many different files over and over work differently when I tested it. I can record my screen with task manager open to show you how the OS sees the RAM add up on my machine

[Chris Cox]

Opening one file over and over vs. many different files over and over work differently when I tested it

Doesn't work at all differently -- Photoshop has no memory of the previous document, it just reuses the memory made available by closing a previous document.

[Overfocused]

What i meant was opening 1 file over and over works much differently than opening many different files to work on them*

I'll just make a video (it wont be very long dont worry)

[Overfocused]

I can't seem to quickly force it to stutter as it usually does the in the way that caused me to finally start this thread in the first place. It happens over a period of time editing, closing, and opening new documents to edit. I opened enough files till CS started using the hard drive, and then closed all the files. CS is shown in task manager as taking 6.7GB of RAM with all the files closed. The program is running slightly slower than it typically does, but the reason I started this post is because many times I have had it down to a stuttering and painfully slow workflow, with CS obviously using just the hard drive as the HDD light churns and churns away for basic tasks and filters inside of CS. There is a correlation somewhere between CS using its maximum alotted RAM and starting to churn the hard drive exclusively even after closing image files on my machine.

Argh. If it only happened rarely I'd suck it up but I remember many times having to reboot CS in order to be able to work on new images.

[Chris Cox]

Again, if you're having to quit and restart Photoshop - something is wrong outside of Photoshop (or you failed to pick up a dot release where we fixed a nasty bug).

Stuttering and painfully slow -- that's usually a symptom of something like a bad third party plugin, a bad driver (usually OpenGL), or a bug in some system utility (like AV).  Once in a while we've seen corrupt font files cause memory leaks and odd performance issues, but we usually catch those issues and fix or work around them.

Yes, Photoshop will use the scratch disk once it runs out of RAM, but unless there's a problem with the disk or you are working on documents much larger than RAM - that should go pretty quick.  If you close all image files, Photoshop will page in a little bit from the scratch and stop.  It should only be really active when using image data that won't fit in RAM.

Have you tried the obvious and reset or deleted the Photoshop preferences?

[Overfocused]

I've deleted preference files before yes. It's happened for as long as I remember using CS4...

Either way, I figure I'll try to set the allocated RAM to a bit less than what it is now. I'm wondering if maybe PS is triggering my OS to start using the page file. If it doesn't ever stutter anymore I may have allocated too much to PS. Technically I haven't, I've left 800MB to the operating system, but it may still be having trouble anyway.

Thanks for listening, for now I guess I'll just do my normal thing and see how it works

[Overfocused]

OK! I have encountered the severe stuttering again, and I know why since I was paying close attention this time. It goes like this:

- I make a panorama. I work on it, do a bunch of stuff to it, save it and close it.

- The PS ram reserve stays the same size that the image + history states combined equaled even after closing.

- I start and finish about 4 more panormas, and now the CS is reserving the full 7.2GB I have alotted it, and the physical memory to my OS is %98 full.

- With such a small gap left at %98, the OS is using the pagefile.

So, this means PS is holding hostage all the free RAM from the OS, for all the previously closed documents I have worked on even though I've saved and closed them.  For those that need to wring every megabyte safely out of their RAM,  this type of workflow presents this bottleneck. I just happen to be one of those guys that  edits many massive images in a row, and has happened to find it since I try to give PS the maximum RAM possible without cutting my OS's life support. So far camera images have become larger and larger in filesize proportionately quicker than the maximum RAM in consumer PCs, and possibly some workstations as well. This bottleneck may just pop up more in the future if there are more RAM hogs such as myself that want to stay away from paging and scratching as long as possible. (Nikon D3 users come to mind)

I believe if PS had the option to cut the reserved RAM its hiding from the OS down to what it was actually using after an image was closed, this wouldn't happen unless a user actually had %100 of their physical RAM full. Including history, I'm currently using maybe 3GB for the current

panorama that is open, and while PS can see the 4.2 GB as a free reserve, my OS any other OS cannot. It is an OS problem, but it is being caused by the way CS manages (or doesn't manage) its maximum RAM reserve.

For now, I will cut the RAM allocated to PS by about %10 and see how it goes. It will probably smooth all but the panoramas I sometimes generate that take ~7gb of data to create and bump into filling the physical memory completely in Windows. Time to restart PS for me. Lol.

[Chris Cox]

So, this means PS is holding hostage all the free RAM from the OS,

No, because much of Photoshop's RAM will have already been paged out, and when the OS starts paging heavily we free up more RAM.

I just happen to be one of those guys that  edits many massive images in a row, 

That describes a lot of users, most of whom don't have your slowdown problems.

I believe if PS had the option to cut the reserved RAM its hiding from the OS down to what it was actually using after an image was closed,

And each operation after that would be much slower, and address space fragmentation would be a much larger problem.

I try to give PS the maximum RAM possible without cutting my OS's life support.

Yes, we give you the option to adjust the RAM limits even to ranges where you probably shouldn't -- assuming that if you change the limit you probably have some idea what you are doing (and why we give the "ideal" range in the prefs dialog).

It sounds like you've set the Photoshop RAM limit too high and need to dial it back a bit.

[Overfocused]

No, because much of Photoshop's RAM will have already been paged out, and when the OS starts paging heavily we free up more RAM.

- Well thats obviously not the case because its coughing and hacking like a 90 year old man with asthma, its bottlenecking with the OS and page file even if PS is paging more RAM, its obviously not able to do it any more quickly in my situation. Funny enough, right after I posted my last post, my machine locked and crashed while trying to save the panorama to close it. I was watching in real time my OS stuttering to manage page + the 7.2GB taken by PS + save a 700MB layered file. To the OS, none is freed it is only full and hence it sees the need to page.

That describes a lot of users, most of whom don't have your slowdown problems.

- Most users don't want to reserve 7GB of RAM for operations... I do

And each operation after that would be much slower, and address space fragmentation would be a much larger problem.

- This is RAM. Fragmentation means nothing. That's why its Random access memory. Nanoseconds, not milliseconds. The idea of defragmenting RAM has never been capitalized on because that is a non issue. Even state of the art processors can't come close to saturating the bandwidth of memory, and if they do, it is only in synthetic benchmarks.

It sounds like you've set the Photoshop RAM limit too high and need to dial it back a bit

- Yes, it does. In this situation I'll just dial back the usage about 700MB.

[Guest]

Overfocused wrote:

- Most users don't want to reserve 7GB of RAM for operations... I do

A previous post on how much RAM can one allocate to PS came up with a general guideline that 1.5 to 2 gigs for the OS the rest for PS seems to work well.  This is not a scientific study, just users experience.  You may be a little light with only 0.8 gigs.

If you need more RAM for for largre images, for PS buy another 8 gigs of RAM.

[Overfocused]

8 is the maximum supported for this laptop, otherwise I would have considered 12 or 16 when I had bought 8.

I toned it down to 6.5GB reserved for PS a bit earlier. If I'm going to do a giant panorama that needs 7GB, I'll just set it to 7.2 for that specific pan generation, and pull it backafterward. Even just 700MB of paging slows it down considerably compared to the speed it achieves when generating a panorama %100 in the RAM.

When I had only 4GB of RAM, it took 50 minutes to generate a 21 image vertical photo panorama (minimal distortion when pans are taken vertically - much better and more accurate pictures) When I had 8GB, it took less than 3 minutes.

This is why I want every MB I can scrape out of my machine. Anyone who thinks paging is a good thing for non-emergency operations can scratchdisk my arse haha (joke)

[Overfocused]

Well, I basically have found my answer here, although not an automatic solution that would allow me to use that extra 700MB permanently. I should have named the title 'CS4 Memory Bottleneck issue,' not poor management.

I'll nail the perfect balance so I can wring out my RAM for every KB that it's worth

[Chris Cox]

, my machine locked and crashed while trying to save the panorama to close it.

Photoshop locked up and crashed?

Or the machine hung or rebooted?

(because those would have different causes)

Fragmentation means nothing.

LOL!  No, address space fragmentation means a lot.  It's less of a problem right now with 64 bit addressing, but a huge problem in 32 bit addressing. (and will be a problem in 64 bit when people get more RAM into their machines)

[Overfocused]

PS hung then my computer hung... my music kept playing, then the machine hung athlough it kept playing my music just fine until I rebooted. PS was maxing out the resources, then when I tried to save it was PS trying to get more and it crashed.

Every other program I've ever used in my life actively manages RAM, whether its games, image editing, video editing, whatever. I don't get why this screwball concept is even being argued about when in fact it does not reduce the amount of ram it is using to the rest of the system after you close an image.

There's no excuse for a crappy 'advanced memory algorhithms not taught in universities!" when every other resource intense program I use can manage its own RAM just fine without bloating the whole machine, even if it is reserved it still only uses the memory that it NEEDS not holds a massive chunk hostage on the side. Other capable programs would use and manage all 7.2 GB if I allocated it to them, and would unload resources it was no longer even using. PS doesn't do that.

[Chris Cox]

Photoshop does actively manage the RAM, and does reduce usage when the OS needs more RAM available.

Just because you don't understand memory management does not mean that it is wrong.

Photoshop is doing what it needs to do to process images quickly.

But the system lockup/hang sounds like you may have other problems than one application using a lot of RAM.

Normally the system would continue working but slow down as it paged more often.

But what you're describing sounds more like a kernel level leak or a bad hard disk.

[Overfocused]

Well CS obviously isn't releasing any RAM here because when I monitor it in task manager, it never goes back down. I can't say it any simpler.

I may as well run it in safe mode, won't be much difference really since my system is minimal on any active system maintenance or management software.

Imma give'r a go in safe mode!

[harinder5E8E]

12 years and this probles still exists.