Known Participant

Question

Photoshop RAM/Memory drain during big batches of automated actions -- any 100% solution?

Forum|Forum|4 years ago
October 30, 2021
4 replies
4682 views

My situation is: My workflow involves really big batches of automated actions.

One set of such actions, for example, involves opening an image, resizing it as needed, then saving it to an external folder. Simple stuff, but done at a very large scale -- and slowly but sure, as these batches run, the RAM/Memory usage -- as seen in the Task Manager -- just climbs up and up, the further along I progress.

Another batch action that gets performed is batch smart-object replacement -- where basically, a smart object template in a series of mockup images gets replaced with images in a folder, then saved. This, too, causes the same problem: The further along it progresses, the more RAM/Memory gets used up.

It was to the point where the Memory usage would max out bigtime and just get as high as possible on my computer -- grinding basically everything to a halt on my computer and slowing everything down bigtime. Only solution at that point is closing out of Photoshop, then re-booting -- often requiring killing the task in the Task Manager.

The only mediocre, half-assed, semi-workaround I was able to cobble together was basically to take the Photoshop files used in that batch smart object replacement step, and just reduce the file size massively by merging a bunch of the layers. In addition, I found that changing the "History States" preference from the default of 50, to 2, also contributed to slowing this down. (But then if I forget to change this back -- which I probably will half the time -- then when I'm back in Photoshop working on stuff and I need to reverse the last group of changes I made -- I'm boned thanks to this setting change.)

This workaround is obviously not the ideal solution, because the same problem occurs over time -- it just prolongs the inevitable and slows down the rate at which the RAM/Memory usage creeps up. I still have to stop the process, X out of Photoshop, and then reboot it to deal with this problem -- especially for very big batches. And it's hard to predict when that will be needed because the different folders contain varying numbers of images -- so some batches will be manageable, others will be very large.

Purge All, variations of trying that, also doesn't really do anything meaningful to help with this problem.

Has anybody encountered this same problem in their workflow -- and found a solution that's 100%?

Maybe there's some kind of setting inside of Photoshop, or inside of my Computer settings, where I can say some variation of: "Cut this nonsense out"?

Having to reboot Photoshop a bunch of times is just a stupid way of dealing with this issue. If I could find a solid solution that completely eliminates this problem, I could just run my workflow all the way through and it would REALLY save me a lot of time and headaches here with this.

Thanks!

I should also add, this isn't just something that only occurs DURING the batches. It's not like, the RAM/Memory Usage climbs up during a big batch -- and then drops down to zero when it finishes. It just climbs up and up and doesn't drop back down until Photoshop gets closed out of.

So let's say one batch will get the Memory usage up to like 6,000MB for Photoshop. After the batch is completed and I'm just sitting there staring at the Photoshop homepage, it just stays at 6,000MB. Then if I run the next batch, it'll start increasing from that already high point, up to an even higher level of something like 8,000MB. Then the next batch starts climbing up from there. Etc until Photoshop is closed out of.

josephlavine

Community Expert

Are you receiving a specif error message or warning? If so, please share.

warmly/j

T

the_king_of_goatsAuthor

Known Participant

No specific error message, it's a performance issue that makes me unable to do anything until I reboot Photoshop. It just gets to the point where RAM usage climbs so insanely high that Photoshop functionally grinds to a halt, my ability to do anything else on my computer grinds to a halt, and any ability to continue moving forward in the workflow just completely stops until I kill the task and reboot Photoshop.

This basically means when I'm doing big batches, I have to literally reboot Photoshop after each batch. And when going through this fully workflow which runs dozens of batches, this can add an hour or more to the process, easily.

And not just reboot I have to flat out kill the task in the command prompt/Task Manager, because even after quitting Photoshop it holds onto that memory for a minute or two -- so if I reboot too fast without killing the task the Memory usage will remain at the previously high level.

I'll sometimes see people in the forums talking about how this is how Photoshop is supposed to work, how it's supposed to utilize as much RAM as possible -- but functionally grinding Photoshop and the computer it's working on to a halt, and making it so you literally have to kill the task in the Command Prompt to be able to move forward -- and needing to do this dozens of times over for a full workflow -- that is DEFINITELY not an ideal way that Photoshop should be working.

D Fosse

Community Expert

That's the term I've been hearing for this lately: "Memory leaks." I am not knowledgeable enough about Javascript or Adobe scripts to know whether the scripts I'm using contain such memory leaks, or if they could be improved upon in such a way that would reduce this issue however.

The main type of batch I'm running that this issue becomes most pronounced is on the Smart Object replacement batches. This uses the script that I posted above in one of my earlier replies.

After experimenting all day with this, if one thing makes the largest impact, it's definitely the History States setting inside of Photoshop. Leaving this at a high default level in the range of 25-30 causes this memory spike to happen extremely fast -- sometimes even a few small batches will cause it to max out and require a reboot (again, when I'm using that above script I posted.)

However dropping the History States down to 1 keeps the memory hovering fairly constant for each batch. It still climbs with each subsequent batch, but while running a specific batch, it seems to raise up a bit initially, then hit a ceiling, around which it will just hover -- even for larger batches containing 150+ images.

I've read online that another similar approach would be to suspend history states altogether for a given function. That may allow me to push this even further, however it's one approach that kind of freaks me out a little given the potential that I might write that script wrong and screw something up inside of Photoshop.

Re Photoshop RAM settings, it's set right to around 70 for what I'm running. Can't screenshot at the moment because Photoshop is running some batches right now.

I haven't yet found a perfect solution -- because my current approach still reboots + task-kills Photoshop every 4-5 batches or so just to be safe -- but I'm leaning on the side of caution in doing so that often because I'd rather the full program run all the way through than end up failing and causing headahes because the memory got maxed out at these steps.

It's also not just as simple as History States, because shrinking the file sizes being handled made an impact, and opening lots of Photoshop files at once to run the batches on -- vs just a small handful at a time, after which I reboot -- also plays a role here. So far those are the most important factors I've found that help me improve performance for these batches.

I've been around long enough to know that new Photoshop releases sometimes come with bugs - and historically, some of them have involved memory management. So I'm not ruling that out.

But first we need to eliminate all "natural causes". From your description, it sounds like this might be very scratch disk-intensive. Whenever you have smart objects, scratch disk usage tends to go up very fast. So how much free space do you have? Can you post a screenshot of the Preferences > Scratch Disks dialog?

I think we can eliminate RAM as a problem here. As I said, it will quickly hit the preference limit and stay there. That's not a problem. Photoshop doesn't "run out of RAM" - it goes to disk and dynamically reuses available RAM as a cache to speed things up.

If RAM usage goes beyond the limit, however, that's a problem. It shouldn't do that.

JJMack

Community Expert

How do batch process your mockup templates smart object using an Action. I just tested my Batch Mockup templates script and can not say I saw any problem where Photoshop was eating up all my machine resources. I ran a test where the script populated 84 mockup templates with 41 design replacements. So the script opened 84 PSD templates and replaced a smart object layer and saver a jpeg 41 time in each template. Photoshop's RAM Utilisation climbed to 4.8GB and Scratch space rose to 8.5GB. It took the script around 3 hours to save those 3,444 jpeg files . That is more or less the normal time different Photoshop version take on my slow 2GHz machine for this test. When Photoshop allocates resources like RAM and Scratch space. Photoshop manages it and reuses it. Photoshop does not return resources to your system till Photoshop is closed down or you have Photoshop purge some of the resources it has allocated using menu Edit>Purge>option

I'm running that test again this time I;m having the script edit the replacement design into the existing smart object. I watching the task manages and scratch space usage, Photoshop is using more resources RAM useage is currently 7GB and clinbing and scratch space is 14.5GB and steady. Ram is now 7.3GB and climbing. The test is onle 15% done and RAM useage is up to 7.8GB and climbing and Scratch is noe up to 15.5GB

JJMack

T

the_king_of_goatsAuthor

Known Participant

Interesting, maybe it has something to do with the way my Batch Smart-Object Replacement scripts are written?

Here is a sample of one such script. All of them follow this same basic template, with the only differences being the Actions run the the Input + Output Folders used here.

/**
 * Batch exports all documents from a given folder after executing an action
 * @9397041 {String} inputPath - Path of the source folder
 * @9397041 {String} outputPath - Path of the destination folder
 * @9397041 {String} actionName - Name of the action to execute
 * @9397041 {String} actionSet - Name of the set containing above action
 * @9397041 {RegExp} [wildcard=/\.(psd|tif|jpg)$/i] - Regular Expression pattern to filter files
 */
function runBatchAction(inputPath, outputPath, actionName, actionSet, wildcard) {
  wildcard = wildcard || /\.(psd|tif|jpg)$/i;
  var files = new Folder(inputPath).getFiles(function(file) { return wildcard.test(file.name) });
  //
  // https://theiviaxx.github.io/photoshop-docs/Photoshop/BatchOptions.html
  var options = new BatchOptions();
  options.destination = BatchDestinationType.FOLDER;
  options.destinationFolder = new Folder(outputPath);
  options.overrideOpen = false;
  options.overrideSave = true;
  //
  // https://theiviaxx.github.io/photoshop-docs/Photoshop/FileNamingType.html
  options.fileNaming = [FileNamingType.DOCUMENTNAMELOWER, FileNamingType.EXTENSIONLOWER];
  app.batch(files, actionName, actionSet, options);
}

// USAGE:
runBatchAction(
  "C:/Users/anton/Pictures/Next-Level Artwork/AUTOMATION ASSETS/PHOTOSHOP FILES/INTERMEDIATE IMAGES/18x24 - ORIGINALS", //Source folder
  "C:/Users/anton/Pictures/Next-Level Artwork/AUTOMATION ASSETS/PHOTOSHOP FILES/INTERMEDIATE IMAGES/18x24 - CANVAS PRODUCT", //Output folder
  "18x24 for canvas mockups", //Action name (case sensitive)
  "IMAGE RESIZING 2" //Set name (case sensitive)
);

J

jazz-y

Legend

What exactly is the problem? Are you getting the message "not enough RAM" or are you just worried that Photoshop is using a lot of RAM?

If the hardware of your computer is up to the task (processing large files), then this should not be a problem.

The simplest solution is to reduce the amount of RAM that Photoshop can use in the performance settings - in this case, the RAM load will be minimal, the main load will be on the paging file (make sure that it is on a fast disk, preferably SSD or NVMe). However, in this case, some of the information will still be stored in RAM, and when Photoshop reaches its limit, you will receive a message about its lack (here you need to find a balance between using the paging file and the amount of allocated RAM, which suits you).

For example, having allocated only 150 MB of memory to Photoshop, I was able to easily open 20 large documents with many layers. Yes, it took a little longer than usual and the swap file increased to 15 GB, but in the RAM, Photoshop takes no more than 1.5 GB

In addition to the number of history steps, you can also reduce the number of caching levels - in this case, Photoshop will take up less RAM than the browser in which I write this post:

If you are bothered by a decrease in performance when a certain level of RAM load is reached, then this is not a Photoshop problem, but a problem of the operating system's memory management - RAM, just like a hard drive, is inevitably fragmented during operation. The frames of the desired data are interleaved with the frames of the unnecessary ones. If all the memory allocated to the application has been used up, then the operating system memory management subsystem bypasses the entire allocated memory volume, marking the currently needed and freed frames, then redistributes them and writes new data (some of the necessary, but currently unused data is written to the system swap file). It all takes time. You can completely free up RAM and remove fragmentation only by restarting the application.

R

RLBoston

Known Participant

Mac doesn't have the fragmentation issue.

D Fosse

Community Expert

It sounds like you're running out of scratch disk space. How much do you have?

There is no such thing as "enough RAM" no matter how much you have. Photoshop moves so much data around that it has to write temporary data to disk, it can never rely on RAM alone. This is the scratch disk.

Think of RAM as a cache to the scratch disk's main memory.

RAM usage will go up to the limit you have set in preferences. You don't want to set this too high - other applications and processes need RAM to. This RAM will be reused, and Photoshop holds it until you quit the application. This is faster and more efficient than requesting it from the OS all again.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded