• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Photoshop RAM/Memory drain during big batches of automated actions -- any 100% solution?

Participant ,
Oct 29, 2021 Oct 29, 2021

Copy link to clipboard

Copied

My situation is: My workflow involves really big batches of automated actions.

 

One set of such actions, for example, involves opening an image, resizing it as needed, then saving it to an external folder. Simple stuff, but done at a very large scale -- and slowly but sure, as these batches run, the RAM/Memory usage -- as seen in the Task Manager -- just climbs up and up, the further along I progress.

Another batch action that gets performed is batch smart-object replacement -- where basically, a smart object template in a series of mockup images gets replaced with images in a folder, then saved. This, too, causes the same problem: The further along it progresses, the more RAM/Memory gets used up.

 

It was to the point where the Memory usage would max out bigtime and just get as high as possible on my computer -- grinding basically everything to a halt on my computer and slowing everything down bigtime. Only solution at that point is closing out of Photoshop, then re-booting -- often requiring killing the task in the Task Manager.

 

The only mediocre, half-assed, semi-workaround I was able to cobble together was basically to take the Photoshop files used in that batch smart object replacement step, and just reduce the file size massively by merging a bunch of the layers. In addition, I found that changing the "History States" preference from the default of 50, to 2, also contributed to slowing this down. (But then if I forget to change this back -- which I probably will half the time -- then when I'm back in Photoshop working on stuff and I need to reverse the last group of changes I made -- I'm boned thanks to this setting change.)

 

This workaround is obviously not the ideal solution, because the same problem occurs over time -- it just prolongs the inevitable and slows down the rate at which the RAM/Memory usage creeps up. I still have to stop the process, X out of Photoshop, and then reboot it to deal with this problem -- especially for very big batches. And it's hard to predict when that will be needed because the different folders contain varying numbers of images -- so some batches will be manageable, others will be very large.

 

Purge All, variations of trying that, also doesn't really do anything meaningful to help with this problem.

Has anybody encountered this same problem in their workflow -- and found a solution that's 100%?

Maybe there's some kind of setting inside of Photoshop, or inside of my Computer settings, where I can say some variation of: "Cut this nonsense out"?

 

Having to reboot Photoshop a bunch of times is just a stupid way of dealing with this issue. If I could find a solid solution that completely eliminates this problem, I could just run my workflow all the way through and it would REALLY save me a lot of time and headaches here with this.

 

Thanks!

 

 

I should also add, this isn't just something that only occurs DURING the batches. It's not like, the RAM/Memory Usage climbs up during a big batch -- and then drops down to zero when it finishes. It just climbs up and up and doesn't drop back down until Photoshop gets closed out of.

 

So let's say one batch will get the Memory usage up to like 6,000MB for Photoshop. After the batch is completed and I'm just sitting there staring at the Photoshop homepage, it just stays at 6,000MB. Then if I run the next batch, it'll start increasing from that already high point, up to an even higher level of something like 8,000MB. Then the next batch starts climbing up from there. Etc until Photoshop is closed out of.

TOPICS
Actions and scripting , Windows

Views

1.9K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe
Community Expert ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

It sounds like you're running out of scratch disk space. How much do you have?

 

There is no such thing as "enough RAM" no matter how much you have. Photoshop moves so much data around that it has to write temporary data to disk, it can never rely on RAM alone. This is the scratch disk.

 

Think of RAM as a cache to the scratch disk's main memory.

 

RAM usage will go up to the limit you have set in preferences. You don't want to set this too high - other applications and processes need RAM to. This RAM will be reused, and Photoshop holds it until you quit the application. This is faster and more efficient than requesting it from the OS all again.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

What exactly is the problem? Are you getting the message "not enough RAM" or are you just worried that Photoshop is using a lot of RAM?

If the hardware of your computer is up to the task (processing large files), then this should not be a problem.

The simplest solution is to reduce the amount of RAM that Photoshop can use in the performance settings - in this case, the RAM load will be minimal, the main load will be on the paging file (make sure that it is on a fast disk, preferably SSD or NVMe). However, in this case, some of the information will still be stored in RAM, and when Photoshop reaches its limit, you will receive a message about its lack (here you need to find a balance between using the paging file and the amount of allocated RAM, which suits you).

 

For example, having allocated only 150 MB of memory to Photoshop, I was able to easily open 20 large documents with many layers. Yes, it took a little longer than usual and the swap file increased to 15 GB, but in the RAM, Photoshop takes no more than 1.5 GB

2021-10-30_12-37-24.png

2021-10-30_12-38-53.png

In addition to the number of history steps, you can also reduce the number of caching levels - in this case, Photoshop will take up less RAM than the browser in which I write this post:

 

2021-10-30_13-55-57.png

 

If you are bothered by a decrease in performance when a certain level of RAM load is reached, then this is not a Photoshop problem, but a problem of the operating system's memory management - RAM, just like a hard drive, is inevitably fragmented during operation. The frames of the desired data are interleaved with the frames of the unnecessary ones. If all the memory allocated to the application has been used up, then the operating system memory management subsystem bypasses the entire allocated memory volume, marking the currently needed and freed frames, then redistributes them and writes new data (some of the necessary, but currently unused data is written to the system swap file). It all takes time. You can completely free up RAM and remove fragmentation only by restarting the application.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

How do batch process your mockup templates smart object using an Action.  I just tested my Batch Mockup templates script and can not say I saw any problem where Photoshop was eating up all my machine resources.  I ran a test where the script populated 84 mockup templates with 41 design replacements.  So the script opened 84 PSD templates and replaced a smart object layer and saver a jpeg 41 time in each template.   Photoshop's RAM Utilisation climbed to 4.8GB and Scratch space rose to 8.5GB.  It took the script around 3 hours to save those 3,444 jpeg files . That is more or less the normal time different Photoshop version take on my slow 2GHz machine for this test. When Photoshop allocates resources like RAM and Scratch space.  Photoshop manages it and reuses it. Photoshop does not return resources to your system till Photoshop is closed down or you have Photoshop purge some of the resources it has allocated using menu  Edit>Purge>option

 

I'm running that test again this time I;m having the script edit the replacement design into the existing smart object. I watching the task manages and scratch space usage,  Photoshop is using more resources RAM useage is currently 7GB and clinbing and scratch space is 14.5GB and steady.  Ram is now 7.3GB and  climbing.   The test is onle 15% done and RAM useage is up to 7.8GB and climbing and Scratch is noe up to 15.5GB 

JJMack

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

Interesting, maybe it has something to do with the way my Batch Smart-Object Replacement scripts are written?

 

Here is a sample of one such script. All of them follow this same basic template, with the only differences being the Actions run the the Input + Output Folders used here.

 

/**
 * Batch exports all documents from a given folder after executing an action
 * @Param {String} inputPath - Path of the source folder
 * @Param {String} outputPath - Path of the destination folder
 * @Param {String} actionName - Name of the action to execute
 * @Param {String} actionSet - Name of the set containing above action
 * @Param {RegExp} [wildcard=/\.(psd|tif|jpg)$/i] - Regular Expression pattern to filter files
 */
function runBatchAction(inputPath, outputPath, actionName, actionSet, wildcard) {
  wildcard = wildcard || /\.(psd|tif|jpg)$/i;
  var files = new Folder(inputPath).getFiles(function(file) { return wildcard.test(file.name) });
  //
  // https://theiviaxx.github.io/photoshop-docs/Photoshop/BatchOptions.html
  var options = new BatchOptions();
  options.destination = BatchDestinationType.FOLDER;
  options.destinationFolder = new Folder(outputPath);
  options.overrideOpen = false;
  options.overrideSave = true;
  //
  // https://theiviaxx.github.io/photoshop-docs/Photoshop/FileNamingType.html
  options.fileNaming = [FileNamingType.DOCUMENTNAMELOWER, FileNamingType.EXTENSIONLOWER];
  app.batch(files, actionName, actionSet, options);
}

// USAGE:
runBatchAction(
  "C:/Users/anton/Pictures/Next-Level Artwork/AUTOMATION ASSETS/PHOTOSHOP FILES/INTERMEDIATE IMAGES/18x24 - ORIGINALS", //Source folder
  "C:/Users/anton/Pictures/Next-Level Artwork/AUTOMATION ASSETS/PHOTOSHOP FILES/INTERMEDIATE IMAGES/18x24 - CANVAS PRODUCT", //Output folder
  "18x24 for canvas mockups", //Action name (case sensitive)
  "IMAGE RESIZING 2" //Set name (case sensitive)
);

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

There seem to be some problem the test has been running all day Photoshop management  of machine resources seem to have been changed I saw Ram utilization climb to 8GB but then it dropped to 4GB however  Photoshop started a second  temp file for scratch space the first is 16GB the second has climbed to 8GB.  It is many hours later the RAM itemization is 19GB there are now 3 16GB scratch space file and a fourth the is 8GB.  Photoshop seem the be looping updated a smart object in the 77th Psd mockup. Photoshop has now been running the script for 12 hours. Time to pull the plug.image.png

Wait capturing the screen did something to Photoshop operation. The script is processing template again,  RAM utilizing has dropped to 3.1GB. Will see if all 3,444 jpeg get saved.  The 81st mockuo is being processed now three more to go. RAM is now 4GB and the fourth Scratch file is now 11GB.

JJMack

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

If has now finished took almost 12 hours where it should have only taken around 3 hours. 

image.png

Photoshop memory initialization stands at 5GB

image.png

Scratch space 70GB

image.png

 

JJMack

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 31, 2021 Oct 31, 2021

Copy link to clipboard

Copied

Thanks for the update on this! What do you think the implications are of these findings for running such scripts -- in terms of potential areas where performance improvements could be made, etc?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 31, 2021 Oct 31, 2021

Copy link to clipboard

Copied

I think there are some bug in 23 that need to be addressed my Adobe.  I have no idea why Photoshop was hung so long updating that one smart object. The objects in the test are updated 40 times in the 84 templates. So the replacement object hung  in one of the 84 templates in 83 templates there was no problem.  Photoshop 2022 version 23 performance is much different then photoshop 2020 version 21. Photoshop 2022 had done some strange things on on my Windows 10 PC.

JJMack

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 31, 2021 Oct 31, 2021

Copy link to clipboard

Copied

LATEST

Yes, some processes are almost twice slower, so I downgraded to 22.5.2.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

Are you receiving a specif error message or warning? If so, please share.

warmly/j

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

No specific error message, it's a performance issue that makes me unable to do anything until I reboot Photoshop. It just gets to the point where RAM usage climbs so insanely high that Photoshop functionally grinds to a halt, my ability to do anything else on my computer grinds to a halt, and any ability to continue moving forward in the workflow just completely stops until I kill the task and reboot Photoshop.

 

This basically means when I'm doing big batches, I have to literally reboot Photoshop after each batch. And when going through this fully workflow which runs dozens of batches, this can add an hour or more to the process, easily.

 

 

And not just reboot I have to flat out kill the task in the command prompt/Task Manager, because even after quitting Photoshop it holds onto that memory for a minute or two -- so if I reboot too fast without killing the task the Memory usage will remain at the previously high level.

 

I'll sometimes see people in the forums talking about how this is how Photoshop is supposed to work, how it's supposed to utilize as much RAM as possible -- but functionally grinding Photoshop and the computer it's working on to a halt, and making it so you literally have to kill the task in the Command Prompt to be able to move forward -- and needing to do this dozens of times over for a full workflow -- that is DEFINITELY not an ideal way that Photoshop should be working.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

What is the RAM allocation set to in PS preferences? Do you see Photoshop RAM usage go beyond that limitation? Please show screenshots.

 

If Photoshop halts, it's not because of RAM usage. As long as there's enough scratch disk space, Photoshop will keep right on working. In fact, RAM isn't all that important. It's just a cache to the scratch disk, it just helps things go a little faster. But it'll work anyway.

 

Photoshop is not supposed to release RAM until you quit the application. People think that's a defect, but it's by design. It's simply much faster and more efficient than requesting it from the OS.

 

But this is also why you don't want to set RAM allocation too high. You don't want to choke the rest of the system. 70% is usually pretty optimal.

 

All that said, memory leaks do happen. Then you'll see memory usage steadily rising until it maxes out. But every single case I've seen (and there aren't many), it's caused by third party plugins. I've never heard of Photoshop itself doing that.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 30, 2021 Oct 30, 2021

Copy link to clipboard

Copied

That's the term I've been hearing for this lately: "Memory leaks." I am not knowledgeable enough about Javascript or Adobe scripts to know whether the scripts I'm using contain such memory leaks, or if they could be improved upon in such a way that would reduce this issue however.

 

The main type of batch I'm running that this issue becomes most pronounced is on the Smart Object replacement batches. This uses the script that I posted above in one of my earlier replies.

 

After experimenting all day with this, if one thing makes the largest impact, it's definitely the History States setting inside of Photoshop. Leaving this at a high default level in the range of 25-30 causes this memory spike to happen extremely fast -- sometimes even a few small batches will cause it to max out and require a reboot (again, when I'm using that above script I posted.)

 

However dropping the History States down to 1 keeps the memory hovering fairly constant for each batch. It still climbs with each subsequent batch, but while running a specific batch, it seems to raise up a bit initially, then hit a ceiling, around which it will just hover -- even for larger batches containing 150+ images.

 

I've read online that another similar approach would be to suspend history states altogether for a given function. That may allow me to push this even further, however it's one approach that kind of freaks me out a little given the potential that I might write that script wrong and screw something up inside of Photoshop.

 

Re Photoshop RAM settings, it's set right to around 70 for what I'm running. Can't screenshot at the moment because Photoshop is running some batches right now.

 

I haven't yet found a perfect solution -- because my current approach still reboots + task-kills Photoshop every 4-5 batches or so just to be safe -- but I'm leaning on the side of caution in doing so that often because I'd rather the full program run all the way through than end up failing and causing headahes because the memory got maxed out at these steps.

 

It's also not just as simple as History States, because shrinking the file sizes being handled made an impact, and opening lots of Photoshop files at once to run the batches on -- vs just a small handful at a time, after which I reboot -- also plays a role here. So far those are the most important factors I've found that help me improve performance for these batches.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 31, 2021 Oct 31, 2021

Copy link to clipboard

Copied

I've been around long enough to know that new Photoshop releases sometimes come with bugs - and historically, some of them have involved memory management. So I'm not ruling that out.

 

But first we need to eliminate all "natural causes". From your description, it sounds like this might be very scratch disk-intensive. Whenever you have smart objects, scratch disk usage tends to go up very fast. So how much free space do you have? Can you post a screenshot of the Preferences > Scratch Disks dialog?

 

I think we can eliminate RAM as a problem here. As I said, it will quickly hit the preference limit and stay there. That's not a problem. Photoshop doesn't "run out of RAM" - it goes to disk and dynamically reuses available RAM as a cache to speed things up.

 

If RAM usage goes beyond the limit, however, that's a problem. It shouldn't do that.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines