Memory endless growth Mac M1

[Corrado282216293d9h]

I have a MacBook Pro 16gb ram and 500g disk. I often need to use it to convert a large amount of ARW sony raw pics to jpeg. I use script to do it in Photoshop in batches of thousands of pics.

My problem is that no matter what operting system (just updated to Ventura) and no matter what photoshop version (running 24.1.1) the system keeps on eating memory up to 80-90Gb than it restart itself.

I have had already a tech from Adobe connect to my pc and went trough the dance of uninstall reinstall a dozen times. System still does the same, memory keeps growing and growing untill it reset the pc.

Same job done on a 5 years old PC running windows works for days without a prob.

Since i found a lot of post similar to mine with ever growing memory in photoshop or in other app like lightbridge.

i have a question for tech dept.

[ Profanity removed by Moderator ]

[Lumigraphics]

Please do not use profanity in posting here. We will try to help you but at this point have no idea what your problem might be.

When you say script, what script? Are you using the Image Processor or??? Batch processing can also be done with Bridge or Lightroom, or a third-party tool such as ImageMagick.

Please post your System info from Photoshop's Help menu so we have a better idea what might be going on, and any crash reports that are generated.

[Ged_Traynor]

@Corrado282216293d9h this is an Adobe user to user forum and we all try to help each other out here, using profanity does not help in getting a response, please try to refrain from using inappropriate language on the forum.

[Corrado282216293d9h]

I am using the function script file--->script to convert batches of ARW pics to jpg.

It starts and memory just keeps growing and growing til 80Gb where the system crahes.

You can see it in the clip i attached, it just keeps growing and growing until crash. In the video it is already more than 10Gb memory used and has been working for around half an hour.

Macbook Pro 16gb ram 500gb disk

Photoshop 24.1.1

GPU disabled, if i enable it memory grows faster and faster.

https://share.icloud.com/photos/072X1SqCvAHkWSDmpw4Xmnodg

[Corrado282216293d9h]

Being an Adobe domain i tought some of their techs would show up t check probs.

This is a huge memory leak, i have had it since i got my M1, looking around it seems i am not alone, some people has same prob on photoshop, some on lightroom.

I already had an Adobe tech connect to my pc to check but he just reinstalled photoshop like i did a zilion times, obviously problem is still there and has been like this for a year now.

[Corrado282216293d9h]

This is what it looks like after a couple of hours of work.

https://share.icloud.com/photos/073GPQnRfhwJ8J_ESduDaQf1Q

[Federico_Calboli]

Are you sure it is not the script that fails to manage the memory correctly?

[D Fosse]

I have to say that a scratch file of 80 GB sounds perfectly ordinary to me - if that's what this is; it's not clear from your description or screenshots.

I routinely see scratch files of several hundred GB when working on large files. That's normal, and nothing wrong with Photoshop. The scratch file contains all history states for all open documents. It will be orders of magnitude bigger than your nominal starting file sizes. All this is normal and expected. Photoshop does not purge memory (RAM or scratch disk) until you close the application. But it will be reused and recycled while open.

If you really are low on disk space, set History States to 1. That means you can't go back, but it will dramatically reduce the scratch size.

For scripts and actions, you should write them so that one file closes before the next one opens whenever possible. If many files are open at the same time, scratch size will explode.

[Barry Rudick]

One strategy would be to reduce the number of history states from default to one or two before running a script.

[Lumigraphics]

You can also use the app.purge() method to purge the Photoshop cache every n images or when app.freeMemory is below a specified value. This would let you reduce memory footprint dramatically.

app.preferences.imageCacheLevels will let you set those, if you want to do so as part of your script.

[Corrado282216293d9h]

Maybe it is not clear, i am using the photoshop scripting functionality. It is under file--->scripts.

In there i set the source directory and destination directory, set jpeg quality to 8 and that is it, nothing more i can do. Should be photoshop taking care of managing memory so that pc doesn't reboot after a couple of hours of work.

I am just using a photoshop functionality that just doesn't work as it should.

[TheDigitalDog]

So Activity Monitor (which shows all processes and memory use) shows it is Photoshop eating all the memory? 
I ask because I saw a similar issue with memory; Activity Monitor ended up showing it was iCloud and syncing contacts eating up all the memory. Never saw this prior to Ventura. 

[Corrado282216293d9h]

Yes, you can see it in the pics i posted. It does it with Ventura and it did it maybe even worse with Monterey.

[D Fosse]

All I can see from your posted pictures is that Photoshop uses 12 GB in one and 48 GB in the other. All well within normal.

But earlier you said,

"GPU disabled, if i enable it memory grows faster and faster".

Maybe that's what the real issue is here. That would point to a bug in the GPU driver code (in this case MacOS code). What "unified memory" means is simply that the GPU uses system memory. That can often translate to half of the available memory under normal circumstances, and with only 16 GB installed, that's not a lot to go around. If Photoshop has already claimed more than the GPU needs, you could get crashing - not in Photoshop, but the GPU.

I'd monitor memory usage closely - don't look at the numbers, look at the graphs. That might give a clue.

[Corrado282216293d9h]

It sounds like it is my fault. 

I am not working on pics, just convertig them, really do not understand why it should use 50gb of memory. In the script i unchecked the flag "open pic" so it sholdn't even open it, just convert.

Today i really got annoyed by the continuous crashes. Uninstalled photoshop and did what i had to do with sips from command line. Guess what i opened 8 terminals all converting different directories, i was able to convert 15000 50Mb ARW pics in one afternoon, no crashes, no memory leaks, it just worked like photoshop should if it wasn't broken.

[Corrado282216293d9h]

Forgot to say, using photoshop, memory pressure gets in the red in about 1,5 hours,. 8 sips instances stayed in the reen the whole time.

[TheDigitalDog]

Converting pic's is working on them.

Photoshop has to have 3-5X each open document in memory. Or scratch disk. 

A product that is far more efficient in batch processing (ARW Sony raw to JPEG): Lightroom Classic. 

[Corrado282216293d9h]

since it converts them one a the time, the pic is 50mb, 3-4 of them are around 200mb, i wonder where all the rest of thosse 50gb are used.

I tried to use lightroom a month ago and it was doing exactly the same, memory out of control after few hours working.

[TheDigitalDog]

I tried to use lightroom a month ago and it was doing exactly the same, memory out of control after few hours working.

This also shouldn't be the case and points to some other issue on your Mac. 

Try logging into another user account (you may need to make one); still a problem? 
Try starting up in Safe mode (hold down Shift Key when booting), still a problem? 

[Corrado282216293d9h]

Will try, thank you.

Looking around the forum i am not the only one with this problem.

https://community.adobe.com/t5/lightroom-ecosystem-cloud-based-bugs/p-desktop-memory-leak-on-m1-mach...

[TheDigitalDog]

Will try, thank you.

Looking around the forum i am not the only one with this problem.

https://community.adobe.com/t5/lightroom-ecosystem-cloud-based-bugs/p-desktop-memory-leak-on-m1-mach...

By @Corrado282216293d9h

That URL concern LR/CC and compared to the user base, this statement is accurate:

This is not a commonly reported issue. 

[Lumigraphics]

I think you are just hitting the limits of what Photoshop can handle. Converting 15,000 images is not a normal use case. I'd probably use ImageMagick instead.

Same principle as why you use R or Maple or SPSS instead of Excel to work with terabyte-sized datasets, or why you take the minivan instead of the sports car to your kid's soccer games.

You might be able to bang on those default scripts with some memory management to get this working, dunno.

[Corrado282216293d9h]

On windows it handles it no problem, just 1 second slower per pic but never hangs. 

[TheDigitalDog]

On windows it handles it no problem, just 1 second slower per pic but never hangs. 

By @Corrado282216293d9h

Which again points to your hardware or a software conflict on your Mac. 

[kevin stohlmeyer]

Not saying it can't be the script - but Ventura and Monterey are both notorious for having memory leaks.

https://forums.macrumors.com/threads/ventura-memory-leak.2367442/

https://discussions.apple.com/thread/254593963?page=1