Photoshop slow performance with 16bit files at 100MP, with mbp 16" too low end?

[pets]

hi

i am not sure, if latest photoshop has optimized code for heavy I/O loads?

i am mostly working on 100MP .psb files at 16bit with about 30-50 layers.

i realized, that the performance is mostly laggy and i cannot work in realtime.

for example, turn on / off one color-adjustment layer needs 3-5 sec to show result "before/after".

i am asking myself, if my hardware is too slow for these i/o loads (need a mac pro?) or if photoshop is not

optimized for 16bit files at 100MP with many layers.

my hardware: 

latest macbookpro 16" with 64GB RAM and 2TB SSD, graphicscard has 8GB RAM.

i use an external thunderbolt 3 ssd as dedicated /empty scratch disc (1TB)

(running on big sur 11.5)

i work on 2 external screens, 1 x 32" Benq sw321c (4k), and a second NEC 27" at 2560px.

enclosed a screenshot of my monitored stats: (istatmenu)

thanks!

[D Fosse]

Photoshop will handle this with no issues, but your hardware may not.

First of all, this requires a very large amount of free disk space for the Photoshop scratch disk. I suspect that's the main bottleneck. An external drive is never ideal, whatever the interface. Try to move the scratch disk to the internal drive. You need at least 500 GB free space for this, preferably more.

If that doesn't help, I would suspect the video driver - which, unfortunately, is integrated into the operating system on Mac. So there isn't much you can do about that. On Windows you could have updated the driver separately.

[pets]

i am questioning this, as i have 64 GB RAM and i am monitoring constantly, when i work with 100MP files in 16bit, RAM usage is at about 50GB "only". i am also monitoring my scratch disc, there is very few swap activity.

i am using a thunderbolt ssd external, as adobe said, the internal ssd, if its the only one, should not be set as scratch disc, if there is OS and work activity, too.

[D Fosse]

The scratch disk is always used, however much RAM you have. There is no such thing as "enough RAM" to Photoshop. Think of RAM as a fast access cache to the scratch disk's main memory.

Working with the file sizes you do, I can promise you that the scratch file will quickly grow to many hundred GB.

Photoshop runs its own memory management and I'm not sure the operating system will record the disk swapping, beyond the disk space that Photoshop allocates and requests from the OS.

Anyway, just try it.

[Test Screen Name]

This puzzles me. "i am also monitoring my scratch disc, there is very few swap activity." There is a huge amount of disk activity to an active scratch disk. But no SWAP activity, that's a system function only.

[Patryk Rozanski]

Kind of funny reading about low end hardware causing the problem having in mind working on 3-4 larger files with photoshop cs5 smoothly with 2 gigs of ram and today getting a garbage performance with smaller files and 10 times more powerfull working station. 

[rob day]

Just clarify, by 100MP you are referring to the .psb file’s pixel dimensions—10,000 x 10,000 px—and you don’t mean the file size is 100MB right? A 16-bit file that size with 30-50 layers would be many gigabytes—what is your Photoshop Info panel showing for Document Sizes?

[pets]

@ rob: yes, 100 Megapixel raw, saved as .psb with 30-50 layers in 16bit (10.000 x 10.000 px)

info panel shows about 10-30 GB filesize.

i am wondering, why PS is not allocating more memory, as i am monitoring, and its mostly allocating 30-40GB RAM.

@ d fosse: may i conclude, that my laptop (mbp 16" 8core, with 64 RAM and external 1TB ssd for scratch) is really too low end, and i need a tower computer with more i/o and 2 ssds inside? (in my case a mac pro 2019 or the mac pro 2022 with silicon)

thanks!

[D Fosse]

Why don't you just try it?

It's not a bad machine, but it doesn't have the ultimate scratch disk setup for Photoshop. This is just what Photoshop requires from any machine. My desktop system would also be "underpowered" if I disconnected all the internal disks and put PS scratch on an external.

If it doesn't make a difference, we'll look elsewhere. But make no mistake, the scratch disk is essential for good Photoshop performance. It's usually the first real bottleneck that people will hit.

[rob day]

i am wondering, why PS is not allocating more memory, as i am monitoring, and its mostly allocating 30-40GB RAM.

There’s the Preferences>Performance>Let Photoshop Use setting, but I don’t hink there is any way a huge file like that doesn’t go to the scratch disk—seems like the history states alone would overrun your available RAM.

[pets]

yes i know that "Preferences>Performance>Let Photoshop Use setting"

i allocated 50 from 64GB RAM, and used the wrong word, i meant "why photoshop wont use more than 30 or 40GB ram, when constantly monitoring.

what do you mean by "Why don't you just try it?"

probably you meant, i should get a tower computer or in my case a mac pro and use 2 built in ssd's, one has 1TB and is always empty for ps. speed would be 3-6GB/sec, as it uses the internal pci express at full range.

yes, true, i can try it. i just wondered why / if the latest mbp 16" is too low end, and i had to clear that question in my mind, so thanks for helping! i assumed, that using a high end, expensive laptop, wether pc or mac is sometimes a bottleneck, yes?

[D Fosse]

What I mean is put the scratch disk on your internal system drive. Not the external. Provided you have enough free space there, as I said it should be at least 500GB, but in your case probably closer to 1TB.

Yes, the main problem with laptops for Photoshop is disk space, and for that reason I usually don't recommend it. With a desktop machine, you can put in as many drives as you want - NVMe for system and applications, including PS scratch, and additional high capacity spinning drives for the images.

Plus, a laptop will usually run all components at lower than nominal speed, to keep them from overheating. There's simply no way you can get adequate ventilation in a laptop.

[pets]

---

@D Fosse  schrieb:

What I mean is put the scratch disk on your internal system drive. Not the external. Provided you have enough free space there, as I said it should be at least 500GB, but in your case probably closer to 1TB.

---

i simply cant right now, as i only have the macbook pro laptop. it has only one ssd.

therefore i said, i must buy a mac pro tower.

i ve been aware of the scratch disc problem, and i am using an empty 500GB external ssd since years,

it runs about 1000MB/sec, but i assume, thats too slow ?? i need 3-6 GB/sec for the scratch disc, so i plan to buy a blade scratch-ssd for the mac pro.

thanks nevertheless

[rob day]

But maybe in the end it’s more of a processing problem—a 10 layer 100MP file could have a billion+ 16-bit pixels

[pets]

---

@rob day  schrieb:

But maybe in the end it’s more of a processing problem—a 10 layer 100MP file could have a billion+ 16-bit pixels

---

yes, i agree. you mean, its natural, that such a huge file cannot be processed in real time with photoshop, no matter how fast the scratch disc is?

i dont know another way, how to process  100MP files with 16bit, as they usually have many gradients and i got heavy banding in 8bit. (it will be printed at the size of a house) maybe i can reduce the layers to 20 and reduce my history to 10 steps. yes.

[rob day]

Sometimes banding can be a screen artifact happening on the conversion from the editing space profile into the monitor profile—it might not happen in the output.

When you print the color has to be screened—either halftones for offset, or stochastic for something like inkjet. Halftone dots definately can’t reproduce more than 8-bit’s 256 levels, stochastic might do a bit better, but for billboard sized output I doubt you will see improvement in the output. The advantage with 16-bits is big color corrections don’t drop gray levels.

Given the difficulties the file size is going to create, it might be worth outputting some 8-bit vs 16-bit tests to see if the banding is actually in the output.

[D Fosse]

I finally managed to do some testing on my own.

I took a 20 000 x 15 000 pixel file, and kept adding layers. Some pixel layers, some adjustment layers, some painting, all of them with active masks.

I saved it as a PSB with 18 layers, and the saved size on disk was 25 GB.

The whole time, Photoshop reacted instantly, in real time, with no perceptible lags. The only time I could sense some slight stutter, was when painting with a 3000 pixel brush (quarter to half second).

This is a Windows desktop machine with i7 - 7700K, 32GB RAM and a Quadro P2200. It has two onboard internal NVMe drives, system on one, primary Photoshop scratch on the other.

EDIT: oops, forgot to check the scratch file size, but that's because I never need to. I know there's always enough for anything.

[pets]

hi d fosse,

thanks for your testing!

did you set it to 16bit?

cause, when i do the same testing like you, (same pixel dimensions and layers) in 8bit, i dont have any lag, too.

it only happens, when it is 16bit and from 10.000 px on upwards. then the lag is happening from 2 - 5seconds for each action i take.

@ rob: yes right, i know that banding is mostly gone when printing in cmyk, but i have to sell the images to my clienst for all purposes, like social media, printing, websites, so i cannot deliver images with banding and avoid this by working in 16bit, if those images have gradients (which is the case with a large number of images)

thanks

[D Fosse]

16 bit. 

[pets]

ok, so my macbookpro 16" with 64ram and 2TBssd is not enough, i need a workstation with 2 internal ssd´s

[pets]

for future users of photoshop:

- i ended up selling my hardware from 2021 and getting a "mac studio m2 ultra" with 128GB RAM and 4TB SSD.

(+ macbookpro m1 max with 4TB SSD)

- reduced history steps to 24

this new hardware solved my issue from 2021 to 90%.

- additionally i prefer working with 8bit files, and transform it to 16bit in the last step, which prevents most banding, too.

- so, as you all mentioned, huge amount of ram and free internal disk space for scratch disc is crucial for photoshop, if working with huge 16bit files (15.000 x 15.000 pixels) at 40GB filesize.

what do you think of adding AI features to photoshop - for users, who are not aware of this issue?

AI could analyze photoshop, and suggest solutions like "your scratch disc is too slow" "your ram is nearly full", "photoshop keeps slowing down as your ram and scratch discs are full"

i know many users (friends & colleagues), who never heard of this scratch disc, history steps and ram issue, and they complain about their slow photoshop performance. i will explain it to them and help