Advice processing super large (20GB-30GB) fles

[Doc_Pit]

Hope I can get some advice on the best way to configure Photoshop to work with super large (20GB-30GB and larger) files.  When Photoshop—running on my honking big Windows PC—tries to swallow those honking big 16-bit files, brush strokes lag, and commands are not executed with reasonable speed.

Some things to note:

1) I’m aware of strategies for working with smaller files that can feed into the base file.  However, when working in the complete base file, it is possible to manipulate any of the other layers to gauge how those manipulations affect my work on the current layer.  That’s critical when I’m trying to blend the content of the current layer into the total image.  Can’t do that when working with smaller feeder files.

2) I have the horsepower (a Dell Precision 7820 Tower Workstation with a Xeon Gold processor, 96GB of the fastest RAM [at that time]; an Nvidia Quadro RTX5000, 16GB video card, a Class 50 1TB SSD for the operating system and an internal 960GB Optane Drive for use as a scratch drive, etc., etc.)  My guess is that I don’t have a complete understanding of the extent to which PS can actually use that computing power.

3) While PS is bogged down with those files, my efficiency is still 100% with several hundred GBs still free on my scratch disks.  (The dedicated internal 960 GB Optane Drive is my primary scratch disk.)

4) I have my preferences set to allow PS to access 85% of that 96GB of RAM.  My understanding is that 64-bit Photoshop can use as much RAM as I can allocate, which comes out to 65+GB of available RAM, which should easily handle a 20GB-30GB file.  My understanding is that, e.g., 50 history states would require 1000GB on a scratch disk for a 20GB working file.  By limiting to 1 history state, I have seen some improvement in performance.  But that’s “unworkable” improved to “marginal.”

5) My Performance settings are as follows: Mem Usage 85% (65023 MB), Use Graphics Processor/Open CL, Huge Pixel Dimensions, Cache Levels: 8, Cache Tile Size 1024K, Multithreaded compositing.  My scratch disks include (1) D:\ dedicated internal SSD Optane Drive w/830GB of free space, (2) C:\ internal Class 50 system SSD w/701GB of free space, (3) O:\ an external SSD with 837GB of free space.

If anyone has any experience working with super large files and can offer any advice, I’d love to hear from you.  Thanks.

[D Fosse]

I can't recall that I've hit the 20 GB mark lately, but I do work on 10 GB files fairly routinely with no problems, with less specs than you - but with max priority on the scratch disk. This is all about the scratch disk, that's where all the action will be. I have several TB, the first 2 TB on fast NVMe.

It has to be said that Dell is notorious for stuffing their machines with all kinds of "helpful" additional software and utilities - but what happens is that all this clutters the operating system and gets in the way of Photoshop. I'd make sure as much as possible of it is killed or put to sleep. Check the Startup list in the task manager.

What's this for? I hope you're not making large prints and are under the false impression that it "has to be 300 ppi" at all reproduction sizes?

[Doc_Pit]

D Fosse and Todd Morgan, thanks for your replies.  Also, Todd_Morgan, with respect to “Engaged, 2 hours ago,” congratulations and best wishes for health and happiness!

D Fosse, regarding “I hope you're not making large prints and are under the false impression that it ‘has to be 300 ppi’ at all reproduction sizes?”  This is my thinking (and I am wide open to correction from someone who has a much higher level of experience and expertise than I do).  Three points:

1)  From a purely philosophical point of view, I believe that an image isn’t just what we see at a given size or distance.  It’s a creation with its own integrity.  My satisfaction comes from knowing that the images I produce are the very best I’m capable of producing.  I sometimes work at high magnification to correct “problems” that no one would ever see.  Why?  Because I know they’re there:  an idiosyncratic (some might say “idiopathic”) perspective no doubt.

2)  I want maximum flexibility in the use of my images.  I want to be able to collect my images in a book of prints while also being able to exhibit those prints at large scale (e.g., 20x30 or 30x40 inches).  Also, I don’t completely buy into the idea that a lower resolution is fine for a 30x40 inch exhibition print because people are viewing at a further distance viz-a-viz a 10x10 inch book print.  In art museums and at exhibitions, I’ve seen many people get up close to paintings to inspect detail.  People enjoy doing that, and (IMO) the image should reward their curiosity.  Some of the fine art printers say they prefer prints that are at least 180 ppi.  The reason I put the money into a PC with 96GB of RAM is because I did not want the hardware to be the limiting factor in the quality of the work I can produce.

3) Regarding the hardware:  My two fastest drives are the internal drives.  I do my work on D:\ (an internal 960GB Optane Drive).  I also designate that drive as my first scratch disk.  Currently, it has 830GB of free space.  I designate my next fastest drive, my internal system drive (C:\) as my second scratch disk because, even with the OS, it still has 701GB of free space.  My third scratch disk is an external SSD with 837GB of free space.   It’s a fast SSD, but not as fast as an internal SSD.

Again, I’m usually running at efficiency in the high 90s-100% even as I encounter this sluggish performance.  My two key questions are: Given the hardware, why is it that PS can not process those very large files efficiently?  Is there some more efficient way to configure my hardware?  Thanks very much for your comments.  Much appreciated.

[Todd_Morgan]

What drive are you working on for your production files? I have my scratch on the system C drive, and do all my work on the D drive. Both SSD. And work on large file format also. My system is an old workstation with 64gb ram. Also... are the files PSD or PSB that you are working on?