Living with lag - 16 bit - have we reached Photoshops limit?

[docubyte]

Working on huge PSB retouch files (Between 12 - 20k pixel dimensions) 16 bit, layered, adjustment layers, often (but not always) smart objects inside. Avoiding vector feathering where possible + avoiding layers styles too, yet still am am having laggy, jumpy experiences with the fastest machines my company can afford. (PC)

The setup for Photoshop I have understood is generally considered (please correct if I'm wrong) - SSD's for Primary and Scratch and as much RAM as you can fit in. Due to lack of PS utilisation of multiple cores, generally fast i7 chips are best.

We are running 64gb RAM, big M.2 drives (primary and scratch), i7 8020k and cards such as a 1080ti. Its about a 2.5-3k (pounds) to build our machines.

I guess the question is - are there further upgrades to the hardware that would see any improvements? Would 128gb Ram be significant? Even 256gb on a Xeon board? The cost does cause issue of course - but we are at a loss of what to do.

Or is there something that I am missing completely here / or should generally just learn to live with lag?

Generally what I see in hardware reviews specifically for Photoshop speeds is how fast a blur will render etc, but what we need is something that we can run enormous files smoothly,

Many thanks

[D Fosse]

Those files are big, to be sure, but your machine should be able to handle it.

Lagging in Photoshop is often caused by buggy video drivers. The GeForces are gaming cards, and the priorities are to get the latest games to work. It's not an unreasonable assumption that this requires the odd shortcut or two in the driver.

I'm not going to tell you to get a NVidia Quadro card, only to hear later "that didn't fix it". But I do have a feeling that might be it. Quadros are optimized for graphics, 3D and CAD - not Biomutant or Dragon SlayerZ or whatever they're called these days.

[docubyte]

Very much appreciate the insight D. Fosse - thank you. I hadn't contemplated the cards - I've not had experience with Quadros but heard good things. My experience with a 64GB ram, it 7090, 1080ti M.2 (scratch + primary) are still that with a 16000px document, in 16bit with multiple layers (and some smart object with smart filters applied) is that zooming in an out (scrubby) will be good for a moment, then lag. Then zoom in, the hang. Brushing - almost painful. I am wondering too if 128gb RAM will help.

[D Fosse]

Slow brushing should indicate the video card - wouldn't you agree, Dave?

[davescm]

Yes I agree  - but it would be good to get some more detail. I have the next card down (the plain vanilla GTX 1080 as opposed to the Ti) and I rarely see brush delay unless I use an enormous brush. I also tried scrubby zoom on the large file without any issues at all.

I'm using Nvidia driver 398.11

Dave

[davescm]

Hi

What operations are you seeing slow down?

I don't usually use files that large but I just tried a 20k x 13k image and built up a few layers (20) including smart objects - blending and adjustments. I also added a few layers of brushing and have seen no slow down. The file generated a 24GB scratch file and Task manager shows Photoshop using 13GB RAM

I have a slower processor than you i7-3930k, but similar RAM ( 64GB) and an  Nvidia GTX1080 GPU.  My OS and apps are on an SSD but my scratch is pointed to a large spinning disk.

In Preferences Performance my cache levels are set to 4 and Tile size 1024k. I let Photoshop use 70% of available RAM and GPU drawing mode is set to Advanced

If you want to link to a file I will download it and see how it performs here

Dave

[JJMack]

It is not surprising that Photoshop will process a small amount of data in a shorter period of time then it can process a large amount of data.   If you create a document with a huge  canvas size add 8000 layer some of which are smart object layers with smart filters adjustment layers, layer style, filter mask and layer mask the amount of data Photoshop has to process to re-render smart filters in order process adjustment layer, layer mask etc to render your  images composite view will be huge and will likely will bog down today's fastest powerful PCs and Macs.  All Processing Photoshop does can be multi threaded well.  Some processing need to be done in a sequence of processes.  Photoshop only run on Mac OS versions and Window OS versions. There is no Photoshop version for other OS systems that is optimize for today's super computers. Adobe does not developed a Photoshop version that tries to run in real time without any lag.

[rayek.elfin]

I have read similar accounts before, and often rolling back to an earlier version of the video card drivers fixed them. Some users reported performance issues with a very specific driver release, while others had no problems: sometimes one driver version may not 'like' your hardware combination.

Also check that the 1080ti is slotted into a PCIe x16 3.0 slot on the main board.

[Trevor.Dennis]

docubyte  wrote

Working on huge PSB retouch files

I've been complaining about unworkable lag with big projects for several years now.  Six years of that was with a 3930K system with 32Gb RAM and lots of SSDs, but for some time I have been using a 7900X system with 64Gb RAM, two 512Gb Samsung 960Pro M.2 drives with lots of space.  Both systems had the same GTX970  (I'll put in an RTX1080 when they become more available).  I let the CPU Turbo up to 4Ghz — it will go to 4.4Ghz if I let with BIOS settings, but it gets too hot)

The projects that have caused me issues had much smaller pixel dimensions, but were complex with a 'lot'  of layers including nested Smart Objects, layer styles, and lots of vector layers.  One of these projects became so slow it was completely unworkable.  I have had discussions  like this before, and people have talked about turning off rulers and other tips and tricks.  What makes it confusing is that I  'think' it is  inconsistent. 

When I first started using this system, I  was thinking that it was  not using the Scratch space, but  I have seen some fairly big Photoshop Temp files  since (>30Gb).     Worst case with a  Samsung 960Pro would be  500Mb/s, and best 3.5Gb/s.   I don't know how clever Photoshop is with using scratch space — it surely would not need to load the entire 30Gb to access a chunk of data in the middle.  I  feel sure that large scratch files are a bottleneck though.

I also feel sure that Smart Objects can present a bottleneck.   How does Photoshop handle them?  Do they live in RAM or on the Scratch drive.  That is probably fluid depending on  project size and hardware.

I TA'd for Bert Monroy at MAX in 2016, and I'd intended to ask him how he handled hardware limits with his HUGE Times Square illustration. On the day I was too overawed to speak properly to Bert (He is my #1  Photoshop hero) but what I do know is that Times Square had thousands of files which were flattened before adding in to the main project.   I have tried  a similar tactic where I open a Smart Object in a new window, and save the PSB file as a new document. Then closing that PSB and flattening the SO into the master document. This helped, but I had to either clear the cache or restart for  best effect, but it was not a  full cure, so other factors were involved.  Thinking back, the worst case was a  clock illustration that used mostly vector layers, but my understanding  from my time with AutoCad, is that vector files are nowhere near as big as Photoshop files. 

So I don't have an answer. I  totally  expected my i9-7900X system to fly and cope with anything, but that simply hasn't happened.

[Trevor.Dennis]

OK, I have just opened that clock file. It currently only has about 50 layers because a lot of SOs have been flattened.  It is also smaller  in pixels than I remembered, at just 2500 pixels square.  The hamster wheel is an SO with just five layers.  I made no edits  after opening  the file, and started to drag the hamster layer about the canvas.  It jerked moving suddenly 500 to 1000 pixels at a time with about a half second lag between jumps.

This screen grab shows what was happening while I dragged the hamster layer.

Now here's  an interesting thing.  I was viewing at actual pixel size in the above situation. Even reducing the zoom level to 50% improved things considerably.  In fact all zoom levels other than actual pixel size, were OK.  That sounds like a video card bottleneck.  Remember I am using the same GTX970, (because I thought a  new NVIDEA generation had to be on its  way, and the card from the old system would be OK till then)

Jeff Tranberry says that cache levels  and tile size makes a lot of difference.  These are my settings

Scratch file size

I am not sure how to monitor GPU usage with Windows 10 and an NVIDEA card.  Anyone?

Trevor.Dennis

docubyte 

Do we know what OS you are using?

BTW I don't think Xeon systems work well with Photoshop, because clock speed is king.  

Have you had a good trawl through the Pugot Systems articles?  Hey, it looks like they might have got their hands on an RTX GPU to play with.

https://www.pugetsystems.com/all_articles.php

Hmmm...  sorry guys. false alarm, but an interesting discussion about them.

[docubyte]

Thanks all for the responses. Good (Well perhaps not great?) to read other are experiencing the same issues. I have heard about turning rulers off etc, and trying to make sure our drivers are up to date. Our exact spec of our latest build is:

64gb Ram

1070ti

NVM 256 Primary / Evo 970 2tb Scratch

i7 8086k 4.00 ghz

I have found zoom levels (when zoomed out) performance is far better. Of course suddenly you then need to zoom in and fix something so - its largely impossible to work like that - unless perhaps the 'new window for' method could at least provide a workaround (but that perhaps has different demands on the graphics card)

On another forum I hear that Quadros are no better than gaming cards: (certainly results are remarkably similar)

https://www.pugetsystems.com/labs/articles/Photoshop-CC-2017-NVIDIA-GeForce-GPU-Performance-899/

https://www.pugetsystems.com/labs/articles/Photoshop-CC-2017-NVIDIA-Quadro-GPU-Performance-900/

I am yet to play with cache and tile size - but I think these paramters should be a good start to aid the experience - but I feel that generally it is the demands of a 16 bit file with large canvas dimension , smart objects and multiple layers etc which will usually often cause a juddery, work a bit, wait a bit experience. In 8 bit the file is fantastic to work with.

[D Fosse]

docubyte  wrote

On another forum I hear that Quadros are no better than gaming cards: (certainly results are remarkably similar)

Yes, but that wasn't the point. The point was bug-free and reliable drivers.

Nominally, the GTX's blow the Quadros out of the water at the same price level, that's no secret.

[docubyte]

Clear and understood - thank you. I guess this will take a bit of experimentation - I have been having strange artifacts with our latest build additionally, and crashing upon resize. Frustrating.

[JJMack]

Trevor.Dennis  wrote

This screen grab shows what was happening while I dragged the hamster layer.

The resource pegged all the time seems to be your Processors add more and faster processors see if that helps any. There is a lot of processing going on and all of your cores seem to be pegged ....

[Trevor.Dennis]

JJ, I had the Resource Monitor set  to Overview This is what it looks  like on the CPU tab. The is one more thread that I could not fit on my screen, even in portrait mode.  Note the CPU is running at 4.18Ghz.  I was not looking at the temperatures,  but the fans only notched up by one step.

This tells a bit more of the story. E: my primary scratch drive, and there is just the one minor blip.  The Photoshop temp file is just 690Mb — not  even a Gb, so drive access is not the issue. 

We still don't know what operating system the OP is using.  If  Windows 10, then the OP can do the same checks to try and isolate, or  rule out things.

JJMack

I have a feeling you once put up something that showed GPU activity.   I don't know how to do that, and it would be interesting.   Right now, the big difference that zoom ratio makes to the lag, suggests the video card.

docubyte  Have you tried different Cache and Tile sizes?  I'm thinking you should be using 6 and 1024k.  I am  sure you  know you  need to restart Photoshop for  changes to  take effect.

How  many screens are you using, and what are there  resolutions?    I have a  2560 X 1440  main screen, and a pair of 1200 X 1920 screens in portrait mode.  I have never thought to look at this aspect, but only one screen is updating,  so I am guessing this  is not a  big factor.   Your GPU is roughly similar to mine, being one level up.  I am going to bookmark this thread, and whenever the RTX GPUs become available, I'll get one and notify you of what difference it makes.

As to Smart Objects, there is no disk activity on the C:\ drive, so I don't see that being an issue.  We so badly miss having Chris  Cox on this forum.  He would tell us exactly what resources different functions used, but sadly, Chris is no longer  with Adobe.

I have also tried using the brush, and I am getting the same huge lag.  This was a pressure size, fully hard, brush just 50 pixels.  Remember this project is just

And does not have a ton  of layers.

I tried Edit > Purge > All  and it made no difference.

Zooming back  to Fit-to-Screen (which  worked out to 48%) has the brush working normally again.    It has to be a redraw  issue.

I just tried restarting Photoshop after changing my cache settings,  and it was a lot better, but still a long way from silky smooth

Photoshop  recommends these setting for Huge Pixel Dimensions, which this document isn't.  docubyte  should be testing these settings I'm  thinking.

[Trevor.Dennis]

OK I feel  daft  now. Monitor GPU was just a wee scroll down on the Resource Monitor.  No activity while  dragging  my SO layer, but it is still working better.  Not great, but definitely usable.  Same thing with that 50 pixel brush.  Still a wee lag, but usable.  So Cache and Tile settings are crucial it would seem.

[JJMack]

Than your overview display does not correlate to your detail  view.  You can also change the way the Task Manager CPU display its view for your processors.  With Window 1809 I no longer get a GPU monitor with my old Nvidia Quadro 4000.  DXDIAG show its driver model is WDDM 1.3 with Windows 10  1809 WDDM 2.0 is required for GPU motoring.   My machines with newer Quadro models do have a GPU monitor in windows Task Manager.

You see no GPU display in my Window 10 1809 Task Manager

Windows 10 Resource monitor Total CPU use look bogus to me.  With my machine idle it shows 80% utilization and spikes up to 100%.  There is something wrong there. Or I do not know how to read its blue graph meaning.  It does not look like what total cpu utilization or cpu available you machine was surely using some CPU not no CPU and you detail view shows not of your processors are 100% utilized..