Performance and Hard Disk Swapping

[keithconover]

Some InDesign tasks that require moving a lot of pages and graphics can take a long time. Many minutes to an hour or more.

I have a desktop Windows 10 Pro PC with:

• Xeon W-2155 @3.3 GHz (13.75MB Cache, 10-Cores, 20-Threads, 48 lanes, 140W, Benchmark 24983)
• 128GB Kingston ECC DDR4 2666
• Gigabyte GeForce GT 1030 2GB Fanless
• Samsung 970 EVO Plus 1TB M.2 NVMe PCIe 3 x4 SSD (3500/3300 MBps R/W)

Given those performance issues, I just doubled the RAM to 256 GB.

Nonetheless, some tasks, such as changing a heading style to move 200 pages with text and graphics, still takes forever (in Internet time, that is): half an hour or longer.

Looking in Task Manager as this so-far half-hour task is whirring, shows the following:

• 9% total CPU usage, of which about 5% is attributable to InDesign (5% of total, not 5% of 9%)
• 6% of total RAM usage, of which about 2 GB is InDesign's.
• about 1% of the SSD disk usage, almost all of which is InDesign's, at 0.6-0.8 MB/s.

This system does have a fast SSD "hard drive," but wouldn't InDesign get my task done in minutes instead of hours if it could use more RAM and avoid swapping to disk, even if it's a fast SSD?

Still seeing an hourglass at 45 minutes. Sigh.

[Randy Hagan]

There are lots of reasons why that might be, but let me offer something quick to try that may help:

1. Open the File
2. Select the File>Save As... menu command.
3. Save the file in its original location with the original name. When the redundant command pops up and asks if you want to replace/overwrite the existing file, say yes.

This minimizes/simplifies the existing file by creating a current state of the existing file, rather than yet another incremental update to the file over its original version, with unknown additional incremental updates of the file.

If that trick doesn't work, and it does far more often than you may think, another remedial fix would be to go back to that File>Save As menu command, and in the Save as Type: options box change it to InDesign CS4 or later (IDML)(*.idml) InDesign interchange file. Close the InDesign document (*.indd) file, then open the .idml file and save it back to your original file location and same name again to replace the offending .indd file.

These two actions help you clean out any debris in the file, and can fix a myriad of file issues that get more difficult by how old and complex the original file may be. It doesn't fix all complications, but it fixes issues often enough that this is the first place I'd start.

Please let us know if this helps, or it doesn't. From there we can offer some more suggestions which may help.

Hope this does help you,

Randy

[keithconover]

Thanks for the tips, Randy!

When I came back after two hours, InDesign had crashed.

It automatically restarted and had the file without the change I had tried to make. (See https://community.adobe.com/t5/indesign/new-page-gt-new-odd-page-kills-span-columns/m-p/11857602); I was making the changes starting at the end of the 200 page document, and moving to the beginning. The slow performance and eventual crash only occurred once I made the change to the very first of the headings.

I tried "Save As" and overwrote the original file.

Again, seeing an hourglass, and will have to wait to see how long the SSD churns this time. Interestingly, though the CPU  usage is about the same in Task Manager, this time the RAM used by InDesign is only 1 GB instead of 2 GB. The SSD transfer rate is about 1.5 MB/s, so significantly faster. Maybe it will finish before crashing. That's progress, I guess. I will report back after InDesign finishes its disk-thrashing.

Thanks again!

Update: after an hour, RAM usage up to 1.6 GB, and SSD transfer rate down to 0.6-0.7 MB/s. Still thrashing.

Another update: after an hour and a half, InDesign had exited. When I restarted it, the unchanged file was there. Saved as .idml and then as .indd, that cut a couple of MB off  the file size. When I again tried to change the paragraph style on that one heading, Task Manager showed InDesign using about 5% of CPU (that's always been about the same), 1.2 GB of RAM, and the SSD transfer rate was up to 1.9 MB/s. Decided to leave it to go out to get dinner. When I got back an hour and a half later, InDesign had once again exited without making the change.

Decided to use the Pages Panel to add a page so that the H1-new-page heading was already on an odd page before changing it to H1-new-odd-page paragraph style. This resulted in an hourglass for maybe a minute and then everything seems fine. So this particular problem is fixed, but it still seems that InDesign could be recompiled to take advantage of more than 2 GB of RAM. (Easy for me to say, I know.)

[Randy Hagan]

I guess the good news is you see measurable progress. The bad news, of course, is that in the total scheme of things it's not much.

Barring some of your hardware somehow being bad, you have more than enough horsepower to push InDesign. Heck, by my meager 16 GB of RAM and SATA 2.5 SSD, you've got four times the drive speed and 16 times the RAM. I'm not worthy.

By my experience, there's not much that could slow down the program to a crawl, per se. I know that some folks have found performance improvements by turning off hardware acceleration for their hard-core video cards. I don't know the specifics of your 2GB video card, but as counter-intuitive as it sounds, folks have found performance improvement by turning their high-end video upgrades off.

Are you having these performance slowdowns with any file? Or just specific files? What kind of documents are you working on? How big are the document files – in file size and page count? You mention working with lots of pages and graphics. I've done catalog work where large sections have slowed my systems down to a crawl. I'm no apologist for Adobe, but when I run across real slowdowns, they're often associated with file issues.

If you can let us know more about the files you're working with, hopefully we can help you past this.

Randy

[Dov Isaacs]

You indicate that your document is “200 pages with text and graphics.” Are those graphics embedded or linked to? How big is the actual InDesign .indd file? Generally speaking, if you embed graphics other than simple vector objects created within InDesign (such as high resolution raster images or pages from a PDF file), you are setting yourself up for performance problems.

In terms of your Windows system configuration, there is absolutely nothing there that would be the cause of the performance issues. 128GB of RAM, much less 256GB is absolute overkill. Your problem is not paging. And it is not your graphics card (the GPU of which isn't used by InDesign anyway, unlike Photoshop or Illustrator). It is just that there is quite a bit of “work” being done when you change the formatting characteristics of styles and effectively everything on those 200 pages must be examined and potentially moved, even microscopically. If graphics are embedded, such movements within the document simply increase the workload.

[keithconover]

Hi, Dov, and thanks for your reply.

The file is 56 MB. The graphics are all linked, none are embedded. Not sure if that counts as big or small for a 200 page file, depends on you look at it.

Randy, thanks for your reply as well.

It has to do with this particular file and this particular paragraph style change. Once I learned to break my 200-page files into Stories by section, I have not had similar problems with the other files. Of course, I have not tried this particular new-page > new-odd-page paragraph style change on them yet, and I plan to. But I think I will use the "add a new page to make the section start on an odd page" procedure first!

[Dov Isaacs]

56MB is pretty big for the .indd file itself. And if you are linking graphics, perhaps the bulk of the document size is the lower resolution preview image stored within the file.

Yes, breaking the file into sections would definitely help.

Good luck!

[keithconover]

OK, it's my own fault. I should have thought this through better. I ended up getting a new PC with Windows 11 presintalled (easier and better than upgrading a Windows 10 install).

Here are the specs:

• Core i9-12900 Desktop Processor
• Kingston FURY Beast 32GB (2 x 16GB) 288-Pin DDRS SDRAM DDRS 5200 (PCS 41600)
• ASUS Dual GeForce RTX 3050 8GB GDDR6 PCI Express 4.0 Video Card DUAL-RTX3050-O8G
• SAMSUNG 980 PRO 2TB PCie NVMe Gen4 Internal Gaming SSD M.2

And I'm still bottlenecked and waiting for 15 minutes for my file to create a PDF. In Task Manager, I can see that InDesign is using 2 GB of RAM, and then having to swap to disk. It's my fault, I know. InDesign is limited in the RAM it can use, so more RAM didn't help. But perhaps I should have invested in a faster CPU.

On the other hand, it's porbably just my fault for creating a single 500+ page document with maybe a hundred cross-references and loads of linked graphics… except that I want to have lots of cross-links in this text, and they only really work in a PDF if they're in the same file. (I know that if you put PDFs in the same folder, cross-references will work, but I can't expect users to know that and use them that way.) Again, advice for those upgrading their PCs: put your money into the top CPU you can buy and fhe fastest RAM and SSD you can get, instead of into more RAM, because over maybe 4 GB (to allow InDesign to exist with other apps) more RAM will have no impact on this kind of performance.

[keithconover]

I have broken my textbook into smaller files, 30 MB is the largest. I also found that I could really decrease the file size and improve performance by:

• After every series of edits:
  • Save a Copy > add DRAFT-xxxx-xx-xx to the end of the filename, where the x characters represent the current date
  • Save As > Save over itself
• Breaking large chapters into sections so that each section, rather than the entire chapter, is an InDesign "story." Sometimes I have to break the sections in two. For example, I checked the number of words in the section Specific Knots & Hitches by triple-clicking in the "story" and opening the info panel. It was 18,823 words, and I was having performance issues with it. I then broke it into Specific Knots & Hitches at 12529 and Multi-Knot Devices at 6294 words. That seemed to help performance significantly. I am sure that hyperlinks, cross-reference, footnotes and the like factor into it, but it's easier and better to break the section in two rather than stripping out things that are part of the meat of the section.

[Robert at ID-Tasker]

@keithconover

Go back to your old machine and create RAMdisk.

[Anubhav M]

Hello @keithconover,

We understand that encountering technical issues can be frustrating. Would you mind trying the suggestions shared in this community post (https://adobe.ly/3xQQUQ0) and checking if they help?

Looking forward to hearing from you.

Thanks,

Anubhav

[keithconover]

Robert, my first laptop (the Timex Sinclair, the Osborne 1 and the Cromemco Z-80 weren't really  "laptops") had no hard drive, just a floppy drive and a RAMdisk. But my current PC has a SSD which is pretty much the same.

Anubhav, I had already done everything in that post except for turning off Preflight. I found this made a big difference. Thanks!

[Robert at ID-Tasker]

@keithconover

RAM is at least 100x faster than the fastest SSD 😉 1000x than average SATA SSD 😉 

But only makes sense, if you've at least 64GB - preferably way more - like 512GB 😉 

[keithconover]

So, if I understand this right, InDesign can only handle up to XXX of RAM after it starts swapping to disk. And even with an SSD, this is the kiss of death: it gets stuck swapping to disk, unless, maybe, you wait several hours to be able to use InDesign again. I've only got 32 GB of RAM in my desktop PC.  I had thought that getting more RAM for the PC didn't make sense, given that limitation. But given what you said, maybe I want to upgrade the RAM in this PC, and get a RAMdisk program like I had back 40 years ago. Do you recommend a particular RAMdisk setup?

[Robert at ID-Tasker]

@keithconover 

InDesign has no control over when swapping will occur - it's OS who takes care of it.

InDesign just gets info from the OS how much RAM is available - not from "where" - RAM or SSD or HDD. 

You can have 8GB of physical RAM - or even less, but if OS can't fit in the RAM by itself... - and "unlimited" amound as a swap. That's the primary reason why swap exists.

First thing I ALWAYS do on a fresh Windows installation - with at least 16GB of RAM (if I buy a PC with less RAM, it's only because I can buy more cheaper) - is to TURN OFF swap.

If you have 32GB - I would definitely turn swap OFF straight away - which should increase performance as OS won't waste time on swapping memory unnecessarily - apps might think that there is plenty of RAM so they get "greedy".

RAM is the cheapest way to increase overall performance of the system - so if you can - MAX IT OUT.

The only reason that I have "only" 512GB RAM in the beast is because I can't install more 😄 

16x 32GB sticks is the max that can be installed - DELL T7610. 

Plus 2x 12x + HT = 48 cores 😉

I do all of my coding on a laptop - only 16GB RAM, but enough for basic testing in InDesign - because it boots in few seconds and wakes up even faster 😉 and uses way less energy 😉 

If I had to work all day long on bigger files - it would be no brainer.

[keithconover]

You've persuaded me. I just ordered 128 GB RAM for my desktop PC. In case anyone wants a quick guide on where the Windows 11 paging settings are, they are at:

Settings > System > About > Advanced system settings > Performance > Settings > Advanced > Virtual Memory >Total paging file size for all drives > Change > Automatically manage paging file size for all drives > (adjust as desired) 

[Robert at ID-Tasker]

@keithconover

ROTFL 😄 nice 😄 

What kind / size / qty modules? 

But it should be set to "0" / "no paging file" - not "adjust as desired"? 

(screenshot found online) 

[Robert at ID-Tasker]

@keithconover

What is the model number of the motherboard? 

[keithconover]

Two things. As far as the model of the motherboard, it's a ASUS ROG Maximus 2690 Hero (WiFi 6E) LGA 1700 Intel 12thGen A TX Gaming Motherboard. I used to build my own PCs but more recently I've gotten the people from https://silentpc.com/ to build them for me. Also excellent support. I just emailed them that I wanted to upgrade from 32 to 128 and they immediately sent me an email with a link to the very specific RAM that they recommended I get from Amazon. I just clicked and ordered. (A very easy button.) This is the fifth PC I've had them build for me. While I have experience with PC hardware, and have a needle point soldering iron and I'm not afraid to use it (also a radio ham) they have much more experience and I'm willing to pay for that experience. Also their support (as above) is outstanding.

Also, as far entering a zero for the Windows virtual memory? I used to do that all the time in the era of smaller hard drives. It deletes pagefile.sys from your hard drive, which makes a lot more room. The default on my PC was 2048 MB (2 GB).

[Robert at ID-Tasker]

You mean Z690? 

Personally, I would go for used on ebay.

Hate ebay as a seller - but sometimes you can find a good deal - especially when person selling stuff has no idea what they are selling. 

[keithconover]

Sorry, typo, yes, it's a Z690. Intel Core i9-12900. ASUS Dual GeForce RTX 3050 8GB GDDR6 PCI Express 4.0
Video Card. SAMSUNG 980 PRO 2TB PCie NVMe Gen4 M.2 SSD. Connected to a LG 40WP95C-W 40” UltraWide Curved WUHD (5120 x 2160) 5K2K Nano IPS Computer Monitor. With that I can work with two pages side-by-side in big size for my old eyes. After several months, it developed a bad row of pixels and I sent it back for repair. Got it back in 2 weeks, using it now. Their tech support chatbot is, like most such things, essentially a hyper-intelligent autistic psychotic six-year-old. But once I got past that, their human tech support was just amazingly good. As far as the RAM, I decided just to go with what they recommended as in the link, all of the right options were already selected. I figured I would waste a lot of time and effort to look for that same exact thing online elsewere. Time is money.

[Robert at ID-Tasker]

[...] Their tech support chatbot is, like most such things, essentially a hyper-intelligent autistic psychotic six-year-old. [...]

By @keithconover

Next time - if you are not doing this already - just type "speak to human" 😉

Works everywhere 😉

[keithconover]

Well, Robert, you're right again, partially. I did disable disk swapping, and now, when InDesign reaches its limits, instead of locking up, I just get a quarter-second or half-second pause and can keep editing, though this does tend to happen over and over. I upgraded from 32GB of RAM to 128 GB of RAM after this. As far as I can tell, there's no major significant difference, still occasionally get one of those pauses, though the system overall seems snappier. And in Task Manager, InDesign seems unable to use more than 2 GB of RAM. Do you know any tricks to tell InDesign that it's OK to use more of that RAM I just put in?

[Robert at ID-Tasker]

@keithconover 

Strange...

I have a laptop with only 16GB RAM and swap off:

1) InDesign just started and iddling:

2) with a 300 pages document with a bit of linked images:

3) after "closing" a document:

Still taking RAM...

But a quick question - becuase you've never mentioned that ...

Do you have Windows 32bit or 64bit installed?

[keithconover]

Windows 11, a 64-bit system. Weird that my InDesign seems to stop eating memory at a bit less than 2 GB.