Participating Frequently

Question

How to speed up PDF generation in InDesign

Forum|Forum|3 years ago
November 1, 2022
7 replies
9416 views

One of the main tasks that we do in our office is:

A. Merging an original InDesign file (approx 7.5Mb) with a CSV file;

B. Exporting that resulting merged document into a PDF file

During export, we've tried different Adobe PDF Presets: (High Quality Print, Press Quality, Smallest File Size, PDF/X-4, etc.) The "Smallest File Size" preset seems to be the fastest although not by much. It generally takes 3-5 minutes to create the merged document which seems a bit slow.

But the real sluggishness is when we export that merged document into PDF format. It takes anywhere from 4-8 minutes and the resulting PDF is only about 2.3Mb in size.

We've tried this on multiple machines. One is an iMac 3.5Ghz Quad Core i7 processor with 32Gb of 1600Mhz DDR3 RAM. The other is a Macbook Pro with 2.6 Ghz 6-core i7 processor and 16Gb 2400 Mhz DDR4 RAM. There's not really thimble's worth of difference in the export time. They pretty much run neck and neck and, when they don't, either one might outperform the other.

Anyway, I have two questions:

A. Is there anything I can do to speed up this process? It takes HOURS of our day for one staff member to create these PDFs. And most of that time is spent just waiting for the machine.

B. What part of the computer is most integral to this kind of task? In other words, if I choose to get a new Mac before the end of the year -- and had to choose -- would I want to go with the fastest processor...or the most RAM? Which would give us the biggest boost in speed for this kind of work?

Thanks in advance. I appreciate your help and would be happy to provide more information if it will help give a better answer.

This topic has been closed for replies.

L

Laubender

Community Expert

Hi @Sherry @ Lacy & Par ,

there are other solutions, however costly, where you can feed InDesign templates and get very fast PDF output.

One is pdfChip by Callas Software:

https://help.callassoftware.com/m/pdfchip

InDesign documents as templates:

https://help.callassoftware.com/m/pdfchip/c/189153

https://help.callassoftware.com/m/pdfchip/l/651973-overview-and-installation

Note: I have no personal experience with pdfChip.

Don't know if it will meet your requirements, but it's worth a view, I think.

Regards,
Uwe Laubender
( Adobe Community Expert )

Robert at ID-Tasker

Legend

There are tools that can fill PDF directly:

https://youtu.be/w6h7s8COkXA?si=3Ds8XsAyvi78leg0

Sherry @ Lacy & Par

Known Participant

No solutions, but watching this thread because I have the same issue.

Doing a postcard data merge, I've taken out all the graphics and created the merged INDD files to be text only overprints. I have 8 INDD files each with 4,000 pages (half of which are blank) and each file is taking a half hour to create a single PDF.... IF it creates a PDF at all, before it crashes.

Running a 3.4 GHz Quad-Core Intel Core i5, 40 GB memory and saving to the internal hard drive.

I know the Data Merge in InDesign is buggy, but it seems to be getting worse.

Thanks for letting me sit in 🙂

It's always been slow, but recently it's practically non-functional.

Sherry Baker

Robert at ID-Tasker

Legend

@sherry @ Lacy & Par

Can you split to smaller files? Like 1000 pages max? Or even less.

InDesign is still single-thread application - so GHz counts - not number of cores.

Can you show your design? Maybe instead of DataMerge - Text could be imported in a different way?

Sherry @ Lacy & Par

Known Participant

I can't show the design as it is a certificate with monetary value, but it's basically a card with a 20 character alphanumeric serial "number". There are 8 certificates per page, with each certificate having its own field. The certificate itself is not part of the merge. Each page contains just 8 serial numbers and header information containing text variables (file name, page numbers, that sort of thing)

I have been splitting the files to 1000 pages per set because it is a tiny bit faster and helps mitagate crashes so I only lose 1 of the 12 files created, not the entire thing; but it's still not ideal.

Not sure how text could be imported without a merge.

Sherry Baker

D

Dirk Becker

Legend

Network files and images were already mentioned.

Another trouble candidate would be fonts - e.g. if your machine is loaded with thousands of fonts, or if the document somewhere refers to missing fonts that have to get substituted. In such a case I've watched InDesign burning some (ehm) time with TypeKit nowadays known as Adobe Fonts even when the option was disabled in CC general control. Not sure whether that is still a problem, could also depend on the mood of the used servers.

You can also attempt to track down the offending subsystem. From Application >> Utilities launch the Activity Monitor app of OSX and target InDesign. Perform your export and somewhere into it where the lengthy operation is likely showing, sample the process. This should give at least some impression in what component (i.e. plug-in file) the time is spent. Like with crash dumps, upload the result to a pastebin service or alike and share the link.

E

Eugene Tyson

Community Expert

@tonyrush have you had any improvements or updates?

Robert at ID-Tasker

Legend

Like already suggested - if you link your files over network - move to local drive.

Memory - never enough 😉

Local drive - do you have SSD or HDD? With enough extra RAM you could create RamDisk - 100s times quicker than fastest SSD 😉

If your data is prepared perfectly - maybe you could do it overnight or during the day on a separate machine - batch processing with scripting?

Sherry @ Lacy & Par

Known Participant

I don't know how much more perfectly it could be set up when it's just a single address on every other sheet, but open to any and all suggestions. 🙂

I've done the overnight runs and have a script that will save the INDD files to PDFs in batches. The problem I'm having recently is with either the script crashing or the program crashing, mid-save.

Sherry Baker

James Gifford—NitroPress

Legend

Some background: We are a mail house and these mail pieces are barcoded and sorted, so I am afraid we really require the variable data features as each address doesn't take up the same amount of space. The background image of each post card is saved as a 2 page (front and back) master file. The variable data is set up as an overprint and the two files are "married" on the digital press.

What I don't understand is having done these as data merges with larger and even more complex jobs for years, how it's only been recently we have been running into problems. Our quantities haven't increased. We aren't doing VDP on graphics, just the text (even the barcode is a typeface) -- literally nothing has changed but the amount of time it takes to create the PDF.

40+ minutes to save a PDF that is only 5.2 mb in size seems a bit much, when I can impose 8 of these files into cutstacks, 4up with crops and color bars, in less than 5 minutes.

I get that it may be time to shell out some bucks for a more robust VDP solution; it's just that it is going to be a hard-sell for my employer when up until now the built in Data Merge feature has been working fine.

I just want to know what changed... because it's not our files.

40+ minutes to save a PDF that is only 5.2 mb in size seems a bit much

I don't think it has anything to do with saving the PDF, which from a static file should take 10 seconds to a minute at most.

It's in the data-shuffling for each page. Either ID has changed something to make this less efficent, your data or methods have changed in a way that slows things down, or there's a system problem. If you say the (same in all respects) process used to be faster, then you need to figure out what's changed. I'd put changes in ID low on that list — there haven't been any updates to DM in a very long time, AFAIK — and wonder if you have system, network or data-format issues that are making the process deadly slow.

Can you use some wholly different machine that runs ID well, and attempt this operation with entirely local components and data, to a local and reasonably fast destination drive?

Loic.Aigon

Legend

To second @Eugene Tyson PDF generation delays take the time it takes depending both the complexity of the file and the capacity of the machine. If there was a "magical" way to speed up the pdf generation, it's likely Adobe would have implemented it by default.

A possibility though is avoiding network ressources, if you have links on your network and a low bandwith, it may be that InDesign struggles to retrieve data from the network. if so, you may try to use local resources and it should reduce generation delays.

Once that said, you may try though scripting with an invisible document and no refresh on screen (see enableRedraw property of script preferences) or through InDesign Server but I don't think you will save that much.

A last resort would either be to sequence PDF exports (page ranges) or to use a Plugin such as DesignMerge from Meadows Publishing. They propose PDF/VT export options which can be helpful for large merging jobs.

FWIW

Loic

E

Eugene Tyson

Community Expert

It depends on the content of the data being created into a PDF.

A plain text file of 2 pages could take nanoseconds. A 1000 pages could take minutes.

A file with 2 pages of vector heavy content could take 10 minute. A file with 1000 pages of raster images could take the same time.

How is the merge taking place?
Are you creating an InDesign first from the Merge resulting in a 1000's of pages of an InDesign File?
Are you going directly to PDF from the Data Merge Panel?

Does it work faster if you split the merge in 2 or into 4 or into 6 or 10 different merges?

It's very hard to direct without any visuals or a clear workflow of what you're doing 🙂

Any more clues for us 😄

James Gifford—NitroPress

Legend

I don't know enough about the merge process, but is what you see in ID a finished merge, or is the merge operation repeated at the time of print or export? If ID is having to update or re/generate those pages on the fly, several minutes to export is to be expected.

Is it possible to save a merged document as a static one, export from that, and thus bypass that second merging/validation process?

Sherry @ Lacy & Par

Known Participant

I believe the result of a data merge in InDesign is always a static piece. There are no live merge options. It acts as though it was a normal InDesign document.

Sherry Baker

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded