• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Why do INDD files bloat over time?

Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

Is there any simple reason that ID files slowly bloat over time, sometimes to absurd sizes? I know most "intelligent" doc formats keep adding app data like undo content, macros, etc., and Word is absolutely terrible at ever cleaning out or removing any of it. Cutting the content to a new file is often the only solution.

 

But I just had an experience with an INDD file that shocked me. Working on a long-time, episodic project (the book in the footer, actually), I started getting frustraing problems with EPUB export. Tried every fix including a wholesale cut and drop into a new doc. No go. So I did a laborious move of content to a new INDD file, which of course meant time rebuilding the TOC and some other details... but bingo. All of the quirky problems went away. Time well spent.

 

Then I compared the files. The new one is about 440k. The older one was a whopping 9.5M! What does ID do to keep adding junk to the basement when a document is edited and maintained over a long period of time?


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋
TOPICS
Performance

Views

1.8K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Correct answer

People's Champ , Feb 08, 2022 Feb 08, 2022

The extra stuff is the "undo" information, which is stored with the document (even though it's unusable as soon as the doc is closed).

Doing a "Save As" will clear out this bloat.

It's been like this forever AFAIK.

Ariel

Votes

Translate

Translate
People's Champ ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

The extra stuff is the "undo" information, which is stored with the document (even though it's unusable as soon as the doc is closed).

Doing a "Save As" will clear out this bloat.

It's been like this forever AFAIK.

Ariel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

I figured undo was a big part of it. However, I 'Save As' at regular intervals to archive development versions, and in trying to debug the glitches I also did an IDML export/open. I'm not sure if any of those fixed the file size but none fixed the weird glitches.

 

I'll keep an eye on Save As results next time.


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

Hi @James Gifford—NitroPress, There also can be a problem with placed images bringing Photoshop ancestor metadata into the document. If that’s happening the metadata has to be cleared out of the placed files.

 

Ancestor metadata bloat usually is more extreme than your example—see this thread which includes a metadata clearing script, in it I posted an example of a single page ID doc, with one flattened PSD link, saving at 256MB:

 

https://community.adobe.com/t5/indesign-discussions/is-there-a-script-to-clear-indesign-metadata-red...

 

 

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

I have a few images in this doc (actually just one, placed at several different scalings) (plus a few small logo files) but it's not large and it's not heavy on metadata. The same image is in the newly rebuilt file as well.

 

I just don't think I've ever seen a doc file do a ~20X bloat, not even the most corrupt, circulated Word doc. 🙂


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

but it's not large and it's not heavy on metadata

 

The bloated metadata is ancestor metadata coming from Photoshop and isn’t always obvious in the InDesign file. The problem you are experiencing might not be related, but there are a number threads in the Photoshop forum on the ancestor problem.

 

This 8-bit blank 1920 x1080 pixel canvas weighs 65MB—the problem metadata is buried in the Raw Data:

 

Screen Shot 3.png

 

When this file gets placed, the Raw Data can replicate itself in the ID file:

 

Screen Shot 5.png

 

Screen Shot 6.png

 

 

 

After running the ID script:

 

Screen Shot 7.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

Got all that; thanks for the detailed explanation.

 

However, does this apply to anything but placed PSDs? (Or perhaps TIFFs?) In a case such as mine, with a ~100k JPEG that's been exported to web standards from Photoshop, is there any place where this crypto data could be pulled into the ID document? As noted, the cleaned up document, with the same image (just a placeholder image of a jewel, used for scaling discussions) is ~400k. If there's not extra data in there, would subsequent edits or manipulation pull it in?

 

(All noted for when I do work with more demanding images at high res and cycled through PS manipulation.)


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

It can happen with any image format, but the only way to know if it’s affecting your document would be to run the ID script I posted:

 

https://assets.adobe.com/public/9a2cbd1d-bd8d-4fdc-5232-98128f54348b

 

In the example I’m showing above, I can delete the placed image files, save the blank page and the ID file size is still 453MB:

 

Screen Shot 9.png

 

Here’s a blog post from @Stephen_A_Marsh :

 

https://prepression.blogspot.com/2017/06/metadata-bloat-photoshopdocumentancestors.html

 

And a Photoshop thread from 2017:

 

https://community.adobe.com/t5/indesign-discussions/file-size-is-too-big/td-p/9370587/page/3

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

Besides the undo's and metadata, I wonder if some of the bloat is a "fast save" feature. Acrobat used to have this problem; in order to save faster, the software did not rewrite the file during save, but simply recorded the changes. 

 

An example I used to use in my classes was to have a PDF with 100 pages and deleting 99 of them. A proper save would leave just 1 page--a "fast save" remembers the 100-99 pages even though once can't access the deleted pages. (A proper save would be the equivalent of a save-as.)

David Creamer: Community Expert (ACI and ACE 1995-2023)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

IDML cleans up the document and rebuilds it, Save As doesn't, it just adds an extra layer of data. So if you want to clean up your file and start with a blank slate and avoid the build up of crud that can cause document corruption, Export your file as an IDML document.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

I haven't fully explored Bob's information above, but here's my results, from experimenting before I made this post:

  • Original file, with EPUB export issues: 9400k
  • IDML export: 430k
  • IDML save-as: 430k
  • Save-As: 4600k

 

None of the save-as'es fixed the glitchy issues, so I moved the content to a new file (in sections; a wholesale move didn't fix things), which started out in the ~500k range again.

 

After a day of fairly minor work, text editing mostly, it's back to 7500k. But without glitches (again) (yet).

 

Sigh.


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

If it is a case of metadata coming from links, a Save As to IDML doesn’t help much:

 

Screen Shot 18.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

Can you share a packaged document?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 10, 2022 Feb 10, 2022

Copy link to clipboard

Copied

It's proprietary.

However, I did the following experiment. On a copy, I deleted all the graphics (3 JPGs, one used several times). Nothing left in the Links panel even though I am sure there are no "forgotten" placed elements.

  • Export to IDML: 400k.
  • Save stripped file: 7500k
  • Open IDML, save as new INDD: 3500k.
  • Do some random editing (cut and paste a few paragraphs), save... 6500k.

 

If this has anything to do with images and metadata, it's being damned persistent about it. 🙂


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 11, 2022 Feb 11, 2022

Copy link to clipboard

Copied

Right, that is one of the symptoms of ancestor metadata, I posted an example above with all the content deleted weighing 453 Megabytes. Have you tried running the script? Run it on a packaged copy with the images.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 12, 2022 Feb 12, 2022

Copy link to clipboard

Copied

I ran the script. It appeared to reduce the INDD file from 7 to 4 MB. But a little trivial editing immediately expanded the file to 7 MB again.

 

The three images involved are 100k and two about 4-5k. They simply do not have any excess metadata in them, and I don't believe PS — under Windows, at least — stores historical data for work files separately.

 

This isn't a problem, per se; it's not causing me any workflow problems and I'm not concerned about it, other than that something (probably pointless) is going on under the hood to create these huge file expansions (from 0.5MB to 4MB to 7MB with essentially the same document file).

 

—


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 13, 2022 Feb 13, 2022

Copy link to clipboard

Copied

I was only suggesting the metadata bug was a possible cause—sorry the script didn’t work. The bug has been discussed and is listed in Adobe’s bug history—see January 2019 (version 20.0.2) release on the bug fix page:

https://helpx.adobe.com/photoshop/kb/fixed-issues-history.html

 

I don't believe PS — under Windows, at least — stores historical data for work files separately

Here‘s an example with 65MB of metadata—look at File Info>Raw Data—if I place this file 5x, the ID doc will bloat to over a Gigabyte:

https://shared-assets.adobe.com/link/7ea0589d-b86a-4fdf-65cb-e9add7cc119a

 

 

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 17, 2024 Feb 17, 2024

Copy link to clipboard

Copied

I'm two years late to the conversation and not techie enough to even ask for a solution. Just wanted to add to the conversation that I've had a possibly related problem with InDesign files exporting to incredibly bloated PDFs after replacing .jpg images with smaller files. The InDesign files were smaller but the PDFs were 5x larger - from 96mb to over 500mb. I did a Save As every time, sometimes more than one, and it still happened. The PDFs did not increase in size each time. They simply quintupled. I was trying to submit a high-quality PDF to KDP (Kindle Direct Publishing) and followed all their guidelines but KDP came up with error messages on the upload, or the upload succeeded but the file processing failed. The numerous bloated PDFs look fine. I finally went back to the 96mb PDF without all the replaced photos, even tho it would print a lot better if I could replace the photos. If anyone else has experienced this, maybe I could benefit for the next time.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 17, 2024 Feb 17, 2024

Copy link to clipboard

Copied

I'd say 90% of this has to do with your PDF export settings. Smaller files in ID don't necessary translate to reduced file sizes in PDF if you have high export PPI selected, and other things.

 

It's very much an end to end process.


┋┊ InDesign to Kindle (& EPUB): A Professional Guide, v3.1 ┊ (Amazon) ┊┋

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 18, 2024 Feb 18, 2024

Copy link to clipboard

Copied

Right. I believe settings were the same  - no downsampling and 300dpi. I didn't want to sacrifice any quality.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 18, 2024 Feb 18, 2024

Copy link to clipboard

Copied

Right. I believe settings were the same - no downsampling and 300dpi. I didn't want to sacrifice any quality.

 

Just to clarify your image export Compression settings—if you’ve turned off downsampling, you can’t specify an export resolution. With downsampling turned off, the images would export at their Effective Resolution, which could be considerably higher than 300ppi depending on how you have scaled the images.

 

For example this image has a starting Actual Resolution of 300ppi, but has been scaled down to 31%, so its Effective Resolution is 943ppi, and that‘s the resolution that will export when Do Not Downsample is selected:

 

Screen Shot 14.png

 

 

If I inspect the image in Acrobat I can see there is considerably more resolution (and file size) than needed:

 

Screen Shot 15.png

 

If you are scaling your images down, your Compression settings would account for the overly large PDF file sizes.

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 17, 2024 Feb 17, 2024

Copy link to clipboard

Copied

The PDFs did not increase in size each time. They simply quintupled.

 

Hi @Tana Reiff , You can check a PDF’s space usage in AcrobatPro via Tools>Optimize PDF>Advanced Optimization>Audit Space Usage. If you have a metadata problem a large percentage of space will be in Document Overhead. Try discarding user data and saving an optimized PDF:

 

Screen Shot 9.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 18, 2024 Feb 18, 2024

Copy link to clipboard

Copied

That is helpful - thanks.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 18, 2024 Feb 18, 2024

Copy link to clipboard

Copied

p.s. I just checked that. You've nailed it. I went through advanced optimization on a 504mb file and a 96mb file. The larger one had the replaces images (smaller than what they replaced). In both PDFs, 99% of the file size was images. I discarded user data and the percentage stayed the same but the final file size of the huge PDF went down to 96mb. No degradation of quality. Sure wish I had posted my question sooner but I'll know for the next time. Many thanks.

 

Just one more question. Do you know if there's a way to discard user data in InDesign before exporting to PDF?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Feb 18, 2024 Feb 18, 2024

Copy link to clipboard

Copied

From your captures It’s not clear to whether excess metadata is the problem, but you shouldn’t see such a large increase exporting the same document multiple times.

 

The link to the metadata script I posted above no longer works, here’s an updated link:

 

https://shared-assets.adobe.com/link/9da08cd8-3bbe-4ba2-6631-79dd8b5e46c0

 

 

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines