Why do INDD files bloat over time?

Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

Is there any simple reason that ID files slowly bloat over time, sometimes to absurd sizes? I know most "intelligent" doc formats keep adding app data like undo content, macros, etc., and Word is absolutely terrible at ever cleaning out or removing any of it. Cutting the content to a new file is often the only solution.

 

But I just had an experience with an INDD file that shocked me. Working on a long-time, episodic project (the book in the footer, actually), I started getting frustraing problems with EPUB export. Tried every fix including a wholesale cut and drop into a new doc. No go. So I did a laborious move of content to a new INDD file, which of course meant time rebuilding the TOC and some other details... but bingo. All of the quirky problems went away. Time well spent.

 

Then I compared the files. The new one is about 440k. The older one was a whopping 9.5M! What does ID do to keep adding junk to the basement when a document is edited and maintained over a long period of time?

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)
TOPICS
Performance

Views

278

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Correct answer

Valorous Hero , Feb 08, 2022 Feb 08, 2022
The extra stuff is the "undo" information, which is stored with the document (even though it's unusable as soon as the doc is closed). Doing a "Save As" will clear out this bloat. It's been like this forever AFAIK. Ariel

Likes

Translate

Translate
Valorous Hero ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

The extra stuff is the "undo" information, which is stored with the document (even though it's unusable as soon as the doc is closed).

Doing a "Save As" will clear out this bloat.

It's been like this forever AFAIK.

Ariel

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

I figured undo was a big part of it. However, I 'Save As' at regular intervals to archive development versions, and in trying to debug the glitches I also did an IDML export/open. I'm not sure if any of those fixed the file size but none fixed the weird glitches.

 

I'll keep an eye on Save As results next time.

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

Hi @James Gifford—NitroPress, There also can be a problem with placed images bringing Photoshop ancestor metadata into the document. If that’s happening the metadata has to be cleared out of the placed files.

 

Ancestor metadata bloat usually is more extreme than your example—see this thread which includes a metadata clearing script, in it I posted an example of a single page ID doc, with one flattened PSD link, saving at 256MB:

 

https://community.adobe.com/t5/indesign-discussions/is-there-a-script-to-clear-indesign-metadata-red...

 

 

 

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

I have a few images in this doc (actually just one, placed at several different scalings) (plus a few small logo files) but it's not large and it's not heavy on metadata. The same image is in the newly rebuilt file as well.

 

I just don't think I've ever seen a doc file do a ~20X bloat, not even the most corrupt, circulated Word doc. 🙂

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

but it's not large and it's not heavy on metadata

 

The bloated metadata is ancestor metadata coming from Photoshop and isn’t always obvious in the InDesign file. The problem you are experiencing might not be related, but there are a number threads in the Photoshop forum on the ancestor problem.

 

This 8-bit blank 1920 x1080 pixel canvas weighs 65MB—the problem metadata is buried in the Raw Data:

 

Screen Shot 3.png

 

When this file gets placed, the Raw Data can replicate itself in the ID file:

 

Screen Shot 5.png

 

Screen Shot 6.png

 

 

 

After running the ID script:

 

Screen Shot 7.png

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

Got all that; thanks for the detailed explanation.

 

However, does this apply to anything but placed PSDs? (Or perhaps TIFFs?) In a case such as mine, with a ~100k JPEG that's been exported to web standards from Photoshop, is there any place where this crypto data could be pulled into the ID document? As noted, the cleaned up document, with the same image (just a placeholder image of a jewel, used for scaling discussions) is ~400k. If there's not extra data in there, would subsequent edits or manipulation pull it in?

 

(All noted for when I do work with more demanding images at high res and cycled through PS manipulation.)

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 08, 2022 Feb 08, 2022

Copy link to clipboard

Copied

It can happen with any image format, but the only way to know if it’s affecting your document would be to run the ID script I posted:

 

https://assets.adobe.com/public/9a2cbd1d-bd8d-4fdc-5232-98128f54348b

 

In the example I’m showing above, I can delete the placed image files, save the blank page and the ID file size is still 453MB:

 

Screen Shot 9.png

 

Here’s a blog post from @Stephen_A_Marsh :

 

https://prepression.blogspot.com/2017/06/metadata-bloat-photoshopdocumentancestors.html

 

And a Photoshop thread from 2017:

 

https://community.adobe.com/t5/indesign-discussions/file-size-is-too-big/td-p/9370587/page/3

 

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

Besides the undo's and metadata, I wonder if some of the bloat is a "fast save" feature. Acrobat used to have this problem; in order to save faster, the software did not rewrite the file during save, but simply recorded the changes. 

 

An example I used to use in my classes was to have a PDF with 100 pages and deleting 99 of them. A proper save would leave just 1 page--a "fast save" remembers the 100-99 pages even though once can't access the deleted pages. (A proper save would be the equivalent of a save-as.)

David Creamer
Adobe Certified Instructor, Adobe Certified Professional, and Adobe Certified Expert (since 1995)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

IDML cleans up the document and rebuilds it, Save As doesn't, it just adds an extra layer of data. So if you want to clean up your file and start with a blank slate and avoid the build up of crud that can cause document corruption, Export your file as an IDML document.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

I haven't fully explored Bob's information above, but here's my results, from experimenting before I made this post:

  • Original file, with EPUB export issues: 9400k
  • IDML export: 430k
  • IDML save-as: 430k
  • Save-As: 4600k

 

None of the save-as'es fixed the glitchy issues, so I moved the content to a new file (in sections; a wholesale move didn't fix things), which started out in the ~500k range again.

 

After a day of fairly minor work, text editing mostly, it's back to 7500k. But without glitches (again) (yet).

 

Sigh.

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

If it is a case of metadata coming from links, a Save As to IDML doesn’t help much:

 

Screen Shot 18.png

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 09, 2022 Feb 09, 2022

Copy link to clipboard

Copied

Can you share a packaged document?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 10, 2022 Feb 10, 2022

Copy link to clipboard

Copied

It's proprietary.

However, I did the following experiment. On a copy, I deleted all the graphics (3 JPGs, one used several times). Nothing left in the Links panel even though I am sure there are no "forgotten" placed elements.

  • Export to IDML: 400k.
  • Save stripped file: 7500k
  • Open IDML, save as new INDD: 3500k.
  • Do some random editing (cut and paste a few paragraphs), save... 6500k.

 

If this has anything to do with images and metadata, it's being damned persistent about it. 🙂

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 11, 2022 Feb 11, 2022

Copy link to clipboard

Copied

Right, that is one of the symptoms of ancestor metadata, I posted an example above with all the content deleted weighing 453 Megabytes. Have you tried running the script? Run it on a packaged copy with the images.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 12, 2022 Feb 12, 2022

Copy link to clipboard

Copied

I ran the script. It appeared to reduce the INDD file from 7 to 4 MB. But a little trivial editing immediately expanded the file to 7 MB again.

 

The three images involved are 100k and two about 4-5k. They simply do not have any excess metadata in them, and I don't believe PS — under Windows, at least — stores historical data for work files separately.

 

This isn't a problem, per se; it's not causing me any workflow problems and I'm not concerned about it, other than that something (probably pointless) is going on under the hood to create these huge file expansions (from 0.5MB to 4MB to 7MB with essentially the same document file).

 

|| Word & InDesign to Kindle (& EPUB): a Professional Guide (v2 now on Amazon!)

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Feb 13, 2022 Feb 13, 2022

Copy link to clipboard

Copied

LATEST

I was only suggesting the metadata bug was a possible cause—sorry the script didn’t work. The bug has been discussed and is listed in Adobe’s bug history—see January 2019 (version 20.0.2) release on the bug fix page:

https://helpx.adobe.com/photoshop/kb/fixed-issues-history.html

 

I don't believe PS — under Windows, at least — stores historical data for work files separately

Here‘s an example with 65MB of metadata—look at File Info>Raw Data—if I place this file 5x, the ID doc will bloat to over a Gigabyte:

https://shared-assets.adobe.com/link/7ea0589d-b86a-4fdf-65cb-e9add7cc119a

 

 

 

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines