PSD file size grows exponentially

[David Cardillo, PRH]

I have a series of layered PSD files that are around 11-13 MB in size, that when I add sometimes a single image layer grow to 40-50 even over 100 MB in saved file size.

There's a few layers — no more than a half-dozen or so — that I'm merging into one, new layer, then turning that layer group off. As far as the image preview is concerned, it's the same image. There's really no reason for it to get that much larger. Doubled, maybe, but not quintupled or more.

The only thing I can think of is part of the processing I'm doing is, after merging the layer group into a new layer, converting the layer into a smart object, opening that smart object and converting the color profile, doing some more color adjustments, adding more layers — but then that smart object is then rasterized back into a flat layer. It's no longer a smart object, there's no history saved, no image preview that's any different. It's just one more art layer in a file with all the other layers turned off.

Converting the layer to a smart object, even though I'm then rasterizing it back into a flat layer is the only thing I can think of that's bloating the file size so much.

I've turned off the options for "Disable compression of PSD and PSB files" and set "Maximize PSD and PSB file compatibility to "Never." 

I've also tried deleting all the disabled (not visible) layers, leaving fewer layers than when I started, and then even saving them as TIFFs with LZW compression, and files that were 11MB to start are still 30-40MB.

I've been told that this is due to bloat in the PSD metadata, and the only way to clear it is to create new files and copy the layers from this one to the new one. Is there no other way to clear out all this extra data?

This is for a graphic novel, and with a couple hundred images, 4x the image data size is having an impact.

(Sorry, but I am unable to share the particular images in question. I'm asking if this is a known issue someone has seen before and knows how to deal with.)

Mac OS 12.6.1 Monterey on Intel, PS 24.1.0

[Stephen_A_Marsh]

---

@David Cardillo, PRH wrote:

I've been told that this is due to bloat in the PSD metadata, and the only way to clear it is to create new files and copy the layers from this one to the new one. Is there no other way to clear out all this extra data?

---

Unless you've confirmed the existence of photoshop:DocumentAncestors metadata and that it is excessive, there is no clear answer, this is just a theory. Just one possible cause.

Info on how to check for and remove this metadata here:

https://prepression.blogspot.com/2017/06/metadata-bloat-photoshopdocumentancestors.html?m=1

[PECourtejoie]

@Stephen_A_Marsh does it still occur, or is the issue dormant in older files ?

[Stephen_A_Marsh]

---

@PECourtejoie wrote:

@Stephen_A_Marsh does it still occur, or is the issue dormant in older files ?

---

I believe that the jury is still out...

Adobe announced that they fixed this issue in the Photoshop CC, January 2019 (version 20.0.2) release:

The raw metadata for some Photoshop files has excessive number of entries of 'photoshop:DocumentAncestors’:

https://helpx.adobe.com/au/photoshop/kb/fixed-issues-history.html#January2019version2002release

Cannot Display Raw Metadata, Contents Too Large:

https://helpx.adobe.com/photoshop/kb/file-info-freezes-cannot-display-raw-metadata.html

However, the issue hasn't gone away since 2019, could this just be legacy bloated data? I don't know.

I do know that at some point the File > File Info display was changed, but that is just sweeping dirt under the rug if looking for excessive ancestors metadata. I'm guessing that this was the fix to file info freezing.

[PECourtejoie]

@David Cardillo, PRH can you check if Stephen is right? File>File info. CMD+ALT+Shift+I 

[David Cardillo, PRH]

not seeing any excessive document bloat here:

[David Cardillo, PRH]

I did an exiftool text export of two images and the closes thing I could find for an explanation was "CXF" data:

CXF                             : (Binary data 75334 bytes, use -b option to extract)

after going over the data again, it appears this is just one line of the embedded ICC profile. The original had a "Generic CMYK" profile, the new file uses GRACoL 2013. The GRACoL profile is larger in total data, but isn't adding much in the way of file size. (about 3.5MB total, saved with vs without).

I'm attaching the exif data here:

[Stephen_A_Marsh]

OK, you can rule out bloated photoshop:DocumentAncestors metadata.

[David Cardillo, PRH]

Another update: this issue persists — not only persists, but if I merely open an image and save it to a new location, it almost doubles in size. The only change? I converted the color profile from SWOP to GRACoL.

One of my images didn't turn out right after I put it through a batch process (it was set up differently from the others) so I literally opened the original and re-saved it over the other file in the new destination folder. It went from 345MB to 647MB. Again, same file, opened, converted profile, and re-saved. Both have their profiles embedded. (Neither of the profiles is that large.)

I processed a folder full of images in a batch to a destination folder. My origin folder was 24GB, yet my destination folder was 42GB. Following the comment below here, I looked and yes, there was DocumentAncestors metadata. Following instructions at that link, I stripped it out. Less than 1MB difference per file. I checked the original files and they contained the same DocumentAncestors metadata, so that's not the cause.

These are pages from a graphic novel, with multiple text layers, panel frames, etc. The process I'm using is adding a single flat image, turning most of the other layers off. There's no reason for them to jump so greatly in size.

And again, for one of them I've just opened the original and re-saved it. It jumped from 300MB to over 600 from that alone. 

I've turned off Maximize Compatibility and enabled PSD compression.

[Stephen_A_Marsh · Accepted solution]

Do you have "dither" selected in the convert to profile or color settings?

[David Cardillo, PRH]

wow, so apparently that was it

I'd always been told to keep Dither checked, as it kept the most accurate color fidelity when converting (avoiding breaks or banding)

[Stephen_A_Marsh]

wow, so apparently that was it

I'd always been told to keep Dither checked, as it kept the most accurate color fidelity when converting (avoiding breaks or banding)

By @David Cardillo, PRH

There are trade-offs in all things! :]

You could do two conversions, one with and one without dithering, layer them. then mask the one with gradations and dither into the one without... Seems like a bit of work though!

[David Cardillo, PRH]

I'm going to assume is the reason this hasn't come up more often is due to the kind of art.

For photographic art, dithering is very likely unnoticeable.

These are illustrations, with mostly solid colors (figures, backgrounds, etc.). Adding digital noise to a section of solid color would certainly explain rising file sizes.

This is like the 3rd or 4th graphic novel I've worked on where I've had to hand back artwork that was double to upwards of 10 times(!) what I was given.

It looks like I need two different workflows.

[GreatSeal2]

I'm having the same problem. I teach Photoshop and have several students using a step by step processes with assignments that have remained the same for several semsters. No problems with these processes create strangely large files. The only change has been updates to the software. Computer setting in my lab haven't changed. All software is reinstalled from a common image every semester. Now, many of my students are having this problem. A simple 16 mg, 2 layer file with 1 clipping group saves out at 120mg. One student had a lrage complex file that showed a working size of 800mgs before saving. After the save it was 2.5 gigs. If reopen one of these large files, it reverts back to the smaller size. But as long as it simply remains on my ssd drive it takes up a ton of space - and takes far more time for me to download with my low band-width connection. This is not something that I or my students have changed. This is an issue with Photoshop. Does anyone have info from Adobe as what is really causing this problem?

[Stephen_A_Marsh]

@GreatSeal2 

Have you checked for excessive photoshop:DocumentAncestors metadata?

https://prepression.blogspot.com/2017/06/metadata-bloat-photoshopdocumentancestors.html?m=1

[GreatSeal2]

Nope, I'll check it out.

[Stephen_A_Marsh]

@GreatSeal2 

What pixel width and height, colour mode, bit depth, number of alpha channels and number of layers?

[GreatSeal2]

So I checked out the metadata on one of the too large files and it looks pretty normal.

[Stephen_A_Marsh]

So does it have ancestors metadata or not? And if it does, is it excessive? You can upload a cropped and redacted version for inspection.

[GreatSeal2]

Don't see any ancestors metadata. 

One example is 16.5 mbs when working and around 150mbs when the student uploaded it. I would expect working size to be in the 65 mbs range. 8X8 inch 300 ppi (2400 X 2400 pixels), RGB, no alpha channels, 6 total layers (4 images and two type layers with Layer Styles. This was a small file. Some of my students files were getting into the gigabyte range.

[Stephen_A_Marsh]

OK, it depends on file format options and 8/16/32 bpc as well as image content.

Your ~65 MB example sounds right for 8 BPC as you describe... But if there is excessive noise/fine detail, then it is possible to break the 2 GB limit when I tested with an artificial example working from a new document with no excess metadata.