PSD Format - how to read compressed data from Image Data Section?

[Anatoly_Paraev]

Has anybody tried to read ZIP-compressed data from the Image Data section in PSD file format? I tried to read data compressed with predication as well as without prediction, but had no success - the inflating function returns no error, but the image is corrupt. I used different versions of ZLib to decompress the data - none of them worked. If you have any experience in this area and are ready to help, it will be much appreciated!
Thank you in advance,
Anatoly

[qu1j0t3]

Which version of the format are you talking about? For v6 (PackBits) this code works. Does any documentation exist for versions beyond v6?

[_videohead_]

Documentation for versions beyond 6 does exist, but is not public. Version 6 however supports ZIP as well as packbits, the latter of course is significantly easier to implement. I know because I've written one.

Anatoly, are you sure photoshop uses exactly the same compression/decompression algorithm as ZLib? I wonder if Chris can confirm this. And what do you mean by "the image is corrupt"?

[qu1j0t3]

"Not public"? That's non-existent as far as 3rd party developers are concerned. There was no sign of it in CS2, CS3 beta or "Advanced" CS2 SDKs.

I've never seen a PSD (even from CS2) that used anything other than PackBits.

[Anatoly_Paraev]

Hi,
Thank you for your responses.
Toby, I saw your code, and RLE decompression works just perfectly, thanks a lot!
ZIP compression seems to be used in some 16bit files (though not in all of them).
I am not sure that Photoshop uses the same modification of ZIP as ZLib it was just an assumption. When I applied ZLib decompression, I got a correct size of the uncompressed data, and I was able to see the outlines of the images that had been compressed, but all the colors were messed up.
Does anybody know which implementation of ZIP I should use to decompress data from a .PSD file?

[_videohead_]

Toby, are you saying that ZIP compression has been dropped from CS? I don't have Photoshop CS, but I don't think that true. There is another PSD format/documentation that Adobe circulates but only to licensed/cleared/paid-up developers complete with NDA etc.

Anatoly, have you considered writing your own decompression module? I suspect that all the information you need to do so will be in the version 6 spec, else it wont be much of a spec. That said, I had to go outside the spec for detailed information on packbits compression. Else, try other libraries.

[Anatoly_Paraev]

Unfortunately, the specification that I have (that one which came with Advanced SDK) doesn't say anything about ZIP - it just says that it is ZIP, and that's all. Probably I have to try to apply different ZIP implementations...

[_Aandi_Inston_]

Bear in mind zlib offers two different decompression/compression
methods, which use the same kind of compression but provide different
wrappers. You don't want the one used to make gzip files.

Aandi Inston

[_videohead_]

Nice idea. Do keep us informed about which implementation works.

[qu1j0t3]

videohead, I've signed the NDA-like agreement for the "Advanced" SDK - and 9 months later I received some snippets of source code for Export and File Format plugins which merely confirmed things I'd already figured out.

No additional documentation for PSD/PSB was provided. It seems there is no "kit" as such, they just send material ad-hoc according to what they feel like passing on. My reply emails asking for further information have been ignored.

That said, I have no information about PSD's use of zlib/ZIP, and nor do I know how to get it, if it exists as you say. But that's not the only undocumented part of PSD, of course...

[_videohead_]

Is this same SDK you pay > $100 for, or the free one?

[_xbytor_]

videohead{at_char_here}mail.com wrote:
> Is this same SDK you pay > $100 for, or the free one?

Both the SKDs are free now. You just have to go through the website and request
them. I don't have the URLs handy.

-X
--
for photoshop scripting solutions of all sorts
contact: xbytor@gmail.com

[qu1j0t3]

I've never seen them distinguished by Adobe, so I don't know what you mean. Is there some supa-sikrit SDK you only see if you're in the Partner programme? Which I now am...

[_videohead_]

Do you mean the Photoshop Format is now distributed free of charge? Does this SDK contain the photshop format info?

[qu1j0t3]

Neither the free CS2 SDK nor "Advanced" SDK - at least the bits I received - contain any information on file formats whatsoever. This is rather less useful than, for instance, the v6 SDK everyone received on CD.

As I said above, I don't know if there's an "Really Really Advanced (We Were Just Kidding About That Other One)" SDK that costs money that might have this info, and nor would I know how to get it, since Adobe personnel have not responded to any probing on such questions.

Naturally I welcome correction on these points by any whose jousts with Adobe have had better results.

[_Aandi_Inston_]

I don't speak for Adobe, but I would guess that Adobe have decided
there is no commercial advantage in allowing third parties to read and
write PSD files, and have therefore stopped updating the
information...

Aandi Inston

[_xbytor_]

The Advanced SDK just adds a couple of things. One of them is a pdf:
"Photoshop CS File Formats Specification."

This does document the PSD file format along with other file formats (.atn,
.aco, etc...). I don't know how current the document is, but for the file
formats that I've been interested in (not PSD, BTW) it seems to be fairly accurate.

-X
--
for photoshop scripting solutions of all sorts
contact: xbytor@gmail.com

[qu1j0t3]

OK, mea culpa, it's possible I just didn't request it. If I didn't, the reason is simple: I didn't want my related GPL work to be poisoned by the onerous agreement.

[_xbytor_]

Toby_Thain@adobeforums.com wrote:
> OK, mea culpa, it's possible I just didn't request it. If I didn't, the reason is simple: I didn't want my related GPL work to be poisoned by the onerous agreement.

All of my work is LGPL. I don't recall there being any problem with any of the
agreements (e.g. no viral licenses like Microsoft's), but you should probably
read over it yourself or, better yet, have an attorney do it.

-X
--
for photoshop scripting solutions of all sorts
contact: xbytor@gmail.com

[qu1j0t3]

Of course I read it carefully myself (all 14 pages). And since I could not satisfy myself that it was safe to write GPL plugins under the terms of that agreement, I discussed it further with always-helpful Mark Niemann-Ross, who gave his verbal opinion (as he has done on his blog) that it would not be a problem. But that discussion was more about the plugin API.

As you say, it's only prudent to get an attorney's opinion before taking the risk, when it comes to PSD docs. It seems safe to assume that Adobe is motivated to enthusiastically prosecute any leakage from controlled documents - as vaguely defined as such leakage may be - which must surely chill any GPL code that works with post-v6 PSD. But maybe I'm the only person on these forums who cares about the GPL (does anyone else actually write GPL plugins?)

[_xbytor_]

Toby_Thain@adobeforums.com wrote:
> But maybe I'm the only person on these forums who cares about the GPL (does anyone else actually write GPL plugins? )

One thing you should consider, then, is talking with the authors of other Apps
that read PSD fil...

[qu1j0t3]

Well, I'm happy with what I've done based on v6 PSD doc - it reads rasters from all subsequent versions, which is why (to get back to topic) I'm surprised if somebody found anything other than PackBits on rasters.

[_videohead_]

Toby, PSD supports ZIP as well, the docs say so. Have you tried reading a ZIP compressed image with your packbits-based program? Did it work?

[qu1j0t3]

No, it won't work (obviously) - psdparse reads only uncompressed data and the single compression type defined in v6: PackBits (table 2-16). I cannot find a reference to ZIP in the File Formats document and I've never seen it in the wild, not even from CS2 - so how would one actually create one?? Perhaps the OP knows.

Here's a run on a CS2-created PSD; note compression type:

$ ./psdparse ~/Desktop/40843316_2534716d2d_o.psd -v

"/Users/toby/Desktop/40843316_2534716d2d_o.psd"

  channels = 3, rows = 1024, cols = 768, depth = 8, mode = 3 (RGBColor)

  (color mode data is empty)



Image resources (31334 bytes):

  resource '8BIM' ( 1061,""):   16 bytes

  resource '8BIM' ( 1060,""):16583 bytes

  resource '8BIM' ( 1002,""): 7600 bytes

  resource '8BIM' ( 1001,""):  120 bytes [Macintosh print record]

  resource '8BIM' ( 1005,""):   16 bytes [ResolutionInfo]

  resource '8BIM' ( 1062,""):   14 bytes

  resource '8BIM' ( 1037,""):    4 bytes [Global Angle]

  resource '8BIM' ( 1049,""):    4 bytes [Global Altitude]

  resource '8BIM' ( 1011,""):    9 bytes [Print flags]

  resource '8BIM' ( 1034,""):    1 bytes [Copyright flag]

  resource '8BIM' (10000,""):   10 bytes [Print flags info]

  resource '8BIM' ( 1013,""):   72 bytes [Color halftoning info]

  resource '8BIM' ( 1016,""):  112 bytes [Color transfer functions]

  resource '8BIM' ( 1032,""):   16 bytes [Grid and guides info]

  resource '8BIM' ( 1054,""):    4 bytes [URL List]

  resource '8BIM' ( 1050,""):  863 bytes [Slices]

  resource '8BIM' ( 1064,""):   12 bytes

  resource '8BIM' ( 1041,""):    1 bytes [ICC Untagged]

  resource '8BIM' ( 1044,""):    4 bytes [Document specific IDs]

  resource '8BIM' ( 1036,""): 4103 bytes [Thumbnail resource]

  resource '8BIM' ( 1057,""):   85 bytes [Version Info]

  resource '8BIM' ( 1058,""): 1094 bytes

  resource '8BIM' ( 4000,""):  268 bytes

  resource '8BIM' ( 4001,""):   28 bytes

  (layer info section is empty)

  (global layer mask info is empty)



  merged channels:

>>> dochannel 0/3 filepos=  31380

    compression = 1 (RLE)

    uncompressed size 2359296 bytes (row bytes = 768)



    channel 0 (@   37526):

       0: <  360> 05fafbfcfcfbfbf8fcfffdfdfcfffdf9fc08fdfcfdfefefdfd ...335 more

       1: <  378> fffbfdfcfffd00fbfefc00fdfbfc01fbfcfdfdfffcf9fd00fe ...353 more

       2: <  361> fffbfdfc07fdfcfcfdfcfcfdfdfafcfefd00fef7fdfefeffff ...336 more

    ...1018 rows not shown...

    1021: <  728> 7b3437373b3c383c3c41464949484e525453544c4445224161 ...703 more

    1022: <  727> ff3c5d3b3c342c2c26343b4144434b4d4f4d4a3c3b3d21535e ...702 more

    1023: <  728> 0036fe37642d2a2e23252c3336333734362e2219293532645a ...703 more



    channel 1 (@  559903):

       0: <  362> 05fafbfcfcfbfbf8fcfffdfdfcfffdf9fc08fdfcfdfefefdfd ...337 more

       1: <  373> fffbfdfcfffd00fbfefc00fdfbfc01fbfcfdfdfffcf9fd00fe ...348 more

       2: <  348> fffbfdfc07fdfcfcfdfcfcfdfdfafcfefd00fef7fdfefeffff ...323 more

    ...1018 rows not shown...

    1021: <  723> 7f6b72777f828085888d92989a999fa1a1a2a39b97987394ba ...698 more

    1022: <  731> 1e656b717876747874778088908f939292959084888c73abbe ...706 more

    1023: <  734> 7f5f666d736f727a7168717a827f7f7979766861768484bcba ...709 more



    channel 2 (@ 1086844):

       0: <  379> 05fafbfcfcfbfbf8fcfffdfdfcfffdf9fc08fdfcfdfefefdfd ...354 more

       1: <  388> fffbfdfcfffd00fbfefc00fdfbfc01fbfcfdfdfffcf9fd00fe ...363 more

       2: <  362> fffbfdfc07fdfcfcfdfcfcfdfdfafcfefd00fef7fdfefeffff ...337 more

    ...1018 rows not shown...

    1021: <  730> 5b707a818a8e8e94989da2a9abaab0b0b1b3b2aaa9aa84a4cc ...705 more

    1022: <  728> 0c6b737b838280858187909aa09ffea26fa49c92989d81b9ce ...703 more

    1023: <  729> 7f656e777e7b7e877e78818c928f8e898985746f869592caca ...704 more

  done.

[_videohead_]

Try saving a document in Photoshop, and take a look at the compression options. You should find a ZIP option in there.

Re the ZIP in the file fomat/spec (version 6): take a look at pages 21, 24, 43.