Why are 16-bit files of all persuasions more than double the size of 8-bit files?

[Guy Burns]

Why are 16-bit files more than double the size of 8-bit? Normally I convert 16-bit scans to 8-bit to save space, but sometimes I want to keep the 16-bit images.

However, there is a huge penalty in file size. All tests below are on the same original image, converted from grayscale 16-bit to 8 bit:

8-bit

jpg2000… 8.9MB

Tiff … 11.5MB (zip)

16-bit

jpg2000… 30.9MB

PNG… 35MB

PSD… 43.7MB

Tiff… 38.6MB (zip)

Ques 1

I don't follow the maths. 16-bits requires exactly double the number of bits. Why 5 times the size in some cases?

Ques 2

Is there a lossless format that is optimised for 16-bit? i.e. if I decide to archive as 16-bit, that the file size is only double what an 8-bit file would be.

[D Fosse]

The answer is compression. You can't compare uncomressed with compressed.

The TIFF compression algorithms were written for 8 bit data and don't work well on 16 bit data. Often they will increase file size.

All things equal, a 16 bit file is exactly twice the size of an 8 bit file.

[JJMack]

Un-compressed an 16 bit images  is twice the size of an 8bit Image. The math is simple. An 8bit pixels color color can vary from 000000 to ffffff that is 16,777,216 colors.  16bit color image color fron 000000000000 to ffffffffffff that is 281,474,976,710,65 colors. So while the number of bits is double,  The complexity of the image is exponentially larger.  Data compression is used do reduce the number of bytes required the saved the pixel data.  Only Jpeg format uses a lossy compression.  Setting a low Jpeg quality cans vastly reduce the size a  Jpeg file will be.  Other format use compression that do not lose image quality.  How well data compresses depends on the complexity of the data. Not the number of bits.  

[Test Screen Name]

I suspect the reason is the randomness of the low 8 bits in the 16 bits. 

Consider how compression works - there are lots of ways but most look for repetition and/or closeness. As you move over a natural photo there are many areas of constant colour, or small change (+-1 +-2) in pixel values. But the remainder of the 16 bits will be pretty much random and unrepeated, so that part of the data may not compress at all, and may throw out the compression altogether.

[gener7]

Save as a Tiff and uncheck "Image Pyramid" (That saves multiple previews for other applications.)  Use LZH compression, it is lossless (doesn't degrade image quality). Let me know what results you get.

If you want to save as PSD, set Maximuze compatibility to "none". I understand that adds to the file size, and if you will stay with Photoshop, should not be a problem.

[Mylenium]

Test_Screen_Name is spot on - by representing data with higher fidelity you reduce compression efficiency because there is more "noise", i.e. slight differences from value to value that cannot be optimized away with simple methods like RLE/ LZW compression that only remove redundant info. They may not even produce visually different results, but technically they are there. As far as losless compression goes, you're pretty much limited to wavelet-based methods once you leave the 8bit plane. I've never done much testing since this isn't that relevant to me, but presumably wavelet-encoded TIFFs might be the best option. Just don't expect it to be a simple x times two thing. There is always overhead that may increase file size beyond doubling the byteword length.

Mylenium

[Guy Burns]

I forgot to mention one thing: all test files, including the jpg2000, were lossless.

Re the Image Pyramid suggestion, it is not an option in my PS.

The randomness/noise suggestion is most likely the answer. I have just converted the 8-bit tiff (originally from 16-bit, 38.6MB) back to 16-bit and it only increased from 11.5 to 16.5 MB. All those padded zeroes compress really well.

This tells me that converting from 16-bit to 8-bit is a good test of whether the original is really 16-bit.

[JJMack]

Think of noise as image detail.  The more details you have for and images the more complex the data you have for you image you have much more different details. If you image were not just converted from 8 Bit the 16 bit that  are worked on some will spread out data.  Converting from 8 bit to 16 bit does not change the image you still have the same image if you do not make adjustments.  However If you starter with RAW files and made two different conversion of the same raw file you would have two different images.  A Camera can only capture the colors in its color range and can Capture some value range of light intensity for its mosaic image.  I think most can capture values in the 10 to 14 bit range.   So If you convert there Raw data to and 8 Bit Image camera full 10 to 14 bit image  will be converted  into two different color image.  Both Images will have color that cover the camera full spectrum ranges of color.  However the  range or colors tones will be vastly different. The data complexity is much different. The 16 Bit image will have many more colors then the 8 Bit image 16 Million 8 bit, 28 trillion 16 Bit.

[D Fosse]

I don't think it tells you anything. Compression introduces too many unpredictable variables that you cannot possibly control. And layers can have a lot of overhead that is also totally unpredictable.

Remember that PSD is also compressed at default settings. There is a preference to disable PSD and PSB compression. There is no save options dialog as with TIFF.

In short, I think this is a bit like trying to find out how many angels you can fit on the head of a pin. There really is no fixed answer. If you need to reduce file size, reduce the properties and bit depth as much as you can, while deciding where the tradeoff is too expensive and you sacrifice too much data.

Let's be realistic. Raster image files will always take up a lot more disk space than other file types. You need to live with that, or you're in the wrong business.