JPEG and color management

[_Mark_J_Peterson_]

Hi. Is it impossible to get 100% identical colors (if we leave aside the jpeg compression artefacts) when saving a .jpeg file with Photoshop using the highest quality setting (12) ?

[_Mark_J_Peterson_]

I have this color chart (PNG file):



I open it with Photoshop CS3, and save it as JPEG with quality 12 and get this:

http://img184.imageshack.us/img184/4564/a3gencolorhighwwwfhoemdjv4.jpg

Most (but not all) of the colors are DIFFERENT to the picture above! The white pixels in this image denote different pixels:



How is this possible ?

[_Mark_J_Peterson_]

Due to jpeg compression, there are obviously differences around hard edges (such as the borders between the color patches or around the letters and numbers), but there are also a lot of entire color patches which have different colors.

I used sRGB IEC61966-2.1 both as my RGB working color space and as embedded profile in the files. So this can't account for the differences, isn't it?

As far as I know, Photoshop disables even color subsampling with quality 12. Also, color subsampling wouldn't account for differences in a whole color patch.

So the only conclusion I can draw from that, is that .jpeg files cannot use color management (but why does the jpeg standard then explicitely forsee and promote the use of embedded ICC profiles in jpeg files ???) or that Photoshop somehow has a problem with color management when jpeg files come into play. Any ideas ?

[Ramón_G_Castañeda]

Why are you saving it as a JPEG, a lossy format? For what possible purpose? <br /> <br /> Well, let's leave that aside, as it is really none of my business. :/ <br /> <br /> I downloaded the PNG straight from the browser to a folder (not my Desktop). I opened it with Photoshop 11, and saved one copy as a PSD, a TIFF, and a Quality-12 JPEG. I see nothing like what your image shows. Here's my JPEG on Pixentral: <br /> <br />c <a href="http://www.pixentral.com/show.php?picture=1vCQGy0rp4Cj826NOvgAtzKM40Z1" /></a> <img alt="Picture hosted by Pixentral" src="http://www.pixentral.com/hosted/1vCQGy0rp4Cj826NOvgAtzKM40Z1_thumb.jpg" border="0" /> <br />c Click on the thumbnail for the full image. <br /> <br /> Dragging the JPEG as a layer on top of the PNG, then setting the top layer to Difference blending mode, I get a solid black rectangle.

[Ramón_G_Castañeda]

When you opened the original PNG, did you ASSIGN the sRGB profile to it?

[Ramón_G_Castañeda]

>Dragging the JPEG as a layer on top of the PNG, then setting the top layer to Difference blending mode, I get a solid black rectangle.


Not only that, leaving the blending mode to Normal and toggling the visibility of the top layer off and on, I see no percetible difference in any of the colors.

[_Mark_J_Peterson_]

Thanks for looking into it Ramón.

> When you opened the original PNG, did you ASSIGN the sRGB profile to it?

yes.

> Dragging the JPEG as a layer on top of the PNG, then setting the top layer to Difference blending mode, I get a solid black rectangle.

That's exactly what I did. Now use the magic wand with these settings:
tolerance: 0
anti-alias: no
contiguous: no
sample all layers: yes

Fill the selection with white and you get the results I posted here. If your image is completely black (or white after selecting and filling), your images are really identical.

This would show, that there is either a problem with PS CS3, or with my settings somewhere.

[Ramón_G_Castañeda]

Now I just went through the same exercise with your own JPEG (a3gencolorhighwwwfhoemdjv4.jpg) with identical results as those described above. No difference.

How are you determining the difference you think you see?

[_Mark_J_Peterson_]

I compared the .jpeg file you uploaded to the original. I used an external command line program to compare the images, not in Photoshop in order to exclude the possibility of any wrong Photoshop settings on my behalf:

The result is:
Your .jpeg file is considerably different (so the black you were seeing, wasn't really black everywhere):
Compared to the original png file, 334103 of a total of 761000 pixels are different (=43.9%).

The different pixels are denoted in red in the following image:

http://www.imageboo.com/files/rjnce6kztdif5drtw75u.png

[large image replaced with link]

[_Mark_J_Peterson_]

> Now I just went through the same exercise with your own JPEG (a3gencolorhighwwwfhoemdjv4.jpg) with identical results as those described above. No difference.

Ramon, use the original png and the image your conversion to .jpeg as posted by you on pixentral. Did you use the magic wand as described above ?

[Ramón_G_Castañeda]

Mark, <br /> <br /> When using the Magic Wand <i>(which I confess I don't understand what that is supposed to prove)</i>, I used your settings, but I also went to the "Refine Edge" button and zeroed out all the values (Radius, Contrast, Smooth, Feather, Contract/Expand), and the selection preview was on <i>"Preview the selection over a black background"</i>. <br /> <br /> The result was a black rectangle with non-continuous marching-ants lines over the spots where the dividing lines between the colors as well as the numbers and letters would be. I expect that to be the JPEG artifacts. <br /> <br /> Because the selection is NOT continuous, this happens when I Fill with White: <br /> <br />c <a href="http://www.pixentral.com/show.php?picture=1mCX2jLqoBGgGrx3neLL8FvolXqo0Z0" /></a> <img alt="Picture hosted by Pixentral" src="http://www.pixentral.com/hosted/1mCX2jLqoBGgGrx3neLL8FvolXqo0Z0_thumb.jpg" border="0" /> <br />c (screenshot) <br /> <br /> Now what?

[Ramón_G_Castañeda]

Mark,

Mind you, I am a notorious JPEG hater and avoid them like the proverbial plague. I only use them to upload screen shots or other images to Pixentral to illustrate a point.

[Ramón_G_Castañeda]

Well, I went over much of the black rectangle at 3200% view and did see detect, by way of the Info Panel, some pixels with RGB values like 0,0,1 or 1,0.1 or 1,0,0.

Frankly, that's a hell of a lot better than I, as a JPEG hater, would have ever expected a damned JPEG to do, even at maximum quality. :D

[_Mark_J_Peterson_]

Thanks for helping in this test despite hating JPEGs

Your screenshots proves that there are differences not only along hard edges (i.e. the borders between color patches), but that also whole color patches are affected.

This would mean, that color management isn't working properly with jpeg files. I hope someone with sound color management knowledge can confirm this or otherwise explain the problem.

> When using the Magic Wand (which I confess I don't understand what that is supposed to prove)

I try to explain once again:

1) Download the original source file (png) from the link above
2) open it in Photoshop and assign sRGB profile
3) save it as jpeg with highest quality settings possible (12)
4) open the jpeg file
5) add it as new layer over the png file
6) select "difference" as blending mode in the layers palette
7) now the whole image should appear almost black
7) select the magic wand tool (with abovementioned settings!)
8] click somewhere into the formerly gray area

explanation: you just selected all completely black pixels (0,0,0) i.e. all pixels that are identical in both layers

9) you should see "marching ants" forming rectangular patterns
10) invert the selection (Shift+i)

explanation: the selection now covers all the other pixels, i.e. all pixels which are different between both layers

11) create a new empty layer and select it in the layers palette
11) set the foreground color to white
12) fill the selection with white (alt+backspace on Windows, accordingly on Mac)
13) set the blending modes of all layers back to normal

explanation: you now have the image I posted above, i.e. you see all identical pixels in their respective color and all different pixels in white


The expected results would be white pixels along the borders between color patches, resulting from jpeg compression, but if color management works correctly there must not be entire color patches with the wrong color (i.e. white here)

[_Mark_J_Peterson_]

> Frankly, that's a hell of a lot better than I, as a JPEG hater, would have ever expected a damned JPEG to do, even at maximum quality.

From my point ov view, jpeg artefacts (along hard edges, etc.) are expected to occur. But entire color patches with the wrong color must not occur if color management works correctly.

[Ramón_G_Castañeda]

Thanks for the extensive explanation, Mark.

I must admit that I still don't quite grasp your methodology, nor do I understand how all that translates into a color management issue rather than a JPEG compression weakness. However, it's certainly not really conducive to re-thinking my aversion to JPEGs. :D

[_Mark_J_Peterson_]

The bottom line is this: We opened a PNG file, saved it as JPG and compared both images in difference blending mode. In this mode all identical pixels are completely black (try it out with two copies of the same layer).

Our goal is to find out which pixels in the JPEG file differ from the PNG file (obviously there MUST be non-identical pixels, as JPEG is a lossy format!). As we know that identical pixels are completely black (as mentioned above), we want to find pixels that are NOT completely black.

How do we achieve this? With tolerance=0 the magic wand only selects pixel which have exactly the same color as the pixel we click on. So when clicking on a black pixel, the magic wand selects all the completely black pixels in the entire image (entire image because contiguous is set to yes). All completely black pixels are now selected. And this means we have selected all the pixels, that are identical on both layers.

Now we invert the selection, so that it covers the non-identical (=different) pixels. We fill this selection with white, so all different pixels are now white.

[end of bottom line]
----

I always thought, that JPEG compression artefacts (at least for maximum quality JPEG images) concern only edges and areas in an image with high local contrast.

But I wouldn't expect large areas with a single color have its color changed in the course of the JPEG compression. Because if colors would change like this, then what is color management good for in JPEG files and why can an ICC profile be embedded in a JPEG file at all in this case if colors change anyway ?

It's funny, because the same results that make you say you might rethink using JPEGs let me find JPEGs a whole lot worse than I thought them to be anyway ...

[_Mark_J_Peterson_]

Ramón, following the detailed steps above, were you able to reproduce the image I posted here ?
http://www.imageboo.com/files/27yo13jqgnjssir7vfqk.png

[Ramón_G_Castañeda]

Mark, <br />&gt;Ramón, following the detailed steps above, were you able to reproduce the image I posted here ? <br />http://www.imageboo.com/files/27yo13jqgnjssir7vfqk.png <br /> <br /> This is what I got: <br /> <br />c <a href="http://www.pixentral.com/show.php?picture=1kFuIZZvqsxOH7FrCGoLfVOnnyQU2" /></a> <img alt="Picture hosted by Pixentral" src="http://www.pixentral.com/hosted/1kFuIZZvqsxOH7FrCGoLfVOnnyQU2_thumb.jpg" border="0" /> <br /> <br /> For the sake of thoroughness, I'll confirm that I tried the very same exercise with a TIFF and a PSD saved from the same PNG, and after all going through all the steps with each, there were absolutely no different pixels.

[_Mark_J_Peterson_]

Okay. Thanks for posting your results. It's good to see that on CS4 (as you said that's what you work on), the results are the same. It's basically the same picture I posted above.

So obviously the question is: What can we see from this picture?
The white pixels in the image show pixels that have changed during the conversion to JPEG. Two sort of differences can be seen:

1) We can see jpeg compression artefacts in the form of random white pixels around the letters and numbers and around hard edges, which is typical for JPEG compression and was expected.

2) We can see entire color patches in white, meaning that the color of the whole patch has changed during the JPEG conversion. This was - at least for me - an unexpected result.

If this turns out to be true, than JPEG does not only suffer from the typical JPEG artefacts, but also from overall inaccurate color rendition.

I hope an expert here can confirm or correct these findings.

[_Mark_J_Peterson_]

Ramón, as I don't have CS4, could you maybe do me favour? Could you open this file http://img528.imageshack.us/img528/8885/fotoqb2.jpg in CS4 save it as BMP, zip it and upload the ZIP file somewhere (for example http://www.mediafire.com) ? I would be very thankful for that, Ramón!
And please make sure, that sRGB is used as color space when opening the jpeg file.

Here is a slightly related thread on this topic, if you are interested : http://www.adobeforums.com/webx/.59b726f1/0

[Ramón_G_Castañeda]

Rats! I just lost a post I was typing. Grrr.

As I was saying, :), if you run your cursor in the BLACK rectangle as per step #7 above, over the areas where we know now that the pixels aren't quite black, you'll see values with minimal deviations por pure black, as I noted above in post #12 ( values like 0,0,1 or 1,0.1 or 1,0,0).

I am surprised that the variations in color rendition are that insignificantly small.

In any event, I'm glad I don't need JPEGs myself.

I'll go download your image now and save it as BMP as requested.

[Ramón_G_Castañeda]

Yes, I'm using Photoshop 11 ("CS4").

Here's your BMP file saved in CS4:

http://www.mediafire.com/?iuojrmltj2t

[_Mark_J_Peterson_]

Thanks a bunch for your help with the BMP, that's really kind of you. I'll post the findings made with that BMP in the abovementioned thread and give you credit for that.

Awaiting expert comment ... [hopefully]

[Ramón_G_Castañeda]

Whoa!
>Awaiting expert comment ... [hopefully]

I see that Gernot Hoffmann and Chris Cox have already made comments on your thread in the Photoshop Windows forum. Just how much more expert would you expect any comments to be, Mark? :o