• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
1

The ProPhoto cyan shadow banding bug

Community Expert ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

Since Noel is back with us, I thought this header might catch his attention. He was the one who first reported it six or seven years ago.

The reason I'm bringing this up now, is that someone in the Lightroom forum had this problem just yesterday (he was sending ProPhoto files to Photoshop and couldn't understand the cyan cast in his dark grays). There's no doubt it's the same issue.

This made me dig up my old ProPhoto test gradient file. I don't use ProPhoto all that much, so I haven't really looked for it in a while. To recap, this is an example of what it could look like (this is an old screenshot that I've kept). Open and view against a darkish background:

test-PP_screenshot.png

But when I opened the test gradient now, there was no trace of it, zero. The gradient is smooth as silk. That surprised me, because while the effect can vary, it always seemed to be there in some form.

Technically, this is an inaccuracy in the conversion from ProPhoto into the monitor profile. This conversion is performed by OpenGL in the GPU when you have the PS preference set to Normal and Advanced modes. In Basic mode it's shifted back to the CPU, which is more accurate so the problem disappears. And you only see it in ProPhoto files because ProPhoto is very compressed in the shadow values compared to other color spaces. The stratospheric gamut has a price, and that's it. With this compression, inaccuracies get amplified.

So what has changed on my system over the last 12 to 24 months? Not much. I still use Eizo Colornavigator as I did then, still producing similar matrix monitor profiles. But one thing has changed - going from GeForce to Quadro, and with that, 10 bits per ch output.

So Noel, if you're reading this - I understand you have Quadro GPU too, and running displays at 10 bit. Do you still see it?

Anyone else?

Views

6.1K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe
Community Expert ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

That "use native os gpu acceleration"-checkbox is grayed out here too. It has to refer to the Metal framework in MacOS. I'd forgotten all about that - isn't that supposed to replace OpenGL in MacOS?

In that case the plot thickens, because then it isn't OpenGL either. You get the cyan banding with Metal too!

So what are we left with? The monitor profile? I use Colornavigator profiling software, but I've done that for many years. Todd uses NEC Spectraview with a PA272, right?

Display system bit depth? No. Noel has a 30-bit system too.

Huh.

WLan, we once got an engineer to promise to look into this, but that was the last we heard (Chris Cox, who later left Adobe).

Oh, and BTW, I've double-checked both on my home and work systems, no difference, but I didn't expect that since they are identically configured in all important aspects.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

https://forums.adobe.com/people/D+Fosse  wrote

So what are we left with? The monitor profile? I use Colornavigator profiling software, but I've done that for many years. Todd uses NEC Spectraview with a PA272, right?

Display system bit depth? No. Noel has a 30-bit system too.

Yes, NEC Spectraview with a PA272, but no 30 bit display mode. I had an issue with DisplayPort not waking sometimes on my Windows 7 system and 30 bit display got broken on one of the updates so I gave up.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

Hi

After restarting each time the revised results in here. This is on an NVidia GTX 1080 OC with driver 416.34

Results:

Prophoto 16 bit - Basic - No (or at least barely visible) banding
Prophoto 16 bit - Normal - Banding

ProPhoto 16 bit - Advanced - Banding

Adobe 1998 -16 bit - Basic - No (or at least barely visible) banding
Adobe 1998 -16 bit - Normal - No (or at least barely visible) banding
Adobe 1998 - 16 bit - Advanced - No (or at least barely visible) banding

My monitors are not 10 bit - hence the "barely visible" comment

Apologies for the false steer earlier

Dave

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

Yep, that's the expected result, Dave.

Interesting problem, huh? What on earth could cause this?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 23, 2018 Oct 23, 2018

Copy link to clipboard

Copied

My inner geek voice has always whispered a suspicion that there must be a stage in the  transform calculation in which, even though GPUs use floating point math, a number near an integer may be subtracted from a nearby integer (e.g., 1 - 0.9999999), which is a sure way to cause precision loss.  Another thought is that maybe there are tone curves that differ by a very small amount for each of the colors.  An amount some profile designer thought was inconsequential 20 years ago.

It is possible the monitor profile is involved, though I have verified seeing the issue using anything from the age-old sRGB v2 profile to newer v4 display profiles as a display profile. 

It's clearly got to do with using the ProPhoto profile as a working space, because another, similar profile (e.g., ColorMatch RGB) doesn't cause it - which led me to wonder further...  Could there now be more than one ProPhoto RGB profile around?  Could you have somehow gotten an updated one, Dag, that no longer causes the precision loss?

I looked on my hard drive and I saw that I actually have accumulated multiple copies of the ProPhoto profile (not surprising), BUT...  I do have two different renditions.  Most are ProPhoto.icm (including the one in C:\Windows\System32\spool\drivers\color, which is the one that matters), dated anywhere from 2005 to 2015.  the one that's different is ProPhoto.icc (note the different file extension) dated 2012, and located in a folder created by a different photo editor (ComputerInsel Photoline).  Turns out it's described as "ProPhoto RGB (linear)" and it isn't another rendition of the same profile, exactly, but its existence DOES support the possibility of there being alternate versions of ProPhoto.icm running around in the world.


Dag, is your copy of C:\Windows\System32\spool\drivers\color\ProPhoto.icm the same size as mine?

ScreenGrab_NoelC4_2018_10_23_194830.png

If so, next we can devise a way to see if they contain the exact same binary data.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

Yes, down to the last byte (940/4096).

But even if the gradient looks fine, there's something funny going on all right. I drew a marquee and cranked saturation all the way up. Why is it consistently red when it should just be neutral with increased color noise? What's that cyan peak in the histogram?

PP_cyan_1.png

Yeah, it could be the PP profile spec in itself. Now, I'm a practical man. I know how things work, but whenever someone tries to drag me into the Hogwarts Color Science Lab, I'm off chasing butterflies. I know nothing about the internal anatomy of a profile - except the broad principles of Profile Connection Spaces. I know there's a table somewhere (way up in the Scottish Highlands), which can be based on either XYZ or Lab. Could this be where the dragon is buried?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

I have both an .icm an .icc ProPhoto RGB profile on my Windows 7 system, but in PS Color Settings I only see one ProPhoto RGB selection. They appear to be identical so it shouldn't matter. The date showing in ICC Profile Inspector is 1998/12/1.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

Hm, that might imply your gradient isn't pure gray, Dag, BUT...  There is definitely some additional weirdness going on!

I tried raising the saturation of my grayscale PNG to +100 and got this:

OneSaturationIncrease.png

Then I did it AGAIN and got this cute little result. 

TwoSaturationIncreases.png

After undoing all the above, I did Image > Adjust > Desaturate it, then two iterations of +100 saturations and got this:

DesaturatedFirst.png

This implies the PNG actually has some values that are non-gray (by a tiny bit).  And here I thought PNG compression was lossless - but I guess not!  The funny thing is that this little revelation seems to be entirely separate from the ProPhoto cyan cast display bug.

The plot thickens!  I'm going to save my test image as a PSD and use that from now on.  I hope THAT's lossless.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

This is OSX High Sierra with CC2018. I'm not seeing any objectionable banding or color shifts

Screen Shot 6.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

It's great to see that it CAN work.  Now to figure out what's common among the systems on which it IS working, and that's proving non-trivial.

Rob, some questions, just to be thorough:

1.  Do you have your Use Graphics Processor setting checked, and are you using either Normal or Advanced graphics mode?

2.  Did you try two iterations of Saturation +100 on that .png I posted?

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

These are my Performance settings

Screen Shot 7.png

Multiple applications of the +100 Saturation makes no difference—the RGB values remain equal, which is expected.

If you add Color Sample marks to your Saturation selection do the RGB values change? Saturation shouldn't have any affect on equal RGB values. Are you testing in 2019?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

I tested above in 2019, but I've seen the ProPhoto cyan cast issue before in 2018 and earlier versions, going all the way back.  Apparently it's appearance is dependent on something we've yet to nail down.

The second issue, which I just uncovered above, may be my fault.  I re-opened the PNG file I posted above and I haven't been able to reproduce bringing out any weird colors with any number of Saturation steps.  I may have opened a different test file from my History to get the effect with the neat patterns.

Here are reproductions of the ProPhoto cyan cast problem, with settings like yours...

Photoshop CC 2018 "girl in drug cloud splash screen edition":

PhotoshopCC2018ProPhotoRendition.png

Photoshop CC 2019 "cloud headed dude splash screen edition":

PhotoshopCC2019ProPhotoRendition.png

As you can see, even with the same settings you're using (e.g., no 30 bit in my case) the color banding remains.

The samples all show equal R, G, and B values, whether or not point samples or averaged samples are made.  It must be purely a monitor display issue.

The cyan banding issue doesn't show up in the Navigator display, by the way.  I believe Photoshop must always use CPU color-management for that.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

It must be purely a monitor display issue.

Or an OS issue?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

No. Could be on Mac; not on Windows. Unlike MacOS, Windows doesn't touch this stuff. It's strictly application.

(EDIT: what made me pick this up again now, was a Mac user on the Lightroom forum three days ago having this exact issue. He's on Mojave and PS 20.0)

Anyway, Noel, you were right of course. Once I punched in the numbers and double-checked with 16 bit readout that R=G=B, I got this, and that's 100+ saturation on the actual finished screenshot. A few stray pixels, that's all:

grad+sat_2.png

On the upside, this is as close to perfection as it'll ever get. So yes, it can work.

It has to be somewhere in the conversion from ProPhoto source to monitor RGB.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Mentor ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

https://forums.adobe.com/people/Noel+Carboni  wrote

Hm, that might imply your gradient isn't pure gray, Dag, BUT...  There is definitely some additional weirdness going on!

I tried raising the saturation of my grayscale PNG to +100 and got this:

Then I did it AGAIN and got this cute little result. 

After undoing all the above, I did Image > Adjust > Desaturate it, then two iterations of +100 saturations and got this:

This implies the PNG actually has some values that are non-gray (by a tiny bit).  And here I thought PNG compression was lossless - but I guess not!  The funny thing is that this little revelation seems to be entirely separate from the ProPhoto cyan cast display bug.

The plot thickens!  I'm going to save my test image as a PSD and use that from now on.  I hope THAT's lossless.

-Noel

I opened your file in PhotoLine to check if the ProPhoto ICC profiled PNG file has any issues, and increased the saturation 6 times. Result: it stays neutral, no colour changes at all.

As far as I can tell, there is no issue with the PNG file itself. There don't seem to be any outlier values.

I did a second test to check if OpenGL might be the problem, and opened the file in Krita (which also uses OpenGL for its viewport). I applied the same saturation test, and no issues were encountered. (I run an Nvidia 1080GTX.)

Which leads me to believe that neither the PNG file, nor OpenGL are the problem here (at least, Nvidia's OpenGL on Windows).

PS A gut feeling tells me it might (just might) have something to do with Photoshop's (2^15)+1 internal representation in 16bit mode and calculation inaccuracies. Noel Carboni expressed similar thoughts a few years ago in this thread:

OpenGL inaccuracies and black levels

But only a Photoshop developer might know. Chris Cox would have known 😉

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 24, 2018 Oct 24, 2018

Copy link to clipboard

Copied

Yeah, I think that weird blotchy color thing was me opening an intermediate file I had created in a different folder on the way to making that proper grayscale PNG.  Sorry for muddying the water with that.

The cyan banding problem with previews of images expressed in the ProPhoto RGB color space is still a real issue, however.

-Noel

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

Rayek, don't forget that this has never been an issue when Photoshop handles the ProPhoto > Monitor conversion natively, in Basic mode.

It only happens when the GPU takes over that particular function, in Normal and Advanced modes.

What could possibly be an issue, however, is if there is an inconsistency in how Photoshop represents data on one side, and how the GPU represents it on the other (or how it expects it to be represented).

Either way, this has to be in the simple chain ProPhoto > converted into > monitor profile. One of those three, or a combination.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

Interesting though that the GPU handles a similar conversion from Adobe RGB without issue, where the same 15+1 bit and floating point math would apply.

Dave

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

Yes, but ProPhoto will naturally be more susceptible to small inaccuracies. That stratospherically huge gamut goes into the same numerical range.

There's no free lunch, everything has a price. Going from sRGB to Adobe RGB is an incremental gamut increase - mainly extending the green primary - but going to ProPhoto is jumping off the edge of the world. It's huge.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

but going to ProPhoto is jumping off the edge of the world. It's huge.

When it comes to CMYK print output I can see some advantage of the larger ProPhoto gamut, but I'm having a hard time finding the downside.

Here's the shadow gradient we've been looking at, which is obviously in gamut with both ProPhoto and AdobeRGB, and is also inside any CMYK gamut. The three color swatches are inside the ProPhoto and GRACol Coated CMYK gamut, but are out of the AdobeRGB gamut. The bottom capture is a Relative Colorimetric conversion of the Prophoto file to AdobeRGB.

Screen Shot 11.png

If I convert both to GRACol I get identical output numbers in the shadow gradient, but the numbers for the saturated colors are different—the AdobeRGB colors have been somewhat clipped.

Screen Shot 12.pngScreen Shot 13.png

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

https://forums.adobe.com/people/rob+day  wrote

but going to ProPhoto is jumping off the edge of the world. It's huge.

When it comes to CMYK print output I can see some advantage of the larger ProPhoto gamut, but I'm having a hard time finding the downside.

I do a lot of scannerless color negative film capture using a DSLR copier. I open the raw files in LR and apply white balance and exposure adjustment to center image data in the histogram. Using Edit in PS I then use an action that inverts the image and applies an auto curves adjustment (Find Dark & Light Colors and Snap Neutral Midtones). I've been using ProPhoto RGB images and working space with good results. If I switch to Adobe RGB the results are considerably different and less accurate (reds more saturated). Not sure why since there is very little image area out of Adobe RGB gamut.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

If I switch to Adobe RGB the results are considerably different and less accurate (reds more saturated). Not sure why since there is very little image area out of Adobe RGB gamut.

That would be expected if you capture the same RGB values but then assigned (rather than convert to) different profiles. Try capturing with ProPhoto and then do a Convert To AdobeRGB.

In my example I'm duplicating the ProPhoto RGB file and converting it to AdobeRGB, so the displayed color appearance doesn't change. The conversion of both to the GRACol print output space generally produces the same output numbers with the exception of the colors like 100% cyan that are outside of the AdobeRGB space, but are in the GRACol space.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

https://forums.adobe.com/people/rob+day  wrote

If I switch to Adobe RGB the results are considerably different and less accurate (reds more saturated). Not sure why since there is very little image area out of Adobe RGB gamut.

That would be expected if you capture the same RGB values but then assigned (rather than convert to) different profiles. Try capturing with ProPhoto and then do a Convert To AdobeRGB.

Nope. I'm using 'Covert to Profile' for the ProPhoto RGB conversion to Adobe RGB.

It probably is due to the Auto Curves function rendering the image differently due to RGB channel differences (see below). Processing color film images in LR/PS can be a PITA due to the  mask and color characteristic differences of the various color film types. However, using ProPhoto RGB with the Auto Curves function works quite well to color correct the film images.

Color Fim Processing.jpg

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

On a conversion I would expect the histogram and image RGB values to change as you are showing, but not the color appearance. What happens to the histograms if you convert the two versions to the same CMYK output profile?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 25, 2018 Oct 25, 2018

Copy link to clipboard

Copied

https://forums.adobe.com/people/rob+day  wrote

On a conversion I would expect the histogram and image RGB values to change as you are showing, but not the color appearance. What happens to the histograms if you convert the two versions to the same CMYK output profile?

'Convert to Profile' U.S. Web Coated (SWOP) v2 for both produces identical Histograms with Cyan and Black channels clipped.

Color Fim Processing2.jpg

Color Fim Processing3.jpg

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines