P: Histogram behaviors are different from prior versions

[FeedbackCommunityMember]

I am experiencing problems with the histogram in 19.1.0. I am a mid-career fine art photographer by profession, and have been a PS user since Version 2. I am also a member of the Authors Guild, and write on photography for various publications. I have been purposely hanging back at PS 2015.5.1, as it has served me well. Yesterday, I decided enough, is enough, and installed 19.1.0.

This first histogram is at Cache Level 1 for the file values of a 36Kx24K pixel 16-bit grayscale file.

 

Please notice how there are "tails" (lines) out each side of the main body of data, indicating that there are small levels of data almost to the limits of range. For me, it is important to know about these tails exist so that I do not end up creating a clipped condition when applying a curve function. We use S-curve limiters to compact the tails without clipping.

Here is the exact same file at Cache Level 1 for the same 36Kx24K pixel 16-bit grayscale file, but this time in 19.1.0.

 

Please notice how there is no tail indicating data extending to the left, and rather a botched one going to the right. This is not helpful! You will also notice that the Mean and Standard Deviation values differ.

Further, it use to be nice to be able to take the cursor and scan across the histogram with a display of level and count showing up for whatever was under he cursor. This seems to have gone away in 2015.5, and is even worse in 19.1.0.

Thanks for your help.

Pete

[FeedbackCommunityMember]

I see another problem showing in the top display.  All three channels are identical but near the tall peak I see a couple of different color values.  That is weird too.
Needs to be looked into.  I used sRGB for my tests as it is most stable version.
RONC

[Todd Shaner]

All three channels are identical but near the tall peak I see a couple of different color values.

I noticed them as well and it appears to be a bug since the original image is grayscale (R=G=B). Jeff spoke with someone in Engineering and they mentioned the Histogram uses a subsampled image. I can see how that would affect the original histogram results, but converting from grayscale to RGB Color using a subsampled image should keep R=G=B in the new Histogram.

[FeedbackCommunityMember]

Subsampling the image previous to doing a histogram should be fine if it not too severe.  I would say if the input is like 200k in one direction, the histogram should be run on segments not the whole thing.  I think the user needs decide whether there is enough variablity in the data to require segmenting.   This something that Adobe needs to plan on this not only for the histogram.  Ergodicity is something that needs to re-evaluated within the processing package.

RONC

[FeedbackCommunityMember]

To eliminate the Grayscale to ProPhoto conversion, I would desaturate the ProPhoto and then run Histogram to see if the color shows.
RONC

[Todd Shaner]

Applying Desaturate (CTRL+SHFT+U) to the RGB converted image removes the color spots in the Histogram palette. Isn't that expected if the subsampled image caused color artifacts (R≠G≠B)  to be created during the conversion?

[FeedbackCommunityMember]

The next question "is the problem in the conversion or in histogram".  If you take the conversion RGB and take the blend/difference with the desaturated RGB image and then view the output of that with Levels with the right arrow moved left to "2" you will see the difference of the images magnified in amplitude.  If the images are different then the problem is with the conversion from grayscale to ProPhoto.  If they are the exactly the same, the problem is with Histogram and ProPhoto input.  You might have uncovered a bug in the conversion too.

Life isn't easy trying to find what causes what happens.

RONC

[Todd Shaner]

Comparing the difference of the ProPhoto RGB conversion to it after Desaturate there appears to be numerous pixels at  level 1 that are different. So it appears the Convert to Profile operation is adding artifacts.

[FeedbackCommunityMember]

If the desaturated version into the histogram looks OK then you and Jeff need to pass the conversion problem on to the coders to fix and you eliminated one problem for the histogram coders.
I'm not going to think about the ProPhoto stuff anymore.
I'll try to to get a post up on recommendations for the histogram soon. Have a health problem first though.
RONC

[FeedbackCommunityMember]

Jeff,
How did you make that image have such a nice shaped histogram? I know of no way to modify the distribution of the amplitudes in PS other than Curves and Levels. I have a histogram shaper that I use sometimes but I'm surprised by the quality of the end tapering.
Regards,
RONC

[FeedbackCommunityMember]

Todd,
I did the same test on my grayscale to sRGB image and it has small differences between the output and the desaturated version.  The conversion tool is buggy.  The errors are mostly in the green channel.
RONC

[FeedbackCommunityMember]

Jeff/Todd,

Another oddity as I looked at the very first displays that Pete presented.  The computation of the Histogram is different as the Std Dev is not the same.  I'm assuming the same file was used as input.  Then in the display, the number of pixels in a bin scaling of the Histogram is different not only on the tails but across the entire function.  This is why the tails fall off the bottom.  Seems that the computation of the display was also changed and not handled correctly.

This is the PS2015vs2019 image compare:


The peak to trough distance is very significantly different.  I don't have either version installed to run a synthetic file test so someone in Adobe must do that for making corrections to PS2020.

RONC

[Todd Shaner]

Ron, Pete did mention this in the original post above. "You will also notice that the Mean and Standard Deviation values differ."


He also states, "Further, it use to be nice to be able to take the cursor and scan across the histogram with a display of level and count showing up for whatever was under he cursor. This seems to have gone away in 2015.5, and is even worse in 19.1.0.


It's clear something was changed that's causing it.

[FeedbackCommunityMember]

I meant to add more on how the tools should be tested.  There should be a suite of images that are tested for before/after of a modified tool.  These should include a number of synthetic images where the answer is known much like the one I supplied but probably with more blobs.  I have since modified it to include zero and maximum values. Then the decision whether the tool is good is up to a group of "Devil's Advocates" who are not coders but very experienced users and they have final say if the tool becomes part of Photoshop.  They would not only validate the tool but its interaction with other tools in PS.  They might have more and different synthetics to use to evaluate the tool.  They can not be overridden by the coder admins.  They answer to a GM or higher.
They are an additional cost but they would have caught what we are seeing when it happened.  It also would have found so many of the problems that have occured in PS, LR, and ACR of the last while.  The coders have to have deadlines not for release of the product but for the evaluation process.
A major point it these tools must e tested against knowns not just whatever the coder has on hand for fixing a problem.  Many of the knowns would be developed by the Advocates and shared with the coding staff.  They must be documented for either coder or Advocate to use to evaluate tools and other things.  Tools should have a history of maintenance including what evaluations were performed, by who, and when.

With push to release code, the coder can't win unless they have their tools and advice from the advocates.

The money lost by Adobe clients because PS, LR..... is never mentioned but it really hurts many of your smaller ones.

Sorry for the lecture but I don't perceive Adobe as a standard in quality that it once was.  You might pass this further up and I'm willing to discuss as there many companies that do things as I stated.  They probably have a different name for the Advocates.  I think "Client Advocates" is a good title and should appear in all stages of the development, sales, and use of a product.

"Deadlines are the biggest enemy of quality."

RONC

[Earth Oliver]

I'm jealous!

[FeedbackCommunityMember]

I'm not blaming everything on the coder.  Often the task is ill defined in the first place.  That part of the whole process needs to have it's methods reviewed to see that they don't cause problems.  That has to be on going just as the Advocates.
There is one thing I have found that some coders do is that they rewrite from scratch rather than debug of even understand previous code.  This makes it very possible to introduce differences especially if the functionality is not well documented.  Something like the histogram could well have suffered from this.
In the end it is up to management at all levels to realize that there need to be checks at all levels to ensure this type of problem doesn't get into production.  How many things like this are lurking where they are not directly visible?

RONC 

[petemyers]

Dear Adobe Support:

Problems first appeared following 2015.5 in regard to the Curves function and Histogram, and I have reported it continuously since 2018---still not fixed. These are fundamental functions of PhotoShop, so would consider them a priority for repair.

Running 22.0.1 on a MacPro 2019 with 96GB RAM, AMD Radeon Pro 580x, and Cataltina 10.15.7. I have been using PhotoShop since Version 2.

First, the easy one; the intercept line does not work in the CURVES function, and hasn't done so for a long time. It is an important part of the tool.

Second, the histogram is still screwed up. As I posted previously, it does not show the tails of the data properly, which is an OR function. The only time it works correctly is when viewing the histogram as "colors" in RGB---not in the selected channels, RGB composite or in monochrome.

I am still astonished that you folks cannot see this problem on your end, nor has it been fixed in YEARS. Every time I bring it up, I get severe push back by the moderator and the company. I have a long history in writing for a wide spectrum of photo publications, and it would seem to me to be enough credibility to take this issue seriously. Why is it not fixed?

Pete

[Jeff Arola]

Yes and in photoshop 22.1 the curves intersection lines don't show at all even though

Intersection Line is enabled in the Curves Display Options. I think versions 20 and 21 were broken as well.

[petemyers]

Grateful to you for your confirmation. That helps a lot.

Pete

[petemyers]

My post seems to have gotten lost in the weeds. Can any one outside of Adobe confirm the histogram issues?

JFA seems to have confirmed the Curves Function intercept line is not working, nor has it worked through multiple versions.

Pete

[petemyers]

Examples: let us start out with a 6Kx4K monochrome file, 16-bits that has significant data tails, then construct and RGB file by pasting the monochrome file into each channel.

First, let us try a Curves Function of this file. Please note two issues; first, the Intersection Line is not showing despite it having been selected. Second, carefully note the extended tails of the histogram data (which is correct here, not in the histograms).

Now, let us try using the Levels Control. Please note how the tails of the histogram data have disappeared.

Now, let us do a proper histogram on the RGB file. This first version uses the COLORS setting on the histogram, and as can be seen if you look carefully, the tails of the data are displaying properly. Hard to see, but a very thin line of green shows the tails on both sides of the data.

OK, so now let us select only the Green Channel from the same RGB file (remember, all channels have exactly the same monochrome image). As can be seen, the tails of the data have once again disappeared. Poof! Gone!

The final example is just doing a straight-up histogram on the monochrome base image that was used to construct the RGB file. What it shows is that the monochrome histogram does NOT display the tails. Since most of my work is monochrome, this is the histogram that is most important to my work. I used the RGB examples above for the reason that most people work in RGB, and the COLORS setting does work properly, while the rest not.

The last time all of the histograms worked properly with the functions was in 2015.5. I have shared this information repeatedly with Adobe over multiple versions of PS, and it has not been resolved.

Please Adobe, these are fundamental tools in PhotoShop. For advanced users, these are the essence of our work. The interception line and histogram issues need to be resolved!

[petemyers]

And just to understand, the tails in the histogram data are a logical OR function. If there is even one pixel in an entire image at a given level, there should be an indication of that on the base-line of the histogram. That is how the histogram has worked the entire life of PhotoShop until recent versions.

As it is now, some one has decided that if that single pixel is outside the statistical relevance of the histogram data, then it will not show. The problem with this logic is that there can be long tails hidden that when ignored representing significant amounts of energy in the photo, resulting in either white or black clipping and holes in the photo.

[J453]

I see that https://feedback.photoshop.com/conversations/photoshop/photoshop-191-histogram-behaviors-are-differe... was addressed by engineering in 22.0.1 or later. Are you still seeing a problem?

[J453]

I see that engineering made some changes to try and address this in 22.0.1 or later. Are you still seeing a problem?

[J453]

The intersection line issue is still open/being worked on: https://feedback.photoshop.com/conversations/photoshop/photoshop-intersection-line-in-curves-not-wor... 

[petemyers]

Hi Jeffrey:

Thanks for checking in.

Yes, both the histogram issues and separately, the interception line in Curves issue are unresolved. Again, I am on 22.0.1. I did not see any bug fix notes to indicate that either of these issues was addressed in 22.1. Do I need to upgrade to check?

It seems from your note that the interception line issue is in engineering to be resolved---which is GREAT! Bravo. Really appreciate it.

But how do we get the histogram issue fixed? There is an obvious difference in the histogram used in Curves or in RGB COLORS histogram, then in a channel or in monochrome. The former being correct, the later not.

Pete