Participant

Question

Can't use exact float values in 32 bit color picker

Forum|Forum|7 years ago
April 2, 2019
5 replies
2918 views

I've a RGB 32 bit per channel image and want to use as exact as possible RGB colour, e.g. 1.224747 1.1708295 1.161287.

The nearest exact IEEE754 floating point representation of these values are: 1.22474694... 1.17082953... 1.16128695...

However, when I type the first RGB values into the Color Picker, the text entry boxes truncate the values to "1.2247 1.1708 1.1613", which is fine but when I OK the Color Picker and then re-open it with the only just set colour, it now shows RGBs of 1.2184 1.1648 1.1553, which are quite a long way past the nearest representable floating point values of the originally entered RGB values.

What's it doing to my desired values so that they land quite a way off what the nearest representable float values should be?

Is there any way to get it to use exact IEEE754 float values and not quantise/snap them to some other value?

32_bit_floating_point

This topic has been closed for replies.

T

Test Screen Name

Legend

I was suggesting areas you might like to investigate, since the chances of hearing from someone who has inside information is really quite small.

A few minutes' tests seemed to show that when all numbers are 1.0 or less there is high accuracy, and if any number is over 1.0 the accuracy of all numbers is very low. I observe that in any such case the colour picker sets "intensity" to a number with two decimal places. I speculate that numbers over 1.0 are stored as normalized RGB values in the range 0.0 to 1.0 plus an intensity. I further speculate that the HDR component is therefore evaluated as F(normalized RGB component,intensity). Since intensity is rounded to two decimal places this would produce really low accuracy. All speculation, but perhaps you can share your own. You never answered, I think, why you find it necessary to specify HDR colour components with such accuracy.

T

Test Screen Name

Legend

No, I'm describing 32-bit IEEE floating point exactly. There is a 24-bit mantissa and 8-bit exponent, giving an accuracy of 7.22 decimal digits. IEEE 754 - Wikipedia

So I would not want to see more decimal places. However, as I said this does NOT account for your issue in full. Is this not clear "Still doesn't explain why the values change. It may be inherent in the colour picker (try a different method to check colour) or it may be unexpected colour management."?

dan72721527

Inspiring

"... so your figures have MUCH to much precision..." : I think you'll find my figure of 1.224747 is very well representable by IEEE754 with 0x3f9cc482 == 1.2247469425201416015625. See for yourself: https://www.h-schmidt.net/FloatConverter/IEEE754.html

"Still doesn't explain why the values change. It may be inherent in the colour picker... or it may be unexpected colour management": I mean, you're asking the same question I am; does anyone know what it's doing exactly? If you don't know, asking the same question again doesn't really get us anywhere. Is this the wrong place to be asking this or do you or anyone know where we can get a definitive answer?

davescm

Community Expert

dan72721527 wrote
Is this the wrong place to be asking this or do you or anyone know where we can get a definitive answer?

Hi

As I said in my earlier post, only developers will be able to tell you. This forum is answered by users not Adobe staff.

You could try the link below - but whether developers will wish to reveal the internal workings of Photoshop, I do not know.

Photoshop Family Customer Community

Dave

T

Test Screen Name

Legend

Actually, some research shows Photoshop MIGHT use IEEE 32-bit floating point internally. Both 32-bit integer and a less precise 32-bit floating point format exists. 32-bit floating point is less accurate, with 24 bits of precision. So your figures have MUCH too much precision. Given that numbers are in the range 0.0 to 255.0, and there are only 7 decimal digits of precision in this format, it would be entirely wrong for Photoshop to show more than 4 decimal places.

Still doesn't explain why the values change. It may be inherent in the colour picker (try a different method to check colour) or it may be unexpected colour management.

davescm

Community Expert

https://forums.adobe.com/people/Test+Screen+Name wrote
Given that numbers are in the range 0.0 to 255.0............

That's not quite correct. Whereas 8 bit and 16 bit split the same range of 0-255 into smaller increments, 32 bit floating extends the range beyond that which can be described in 8 bit or 16 bit. In other words it can describe lighter colours than the whitest 8/16 bit or darker colours than the blackest 8/16 bit. Hence it's use in linear HDR imaging, and the need for tone mapping when converting back to 8/16 bit.

As to why there is a limitation on direct entry in the colour picker, only the developers could answer, but I would question whether any finer precision entry would actually be visible.

Dave

T

Test Screen Name

Legend

32-bit colour is not stored as floating point. It's stored as integers on a range. Furthermore, it is not stored as your own RGB colour values, but colours in a specific wide gamut space. So colours are always converted to your RGB for display. There's rounding go on.

But these changes suggest something else.

Mylenium

Legend

Well, let's start with the fundamentals: Are you actually using color management, a suitable monitor and al lthat good stuff? furthermore, from where do you glean those float values even?

Mylenium

dan72721527

Inspiring

Mylenium: Can you explain what's the relevance of having a suitable monitor and finding out where the numbers come from, regarding the apparent snapping of 32-bit floating point values? To put it another way: Suppose I asked about why, when I enter a value in some rudimentary calculator tool (that might appear to use 32-bit floating point numbers), do I see it snap to a number a long way away from the nearest representable IEEE float value. See https://www.h-schmidt.net/FloatConverter/IEEE754.html to play around with standard 32-bit floating point numbers. E.g. Entering '1.224747' shows that the nearest exact representation is '1.2247469425201416015625'... photoshop's number/text field appears to snap to '1.2184'.

Test Screen Name: It sounds like you're trying to describe 'fixed point' instead of floating point? Can you show me how you think it's using a 32-bit fixed point numerical format, please? It would seem to be an odd choise for 32-bit numbers in photoshop. Anyway, even if 32-bit fixed point (with 24 fractional bits) was used, over a numerical range of 0.0 to 256.0 then 1.224747 is very well representable with 32-bit fixed point value of ‭20547845‬ ( 00000001 001110011000100100000101‬ ), coming it at 1.224747002... which is a little above the entered value, so, supposed it rounds down; the next value down (20547844) comes in at 1.2247469... both of which are much better than 1.2184.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded