Change in EXR open from CS2 to CS3 can this be fixed?

[_progress_]

It seems the monkeys have been at the file formats again...!

Open an exr with an alpha in CS2 and the image displays normally and the alpha is retained.

Open an exr with an alpha in CS3 and the alpha channel is applied to the transparency and then lost... which is really STUPID considering you might apply 0 alpha values to parts of the image you retain visually, as you might just want to use the alpha to drive an effect and not just be myopic and think it's just for transparency.

So, can this be fixed? I can't see any info on it?

Will CS2 non intel plugin work on an intel system in CS3

If not, effectively PS is useless for exr work for us.

Or is this fixed in CS4?

[Neil_Keller]

This topic, although quite heated with comments from many first-time visitors, has none-the-less been pretty civilized and articulate until post #115.

It is rare to have someone as directly involved in Photoshop development as Chris Cox appear and respond as he has here. Whether you agree with his viewpoints or not, he's a valuable asset to these forums, particularly to the many who have been regular participants over the years.

For this topic, everyone is welcome to make his or her point so Chris can use them to evaluate any future changes in the app. It's to no one's benefit to make him feel unwelcome.

So please, let's keep the topic to the point and name-calling and accusations out of here. I do not want to be put in the position of closing this long topic.

Thanks!

Neil
Forum Host

[_Christopher_Prosser_]

Hi Folks,

This is quite a thread. I'm writing as one of the people on the After Effects team that finally chose to throw in the towel on spec purity and add options are users needed.

To summarize this whole thread in my own words:
The spec is very clear that OpenEXR images are linear data and premul with their alpha data. What Photoshop CS4 extended is doing is technically correct and following the standard.

Alas, many, many other products, and people using OpenEXR's manage to stuff all sorts of data into RGBA which doesn't conform to the standard.

In the AE CS3 beta cycle we tried to conform to the standard. But in the end, we found so many 'perverse' OpenEXR files 😉 , that we decided it was better for After Effects to be flexible enough to deal non-conforming files and further pollute the workflows than to follow the standard.

As Chris Cox pointed out, the Photoshop customer base is different than After Effects and the PS team has a different risk tolerance for adding support for non-conforming workflows. I'm not going to try and guess what percentage of Photoshop users are trying to work with non-conforming OpenEXR files. Clearly there are at least the number of posters in this thread ;).

Luckily both AE and Photoshop support file format plugins. For those of you who need support for non-conforming OpenEXR files in PS CS4, Brendan Bolles sells the ProEXR plugin. AE CS4 actually ships with his OpenEXR plugin for AE.

--chris prosser
ae engineering manager

[Guest]

Hi folks,

A number of people have written to bring this thread to my attention. I haven't had time to read through all 120+ posts (!), but I'll try to do so shortly (isn't that what weekends are for?).

At the end of the day, we have to do what's useful to you, and we will.

Thanks for your passion & patience,
John Nack
http://blogs.adobe.com/jnack

[_Zap_Andersson_]

> To summarize this whole thread in my own words:
> The spec is very clear that OpenEXR images are
> linear data and premul with their alpha data.

But, as I said many times before, to say this is to *fundamentally misunderstand* what "premultiplied alpha" means.

All it *really* means is DO NOT MULTIPLY in your "over" operator.

I.e. do NOT do

r = fg * fg.alpha + bg * (1 - fg.alpha)

but do

r = fg + bg * (1 - fg.alpha)

That is ALL it means. Which, if you actually read the technical document, to OpenEXR as Florian himself (part of the team writing it!!!) posted above, *actually says*.

http://www.openexr.com/TechnicalIntroduction.pdf

Yes. It *explicitly says* that the format of the RGBA data is such that the "over" operation should be

r = fg + bg * (1 - bg.alpha)

To draw from this the conclusion that a color with zero alpha and nonzero RGB is "illegal" and can be thrown away is quite an unfounded extrapolation to make!!

And also note that the spec says that this is *by convention*. Not rule, nor law, but by *convention*. I.e. "the most common case".

> What Photoshop CS4 extended is doing is technically
> correct and following the standard.

So not using the channel labeled "alpha" as an alpha channel is tecnically correct? To throw away fully legal RGB data just because Alpha is zero is correct?

/Z

[Neil_Keller]

Chris,
>For those of you who need support for non-conforming OpenEXR files in PS CS4, Brendan Bolles sells the ProEXR plugin. AE CS4 actually ships with his OpenEXR plugin for AE.

Please correct me if I'm wrong, but the solution is as easy as cloning and installing the same ProEXR plug-in in Photoshop CS4 as is included with After Effects?

Neil

[Neil_Keller]

Chris,
>For those of you who need support for non-conforming OpenEXR files in PS CS4, Brendan Bolles sells the ProEXR plugin. AE CS4 actually ships with his OpenEXR plugin for AE.

Please correct me if I'm wrong, but the solution is as easy as cloning and installing the same ProEXR plug-in in Photoshop CS4 as is included with After Effects CS4? Or is there a specific version for Photoshop?

Neil

[_Brendan_Bolles_]

Neil, the ProEXR plug-in for After Effects is an AE-specific plug-in. It uses the existing 3D channel API to send those extra passes through. The AE plug-ins are free, open source, and ship in CS4. It doesn't do anything special to the alpha channel because AE already has settings to deal with that.

For Photoshop, it's a totally different plug-in (Photoshop API), doing a totally different thing (making layers), anderumit's not free.

Brendan

[_Florian_Kainz_]

Chris,

since you claim to know exactly what I mean, without actually
discussing this with me outside this rather bizarre thread, I
would like to clarify that I, as the original author of OpenEXR,
completely agree with Zap Andersson's post earlier today.

"Premultiplied alpha" describes how the "over" compositing
operator is supposed to work. Premultiplication does not mean
that A == 0 forces R, G and B to be zero. As I mentioned earlier
in this thread, zero A with non-zero R, G and B is a perfectly
legal combination. It represents a self-luminous but completely
transparent object, not an abuse of the OpenEXR file format.

Photoshop is an outlier in how it treats OpenEXR's A channel;
Nuke, After Effects, and many other programs handle A differently.
The OpenEXR specification does not say that Photoshop is right and
everyone else is wrong.

I agree that Photoshop's implementation can be considered "correct"
(except for the A == 0 case, which started this whole thread in the
first place). However, there are other correct - and more useful -
ways to handle the A channel.

Frankly, I don't understand why OpenEXR is "the wrong file format
for what [some Photoshop users] are trying to do." Plenty of other
applications let users do what they want to do. Personally, I'd
say this is strong evidence that the file format isn't really the
culprit, but Chris, I'm sure you'll find a way to refute that.

Florian

[Chris Cox]

Sigh. I guess I need to write a tutorial on what premultiplied and non-premultiplied color mean and how they relate in different applications and contexts. (and probably document why straight color works better for editing)

From the viewpoint of an application that works with straight color, premultiplied does mean that you multiply the color values with the opacity channel (same as compositing over black while leaving the opacity channel intact).

From the viewpoint of an application that works entirely in premultiplied color, you could have non-zero color values even if the opacity is zero. Whether those values are useful or garbage is debatable (since if you just add them you suddenly have changes where you claimed to have no opacity).

But you can't get from the straight color system to the premultiplied system without multiplying the color values by the opacity values. Wherever the opacity was zero, the premultiplied color values become zero.

When going from premultiplied color to straight color, you have a clamped division operation. And what color is the result of a non-zero color value divided by zero opacity? Unfortunately, it is undefined (infinity is an easy answer, but that kinda breaks later processing).

[Chris Cox]

Florian - partly I'm going from what you wrote in the spec., and partly from previous conversations (email, phone, and at your office).

There is no notion of "self luminous but completely transparent" in straight color -- you must have some opacity for any color to appear. That concept of color without opacity only works in a premultiplied pipeline (but still breaks some filtering stages).

There are 2 requests here so far:

1) to treat the color in EXR as straight color, not premultiplied (or at least open it that way and let them do other work on the data)
2) to open the A channel as an arbitrary alpha channel, not opacity

Both of those go against the spec. as written and break interoperability for EXR files. If you intended for those to be allowed, then it was not communicated in the EXR spec. And I strongly feel that allowing them will create more confusion around the interpretation of EXR files. What is written into the file should have one, clear, unambiguous meaning. Having multiple potentially valid interpretations of the same file data leads to big messes (even bigger than we're dealing with here).

[_Florian_Kainz_]

What "premultiplied" means, in fact what any word means, depends on
who uses it. That's why technical documents generally try to define
important terms before using them.

Since the OpenEXR documentation is ambiguous enough to allow the
present discussion to take place, I will augment the documentation.
On about page 5 the "Technical Introduction" states:

[The A channel represents] "alpha/opacity: 0.0 means the pixel
is transparent; 1.0 means the pixel is opaque. By convention,
all color channels are premultiplied by alpha, so that
"foreground + (1-alpha) × background" performs a correct "over"
operation.

I will add following (or something close to it):

Note: describing the color channels as "premultiplied" is a
shorthand for describing a correct "over" operation. With
un-premultiplied color channels, "over" operations would require
computing "alpha × foreground + (1-alpha) × background."

"Premultiplied" does not mean that pixels with zero alpha and
non-zero color channels are illegal. Such a pixel represents an
object that emits light even though it is completely transparent,
for example a candle flame.

Since pixels with zero alpha and non-zero color can and do occur
in OpenEXR images, application software must avoid discarding the
color information in pixels with zero alpha. This can be a problem
for applications that internally use an image representation where the
color channels have not been premultiplied by alpha. After reading
an OpenEXR image, such an application must undo the premultiplication
by dividing the color channels by alpha. This division fails when
alpha is zero. The application software could set all color channels
to zero wherever the alpha channel is zero, but this can alter the
image in an irreversible way. For example, the flame on top of a
candle would simply disappear and could not be recovered.

If the internal un-premultiplied image representation uses 32-bit
floating-point numbers then one way around this problem might be to
set alpha to max(h,alpha) before dividing, where h is a very small
but positive value (h should be a power of two and less than the
smallest positive 16-bit floating-point value). The result of the
division becomes well-defined, and the division can be undone later,
when the image is saved in a new OpenEXR file. Depending on the
application software, there may be other ways to preserve color
information in pixels with zero alpha.

[Neil_Keller]

Brendan,
>the ProEXR plug-in for After Effects is an AE-specific plug-in.

Thanks for the clarification. I suspected that was the case.

Neil

[Chris Cox]

Florian - ok, I think we can deal with that, and that minimizes the loss due to premultiplication of the color data.

But that does not address the requests here to treat the A channel as non-opacity, or the requests to treat the color data as straight color. Those requests might be based on misunderstandings of the situation, but it's difficult to tell from the posts made here.

[_Ean_Carr_]

In Florian's words in the tech doc, and like others have pointed out, it's only a "convention" so that should leave the door open for better PS flexibility in the EXR reader: an option to treat the A channel as non-opacity. At least until you have multi-channel EXR support, like Erik said.

Pretty please? With a cherry on top? Whipped cream? Sprinkles?

[_Christian_Bloch_]

Wow, we've come to a conclusion!
Thank you, Florian, for the addendum, and thank you Chris for listening patiently. It's really appreciated.

Layer masks may be technically very different from Transparency, however, in a user's perspective a layer mask does the same job and is more accessible. The root of the problem is that Photoshop hides the Transparency channel. If Transparency would just show up in the Channels palette, and would be editable, we wouldn't have this discussion in the first place...

Here is a constructive approach on how to provide options for advanced users without confusing everyone else. How about triggering the options dialog with a modifier key when the image is loaded? Hold CTRL on PC, or Option on Mac - and you get to all the advanced and (and possibly dangerous) settings: Alpha behavior, display/data window, linear/gamma. The standard behavior can stay as it is, and the casual HDR photographer won't know and won't be confused.

I'm very happy that such a discussion can actually be held in a constructive and civilized manner. Thanks to everyone for their patience.

[Chris Cox]

We don't have a conclusion just yet - but responsible parties are talking offline.

Exposing the layer opacity channel would require quite a few changes, and a lot of testing. That's not going to happen quickly.
What I have proposed is a way to move the opacity data from a layer into a layer mask while setting the layer opacity to 100% opaque (currently you can do that for 8 bit easily, but can't do it correctly for 16 and 32 bit/channel). That exposes all the color values, and allows for independent editing of the opacity values. The command to reverse that transform is already in Photoshop (Apply Layer Mask). While that will help one nice person who really needed to edit his opacity values post-render, it does not solve any of the premultiplication issues.

Florian's suggestion to pin opacity to a really small number may help the premultiplication issues. But I'm not sure it'll solve everyone's issues as posted here.

[_jonah_]

Thank you Chris and Florian and everyone. I know I'm a total noob in this crowd and that I don't have much more than a "just because" argument.

This has turned into a great thread. I have read every word and I really appreciate all of the input and information. I've learned a lot. It's almost as helpful and constructive as a VRay forum thread!

We have one license of ProEXR (which we love) on a community workstation and that is solution enough for the time being. I look forward to reading further discussion.

Thank you everyone.

Jonah

[_Florian_Kainz_]

One last thing: for everyone's testing pleasure I shot two new photos
for the OpenEXR sample image collection. The new images demonstrate use
cases for pixels with zero A and non-zero RGB values. You can download
the files from here:

http://cvs.savannah.gnu.org/viewvc/OpenEXR-images/ScanLines/PrismsLenses.exr?root=openexr&view=log
http://cvs.savannah.gnu.org/viewvc/OpenEXR-images/ScanLines/CandleGlass.exr?root=openexr&view=log

[_Brendan_Bolles_]

Wow, you ILM guys have some cool toys, including now a camera that can shoot alpha channels?!? ;)

Neat images, Florian. I composited them in Photoshop using the 2-layer technique I mentioned earlier. BTW, Photoshop's Invert function is disabled in 32-bit mode, so I had to use Levels and set Output Black to 255 and Output White to 0. (Don't ask me why Levels uses 8-bit values in 32-bit mode.)

Brendan

[Chris Cox]

Invert doesn't work so well when your limits are -infinity to +infinity

Level is just using UI that customers are familiar with....

[_Brendan_Bolles_]

Chris, I think the standard Invert math works perfectly well in float and I imagine anyone who uses Invert in 16-bit will miss it in 32-bit (especially when working with a layer mask or something like that). Yes, anything >1.0 will end up negative, too bad there isn't some sort of clip operator in Photoshop to take care of that.

The Levels UI situation is very problematic in linear space because the Input Black and Output Black settings get far too sensitive and the difference between 0 and 1 (8-bit code values) is too much. I'd like to enter 0.5 in there if only it'd let me. Or give us real floating point 0.0-1.0 entry. I doubt anyone working in 32-bit mode would be put off by this.

Brendan

[Chris Cox]

Brendan - while the math might work, the result is next to useless.
(1.0 - x) works when the values are all between 0.0 and 1.0, but isn't very useful when the values are -infinity to +infinity. While some people might be able to understand the math and make use of it (if the exact math was documented for them) -- the vast majority would not (and would file bugs about it).

We're still working on a better UI with more precision (that doesn't cause other problems). The same goes for a number of other areas related to 32 bit images.

[_Zap_Andersson_]

I'm really happy this discussion is getting somewhere. Special thanks to Florian.

And that Chris (& Adobe) is finally seeing the light of what "premultiplied" really means. (Perhaps even luminiscent yellow light 😉 )

I do however hope everyones understands that this definition of "premultiplied" applies to every format with an alpha channel, i.e. TGA's, etc. etc, i.e. 1,1,0,0 is a legal color in a TGA file as well (well, it'll probably be 255,255,0,0 but you get the idea).

/Z

[_progress_]

i I'm really happy this discussion is getting somewhere.

Likewise.

Special thanks to Florian & Chris and their professional ability for being able to the point through the heat haze.

Now Chris, how do I get onto the beta?

[Chris Cox]

No, many file formats are not premultiplied, and some file formats support an alpha channel that has nothing to do with transparency/opacity.

TGA is not premultiplied (some people think it supports opacity, and some don't).
PNG is not premultiplied, and only supports opacity.
TIFF is premultiplied if you use opacity (associated alpha), and not if you only use unassociated alpha channels (because they aren't opacity, the color data cannot be premultiplied).

Just because the format has a place for some extra data - that does not mean that the format supports your interpretation of the data. Most file formats are pretty specific about how the data should be interpreted, but some are not, and some (TGA) have grown a following of users who just ignore the specification entirely.