Change in EXR open from CS2 to CS3 can this be fixed?

[_progress_]

It seems the monkeys have been at the file formats again...!

Open an exr with an alpha in CS2 and the image displays normally and the alpha is retained.

Open an exr with an alpha in CS3 and the alpha channel is applied to the transparency and then lost... which is really STUPID considering you might apply 0 alpha values to parts of the image you retain visually, as you might just want to use the alpha to drive an effect and not just be myopic and think it's just for transparency.

So, can this be fixed? I can't see any info on it?

Will CS2 non intel plugin work on an intel system in CS3

If not, effectively PS is useless for exr work for us.

Or is this fixed in CS4?

[Chris Cox]

Why are people joining the forums and making their first forum post to this topic without reading the previous posts or taking a few minutes to find out what the heck they're talking about?

If you want the Photoshop behavior with regard to the A channel in OpenEXR to change, you are going to have to take it up with the folks who write the OpenEXR spec. That spec says that the data is to be interpreted one way, and one way only -- which is exactly what Photoshop is doing. If you want the data to do something else, or mean something else: please use another file format that does what you need, or change the EXR spec.

[_janz_]

Chris Cox wrote:
Even if you work with it unpremultiplied while in the application (like it always is in Photoshop) - you already premultiplied it when it went into the EXR file. YOU ALREADY LOST DATA WHEN YOU SAVED. Oh, I'm sorry, am I shouting?

Hi,
Where in the spec do you read the above, about doing multiplication of the RGB channels with the A channel as the data gets written to disk?

All I can find is; "The [channel's] name tells programs that *read* the image file how to *interpret* the data in the channel." (page 4 of TechnicalIntroduction)

I can't find the paragraph, where it says you have to multiply the RGB channels with the Alpha channel as the data is written to the EXR file, thus loosing data.

Cheers

[_m__hutch_]

This is seemingly a dead end, in terms of working with this particular representative. Is there another Adobe representative that can weigh in on this topic?

[Chris Cox]

janz - read the existing posts.

m. hutch - read the existing posts. You've already burned the bridge with the one person who could or would have helped.

[_sam_cuttriss_]

These "people" you are so flippantly addressing, are industry professionals who have spent the last 2 decades or so up in the small hours of the morning writing elaborate scripts or awkwardly contorting already strained pipelines to decieve Photoshop into reluctantly relinquishing relivant images, so you can casually claim your product was used on [insert film here].

Adobe Photoshop Engineers would be wise to listen to their user base, especially when they have so patiently explained the situation. And rest assured, as gut wrenchingly painful as it is to read this mindlessly repetative thread; we have.

consider this fromt he open exr documentation ( http://www.openexr.com/photoshop_plugin.html )
"Un-Premultiply: by convention, OpenEXR images are "premultiplied" - the color channel values are already matted against black using the alpha channel. In Photoshop, it's more convenient to work with unmatted images. It's important to use this option rather than un-premultiplying the image within Photoshop, because the plug-in will un-premultiply before applying exposure and gamma correction.
This option will have no affect if your image does not contain an alpha channel."

by convention OPENEXR images are premultiplied, sure, by convention, you shouldnt wear white socks with sandals, but that wont stop vfx folk from doing it!

Point being, this is not an unreasonable request, it is entirely trivial to impliment, and noone has been anything but patient when requesting it.

would you kindly reconsider.
_sam

[Chris Cox]

No, OpenEXR files are premultiplied by definition. If you want to change the definition, talk to the OpenEXR folks. Photoshop is using the existing definition for the file format.

We do listen to users, a lot. But sometimes users make mistakes. Sometimes they ask for things that would do more harm than good. Sometimes, they even ask for things they really, really don't understand. And we try to explain, we try to help them understand (and help us understand why they're asking for something bizarre). Are you listening?

When the OpenEXR file format specification changes, we will reconsider. Until then, I strongly suggest that you use a file format more appropriate to your workflow needs.

[_Alan_Boucek_]

Hi Chris,

I read all the posts before I posted. Yes, this has been cross-posted to the Nuke users list.

Strict adherence to the spec may work in the world of engineering, but we're in a business where specs evolve. The spec was designed for doing professional visual effects work. If you're going to implement the spec, it would be helpful if you learned a bit about how it is used in professional visual effects/CG production. I've seen contributions to this thread by people whose names are on the spec. They might be able to point you in the right direction.

The number one requirement that all of us have is that if a tool reads in data, the tool should be able to write it out without changing it. Clearly, this is broken for some users.

You're reinforcing a frustration that so many of us have with working with Adobe products- there's the Adobe interpretation of computer graphics, and there's an industry consensus, and they're not the same, and the ways that they are different often seem pointless.

[Chris Cox]

Photoshop does read EXR data in and write it out with minimal change.

Yes, specs evolve. But the EXR spec. has not changed. Again, we'll reconsider when the spec. changes. Trying to "evolve" the spec. by ignoring it just leads to trouble (and I'll offer this topic as exhibit #1).

If you try to use OpenEXR and expect un-premultiplied behavior - then it was broken the moment you wrote the data into an EXR file.
But according the the EXR spec. Photoshop is doing what it is supposed to do. If Photoshop did what you ask, it would break interoperability, and would still not do what you need (unless everyone agrees to always treat EXR as non-premultiplied all the time, and that just breaks all existing files and existing versions of applications).

No matter how many people try to use a screwdriver to drive in a nail, that doesn't make it the right tool for the job. You have other tools, that are more appropriate for the job -- use them.

[_Susumu_Yukuhiro_]

Have you seen the ILM's photoshop exr loader Chris?

You can downloaded from the openEXR site.

It let you unpremult the alpha and even gives you an option
to change the gamma and exposure.
I've been using it and I love the OPTION I have with that plugin.

The reason I wish you guys would come up something like that is that
the ILM exr plugin doesn't support 32 bit, and it's been causing
some issues in recent shows we've been working.

I assume over 95 % people who use the exr format in Photoshop are people in the VFX related, and at this point as I read this thread, pretty much 100% of the users WANT to have the option to unpremultiply the alpha.

Even if you are right about how you implemented the spec, no one is happy with it, and don't want to use it.

Then, I don't see why you would even have exr format support in photoshop. You might as well mention in the spec sheet that Photoshop supports exr but no one in the VFX industry likes it, and recommend to buy proEXR as a alternative solution.

Wouldn't you want to have a feature that makes people happy, and actually use?

[Chris Cox]

You seem to be confusing your segment of an industry with the larger audience of Photoshop users. People are using the EXR plugin shipped with Photoshop, in many industries. Only a few have complained. VFX represents a very small fraction of the people using the EXR file format. But Photoshop has to support everyone using the format, in many different workflows, interoperating with many other applications.

If you don't like the way EXR is designed, you are free to use another file format, or petition the EXR folks to update their spec. But we implement the current spec.

None of you have really talked about what you're trying to accomplish -- you're still asking for your imagined solution to a problem as you understand it (back the the screwdriver for nails thing). If you want a useful solution, we're going to have to talk about the larger problem, larger workflows, and consider alternative solutions (you know, reach for a hammer).

PS. Inviting drive-by postings by people unfamiliar with the issues really is not helping your case.

[Chris Cox]

PPS. This forum software blows when you want to format the text.

[_Diogo_Girondi_]

Break interoperability with what exactly?

Because for now the interoperability you say is already broken and prevents me from bringing any EXR that went trough PS to my comp package as it was and should remain.

And even tho I do get your point regarding the specs I still can't see how something as simple as this, which could be easily solved by a really silly PS Action if it worked the other way around would do more harm than good.

The OpenEXR docs haven't been updated since 2006 and it says "By CONVENTION, all color channels are premultiplied by alpha" and not "All color channels MUST be premultiplied by alpha", so even by that time they've left room for different ways of dealing with this.

But as you said there are other tools that are more suited for the job, in this case the current Photoshop built-in EXR reader just isn't one of them.

Meanwhile if you really need OpenEXR files in Photoshop better spend 95 bucks on the ProEXR plugin from fnord or just dump Photoshop till ILM takes the time to update those three text lines on their docs to make Adobe happy about it.

[_sam_cuttriss_]

I posted this to the nuke list in the hope the people most affected by the current implementation would share their wisdom and help improve future versions of Photoshop.

These are the people that you should be turning to for guidance, its their visions that will become the commonplace features of Photoshop CS18.

This is the exact opposite of using a screwdriver to hammer a nail, the founding intention of exr was to provide a FLEXIBLE format for the VFX community who were feeling restricted by the regular formats available in 99.

I can guarantee the other industries enjoying the pleasures of exr would be equity elated to see the requested feature.

At its most basic:
A ) If an element cant pass through our studios pipeline utterly unchanged, I have failed.
B ) If an element cant pass though an image manipulation package utterly unchanged you have failed.

Saddly A is dependent on B,

And a specific practical example, um.... i have an exr with information in the A channel, i save it from photoshop, at a later date the A channel is deemed unsatisfactory, i open it again.....(i fail )

[Mobius Strip]

Why does the EXR spec push premultiplication?

[_dekekincaid_]

Most of the troubles are issues with how we use exr in other applications that don't tie the rgb to the a. For example textures on a 3d model. We often times paint beyond the border of the alpha because depending the filtering a 3d app uses when it maps it onto the 3d model, you will often times get 1-2 pixels of black on the borders of the UV's. So it is very important to keep that extra data beyond the alpha.

With compositors we will get matte paintings with extra information outside the alpha which often times we will use at a later date because we are going to change the alpha inside our compositor. The current way photoshop reads and writes openexr forces us to write unpremultiplied images to bring into photoshop and then when we do have mattes for separate layers we have to write them to a separate files so photoshop doesn't nuke the information outside it.

Another big issue with the exr reader is it crops the image down to the bounding box(DOD). When I give someone a 1920x1080 image, I expect it to read into photoshop the same, not as a 1024x1024 image. The placement of objects in the frame is very important.

>You seem to be confusing your segment of an industry with the larger audience of Photoshop users. People are using the EXR plugin shipped with Photoshop, in many industries. Only a few have complained. VFX represents a very small fraction of the people using the EXR file format.

Exr was made by vfx artists for vfx artists. We are the audience it was made for, not the larger audience of Photoshop users. Even though the spec doesn't specifically lay it out that way it would be nice if there was a check box so the image is read in as unpremultiplied with the alpha in the channels as a b&w alpha and not as transparency.

[_Susumu_Yukuhiro_]

I am very aware of that VFX is a small potato for Adobe.

But I was specifically talking about photoshop users who use EXR file format, and if you say there are other industy using this format more than VFX industry, could you tell me who that is?

I might be the ignorant one here if DTP or web are using EXR format
more than VFX industry, and that would be a big news for me.

At this point, I am convinced that there are nothing we could change your mind, and I am giving up to have any hope for adobe to implement anything good for VFX.

Very sad.

[_Matthew_Scheuerman_]

I currently work in web and I am an aspiring matte painter.

I have never used the EXR format for web design or DTP. In fact, the only time I have read, heard, or talked about EXR it has been in relation to VFX. I have never used EXR in my web design career. Most of my colleagues/friends who are amateur/pro photographers don't use EXR. The only other industries that I can think of that uses Photoshop would be medical image analysis, crime scene image analysis, and architecture. I doubt these industries use EXR on a daily basis.

Are there any other industries using EXR?

Regarding some of the messages that have been posted here. Everyone knows what's been said. There's no reason, other than to be facetious, to quote some of the remarks made. But I do feel that it's important to note that for matte painters such as Susumu Yukuhiro, Photoshop is our only option for finished matte paintings. There is no other tool besides Photoshop for compositing/painting photo-real matte paintings. Cinepaint in it's current form can't offer the functionality that a feature film matte painter requires.

I hope that the EXR documentation can be updated so that the Photoshop crew can implement updates to the way that Photoshop handles OpenEXR. However, it would be also be great if Adobe implements features that its customers ask for.

[_Zap_Andersson_]

The error here is that Chris is interpreting the word "premultiplied" literally. Many people do, incorrectly. I really hate the word, actually, because it leads you to believe it's literal meaning, that something has been multiplied with something "before" (i.e. "pre" something).

THAT IS WRONG. It is a total misrepresentation of what "premultiplied" means.

I don't know what a better word would be, but it's meaning is "dont' multiply the color-channels with alpha".

I.e. in non-premultiplied (aka "straight" alpha), a compositing operation would be

r = fg * fg.alpha + bg * (1 - fg.alpha)

And in "pre multiplied" the compositing math is

r = fg + bg * (1 - fg.alpha)

Notice the lack of a multiplicatoin on the fg? This is what lead the people to name this to say it is "pre multiplied". It isn't, really. It just means IT HAS THE PROPER RGB DATA TO BE ADDED TO THE MATTED BACKGROUND. This is *****NOT****** and I repeat * N * O * T * the same as "having already premultipled it with alpha"

Because this is a completely legal premultipled RGB color (1.0,1.0,0.0,0.0).

This is 100% transparent luminiscent yellow.

When composited properly, according to r = fg + bg * (1 - fg.alpha) it ends up ADDING yellow on top of the (unmodified) background. This is a very common workflow that renderers output such channels.

The whole idea that for a zero alpha, RGB data is always zero (or can legally be thrown away) is WRONG. It is a complete misinterpretation of the concept of "premultipled" alpha. It is a total misunderstanding, going so far as some documents claiming this "illegal" completely erroneously.

(For street cred of my position, since that seems necessary here, I had this discussion with Alvy Ray Smith, the guy who *invented* the Alpha channel, and he agrees with me on this).

/Z

[_buck_forester_]

Thank you Zap

[_Dragos_H_Stefan_]

"VFX represents a very small fraction of the people using the EXR file format."
Could you read again what you have written? As one user pointed out, OpenEXR was *created* by this industry, specifically for it's use.
I do suppose that your statement is either a lie to server your argument or simply ignorance, as I'm pretty sure the VFX industry is the most important user of the OpenEXR format. But maybe I'm wrong, so enlighten us and tell us which other industries have a number of users of OpenEXR which is significantly bigger than the "very small fraction" in the VFX industry.
Nobody asks you to break anything.
Use the specs as you see fit, but offer the OPTION to your CUSTOMERS to use the software in the most useful way to do their job.
I don't care about the spec. I want an OPTION in the loader to let me ignore the alpha if I WANT TO. Because *I* know how my pipeline works, not you, neither Adobe.
I'd also like you to show an example of how this addition of this option would break the workflow of someone, who would that someone be and in what industry would he/she be involved in (supposedly with a greater number of user of OpenEXR than the VFX industry).
This is so typical of Adobe arrogance. "Screw the users, we know better and we will force them to break their workflows if it's more comfortable for us". Your users ask your for *options* to make their life easier -- or at least don't make it harder -- (again, they don't ask you to break something) and you hide behind the "spec" paper and tell them to leave you alone.

Dragos

[_erik_winquist_]

chris,

i've read through this entire exhaustive thread now and just want to mention a point that nobody else seems to be bringing up.

I can see your position with regard to CS3+4 treating the "A" channel as transparency by default, but as you have pointed out multiple times, photoshop's EXR plugin doesn't currently support more than RGBA OpenEXRs. if we want to bundle a matte with our RGB channels that does /not/ represent the transparency of the image, the only channel we have at our disposal is that A channel. anything else currently gets thrown away on import.

the suggestion of, "finding a more appropriate format" Chris Cox, "Change in EXR open from CS2 to CS3 can this be fixed?" #80, 3 Feb 2009 7:40 pm
isn't particularly helpful or realistic. visual effects facilities have entire pipelines built around specific image formats and re-structuring those pipelines around an inflexibility in a piece of software that likely makes up a very small corner of that pipeline just isn't going to happen.

the plea here is to--until the day when full multi-channel EXR support makes its way into photoshop--give your VFX user base some options with regard to how that precious "A" is handled. fine, let the default be the "right" way to interpret the data, but don't cut us off at the knees. being flexible is of primary importance in production.

regards,
erik winquist

[_Brendan_Bolles_]

Wow, this thread is pretty intense. I wish someone had pointed me to it earlier. I'm a bit biased, but I agree that ProEXR solves all the alpha problems mentioned here. It was created by someone who works in the VFX industry (me). You can try it free for 15 days.

This is covered in the ProEXR manual and in this thread, but to recap: OpenEXR files use premultiplied images, while Photoshop long ago chose to use straight. Turns out this was an unfortunate choice because, as Florian and others pointed out, any non-black premultiplied pixels with a black alpha will get decimated when converted to straight. Nuke, Shake, and Fusion are premultiplied, so they don't have this problem. I can't think of any advantage to using straight, but it would probably be hard to switch back at this point.

Sometimes VFX artists don't actually want Alpha to be transparency, although that is the standard thing to do and it's the ProEXR default as well. But we also give you a dialog for changing the behavior if you like. You can keep the alpha channel separate and choose whether to un-multiply the RGB. I don't know if Adobe should change what they're doing in their own plug-in, but I'm glad I could swoop in as a third party developer.

If you have other ideas for making ProEXR work better in a film workflow, let me know.

Brendan

[_Christoph_Bolten_]

You seem to be confusing your segment of an industry with the larger audience of Photoshop users. People are using the EXR plugin shipped with Photoshop, in many industries. Only a few have complained. VFX represents a very small fraction of the people using the EXR file format. But Photoshop has to support everyone using the format, in many different workflows, interoperating with many other applications.

Chris, we've been over this before on the CS4 beta forums and it really surprises me how you stick so much to the "EXR File Specs"

Obviously a lot of people are really annoyed that they can't use EXR (a file format created BY the FX industry FOR the VFX industry!) anymore like they did in CS2. First I thought it was just Mental Ray's Problem, but it turns out that Renderman creates the same problems.

I really think it is Adobe's time to move on this and make a little File Open dialog where you can choose how to treat Transparency and Alpha channels.

It doesn't matter if it's different from the EXR specs Chris!
People need to be able to use their renders and pipelines and need this flexibility!

We also use ProEXR sometimes, but it is really sloooooow, so at the moment we have a separate machine with CS2 installed and save all our EXRs as PSB files, which is very annoying too.

Best Regards - Christoph C-:

[_Thomas_Helzle_]

Chris Cox:

- Either make it an option in preferences ("Treat additional channels as transparency or leave them alone")

- Or apply the Alpha as a layer mask, so the user has the option to enable or disable it at will.

The current implementation is destructive and very very very annoying.

And this is true for EVERY single format that supports more than RGB, not only EXR.

I do graphics for 15 years and I NEVER EVER even once wanted the behaviour that Photoshop is showing.

Photoshop it THE melting pot for billions of images from thousands and thousands of sources.
This is not about being right or SPECs etc. (you can't win this uphill battle anyway)...
This is about creativity software.
People want a choice.

Cheers,

Thomas Helzle

[_Alan_Boucek_]

Hi Chris-

This is not a game where you score points by belittling the work of your customers.

The folks who have commented on this thread as a result of cross-posting in the nuke list are not a bunch of drive-by hecklers, and we seem to be doing our best to add some clarity and represent our interests.

The computer graphics and visual effects community does have a sense of ownership over OpenEXR, because it was designed to work well in our pipelines. Adobe's customer base is so large that a seemingly miniscule change like we're talking about can break the things that we depend on. TIFF is a good example of this.

The EXR spec hasn't been updated for a while- probably because the folks who wrote it have moved on to other problems. Perhaps if you're not interested in listening to the concerns of the community, you could simply not support it, and give some breathing room to the plugin developers.