Change in EXR open from CS2 to CS3 can this be fixed?

[_progress_]

It seems the monkeys have been at the file formats again...!

Open an exr with an alpha in CS2 and the image displays normally and the alpha is retained.

Open an exr with an alpha in CS3 and the alpha channel is applied to the transparency and then lost... which is really STUPID considering you might apply 0 alpha values to parts of the image you retain visually, as you might just want to use the alpha to drive an effect and not just be myopic and think it's just for transparency.

So, can this be fixed? I can't see any info on it?

Will CS2 non intel plugin work on an intel system in CS3

If not, effectively PS is useless for exr work for us.

Or is this fixed in CS4?

[_jonah_]

holy cavalry batman!

Thanks for the backup from the pros!

Chris is one stubborn dude. Good luck!

[_Konstantin_Hristozov_]

The time wasted arguing here could've been spent implementing all the remaining useful features of OpenEXR.

[_Brendan_Bolles_]

"We also use ProEXR sometimes, but it is really sloooooow, so at the moment we have a separate machine with CS2 installed and save all our EXRs as PSB files, which is very annoying too."

Christoph, I'm aware of the speed issues when dealing with big images. The slowness is actually not within ProEXR itself but with Photoshop when I request/provide pixels. You can verify this by checking your CPU meter in those instances and seeing that none of the processor cores are very active. ProEXR is currently optimized to use as little memory as possible, but perhaps that's not meshing with however Photoshop works internally.

I'm in the process of experimenting with different combinations of memory usage and pixel region requests - hopefully it'll yield some improvements.

I'd welcome any help from Adobe on this one!

Brendan

[Neil_Keller]

Travis,
>C:\Program Files\Adobe\Adobe Photoshop CS3\Plug-Ins\File Formats

Note that this is a Macintosh forum.

Neil

[_Travis_Lewis_]

Apologies!

[Chris Cox]

Brendan - you should be working in larger tiles (256 rows, or 256x256 makes a good starting point) instead of writing a row or a small area at once. The fewer times your plugin calls back into Photoshop, the faster it'll run. (we have a lot of cross DLL overhead to deal with, plus error checking on each and every call)

[Chris Cox]

Zap - Please read some file format documentation. Your discussion on premultiplication is mostly wrong (with just enough right to confuse people who aren't familiar with the math).

[Chris Cox]

Photoshop supports more blending options and filter options than most packages -- and for what we do, unmultiplied data is much faster in the majority of cases than using premultiplied data. (otherwise you spend a lot of time un-multiplying, operating on the color data, and remultiplying -- which some other applications do)

With recent (1994 or later) CPUs, compositing operations are almost entirely limited by DRAM bandwidth or cache bandwidth and not by calculations - the small amount of calculation saved by premultiplication is irrelevant.

[Chris Cox]

If you want to change the definition of OpenEXR - you shoud seek a change the official spec. We implemented OpenEXR support in a way consistent with the spec. to ensure interoperability. That spec. exists to make sure that there is no confusion in interpreting the contents of the file.

You are asking me to add confusion to the interpretation of every EXR file, to throw away interoperability. With your change, nobody using an OpenEXR file would know whether the A channel should be opacity or an unassociated alpha, or whether the A channel was premultiplied or not. If they guess wrong, things will break (and they only have a 1 in 4 chance of getting it right). Your request would break existing workflows that rely on the specification, that use the intended interpretation of the data.

You are asking me to add extra UI on opening and saving an EXR file that will slow down the workflow of anyone using the OpenEXR file format (and probably confuse the blazes out of those not in the VFX industry).

I need a *REALLY* good list of reasons to make a change that affects so many users, especially when the change is only benefitting a relatively small number of users. And even more so when you are clearly ignoring standards or misusing the tools ("But I want my screwdriver to drive nails better"). If you have good reasons, please communicate them. But you've got to stop the drive by postings and "because we said so" arguments - they're still doing your cause more harm than good.

[_Brendan_Bolles_]

BTW, everyone, you can get back the functionality of the Over mode found in Nuke/Shake by manually reconstructing the transfer mode with two layers. So with an RGBA EXR you'd:

1. Open with ProEXR, using the option to put the A channel on a separate layer. It should appear beneath the RGB layer.

2. Put your background layer beneath these two layers.

3. Invert the A layer and set the transfer mode to Multiply.

4. Set the RGB layer to Linear Dodge (Add).

And there you have the exact math Florian described for compositing premultiplied images. Now, this approach can definitely be fraught with problems and headaches dealing with two layers, but it can come in handy in a pinch.

After Effects has the Luminescent Premul transfer mode to do the same thing. It also can be problematic, especially when you try to use plug-ins that are assuming a straight alpha.

Brendan

[_buck_forester_]

Chris, would you mind explaining how a simple option (default being it's current implementation) will make matters worse? Or how it will confuse everyone?

[_Konstantin_Hristozov_]

I think one solution to the "breaking the workflow" scenario would be to have a button/tab called "Advanced": If you don't know anything about EXR, then you probably should not use the "Advanced" feature. Many applications out there use this approach. Perhaps it's worth considering?

Cheers,

Konstantin

[Chris Cox]

Konstantin - A semi-hidden advanced option might work, and at least minimize the confusion on the UI side.
Right now there is no UI for OpenEXR in Photoshop, so just adding a UI will slow down some workflows (especially those that are currently automated and don't want to stop for a UI). BUT, I'm adding UI for saving different bit depths and other new EXR options - so that issue isn't a big deal.

The bigger issue is the interpretation of the file contents, and how that works when sharing the file with other users (especially ones at another company or site).

[_sam_cuttriss_]

I think anyone twisted enough to click the advanced exr tab will be comfortable being personally held responsible for the outcome.
I am confident the only impact on the industry will be more smiles.

Thank you for considering this option.
Would you mind sharing your intentions with the remainder of the exr spec implementation you mention above?

[_Graham_D_Clark_]

"A semi-hidden advanced option might work, and at least minimize the confusion on the UI side. "-Chris Cox
Could we enable what we want through the Photoshop API, via a script? If it can be implemented earlier than a new gui element, that would work at Warner Bros and maybe for a few others, for now until the hidden gui is available.
Thanks for helping us, (not my first post in pshop forums) Graham

[_mrrarfs_]

The big post houses who have had the most experience so far with the .exr format use tools that require their artists to use a transparency channel as data in whatever way they require.

There are plenty of situations where for artistic / technical / economic reasons an artist does not want to mDiv a plate by the transparency/alpha channel or wants to use a transparency/alpha channel embedded in a different .exr

Chris is right that maybe us VFX guys should have two channels, one for transparency and one for alpha to avoid confusion/mistakes i.e. for those that don't understand the multiplication/division process. But implementing this in a workflow or comp would just be messy and expensive.

It matters not if the .exr has beens saved 'to standard' but if the workflow for that artist/post house gets the best result with the available resources. We want to use Photoshop, as it is the best paint tool on the planet, but we cant because it breaks our workflow. All of our tools let us save the channels in whatever state we choose to get the job done, and consequently we need that same choice when opening up an .exr

This thread has done the rounds in many of the big post houses now and I don't see any of those recognized artists come forward and tell us we are all talking rubbish here, just people like Chris and Zap trying to clear up some common misconceptions about various definitions.

[_erik_winquist_]

chris,

>I need a *REALLY* good list of reasons to make a change that affects so many users, especially when the change is only benefitting a relatively small number of users.

this thread has made me really curious.. the question has been asked a couple times now and not answered. among adobe's customer base, who is using OpenEXR with photoshop besides the VFX community?

i'm asking this with complete sincerity, not trying to be snarky. other than education/research, what other sectors are making use of the format that makes the VFX community such a tiny slice of the pie?

regards,
erik

[_zDarius_]

Can someone from Adobe with a personality and the ability to communicate in a lively discussion without being a smart ass please chime in. There are several points here that Chris is choosing to ignore. First, no one asked you to change the implementation of OpenEXR in PhotoSlop. And secondly, everyone is asking you to change it back and you are telling them they are all a bunch of idiots and they do not know what they are talking about. Can Adobe please stop screwing up the application every time you release one of these overpriced updates that should be a free online update? All you are doing is destroying the existing workflow that we have all spent years developing in order to justify these so called updates, and doing it without the users requests or input.

Please, someone else from Adobe reply to this thread, as we do not need one more redundant reply from Chris Cox telling us we do not know our you know whats from a hole in the ground. Instead we need answers and measures to address the problem, possibly an option within the Adobe implementation that allows a check box or options. At the very least we need to be listened to, because this stubbornness and arrogance is really making me wish I had competing options of software.

[Chris Cox]

Erik;

Photographers are using OpenEXR for HDR photo work (someone told them it was the best format for HDR), 3D artists are using it for some textures and final renders (not the same as VFX, and so far they use the transparency as per the spec.), some studios have standardized on OpenEXR files (but most use it as defined in the spec.), and some effects houses have also standardized on OpenEXR - but some use it according to the spec. and some don't.

There are other market segments experimenting with OpenEXR, but that haven't standardized on it that I know of (astronomy, medical, etc.). Most of those would probably be better served with a different format, and the number of users isn't huge.

Basically, OpenEXR was described as the answer for all HighDynamicRange image needs - and people have been adopting it whenever they have HDR data. Some people find EXR too limiting and go to TIFF, a few are still using Radiance files, and those just curious about HDR and using Photoshop can stay with the PSD/PSB format. (and researchers just write their own file format for each new project anyway) The ILM name has helped "sell" OpenEXR to people well outside the VFX and movie/video industries.

I don't have hard numbers for EXR users - HDR usage is growing and changing rapidly. But we have done a lot of customer visits, we talk to a lot of customers, and try to keep up with forums, publications, newsletters etc. to see what people are doing and using. From that I have estimates on the sizes of the various market segments using the EXR format. I wish I had real market data I could throw around - but I don't do marketing, and it's not like cameras (or software) where I can just check someone's sales figures.

Sigh.

The bad part of working on Photoshop is that we have to think about how changes we make will affect many people across many industries and market segments. There are lots of changes that I would like to make, that make sense to me, but would confuse the blazes out of almost everyone else (well, other than Stu). So, either I find a better way, or don't do it.

Let's see if an analogy will help: when you're canning fruit for yourself and a few dozen friends - you can take shortcuts and nobody will care. But when you're canning fruit for millions of people to consume - you have new rules, regulations, insurance, and a lot of other things to take into consideration if you want to stay in business and out of jail.

I don't mean to belittle anyone here -- I'm just trying to help you understand my perspective on this.

[Ramón G Castañeda]

zDarius,

Adobe staff does not regularly monitor the Adobe user-to-user forums. When members of the Adobe team show up here from time to time, they do so as volunteers and on their own time.

We are very fortunate in this particular forum to have Chris Cox contributing after a long absence of about a year. I don't recall any one higher in the Adobe chain of command than Chris ever participating here in the last six yearswith the possible exception of an isolated post or two by someone else about three or four years ago.

If you want to address Adobe, use the Contact link at the top of this page.

Personally, I don't give a damn about EXR, Open EXR, "the VFX community" or anything related to video or the film industry. As far as I'm concerned, your whole "industry" could disappear without my ever noticing it. However, I do have a keen interest in preventing this forum from becoming a hostile environment in which Chris Cox is not inclined to participate any longer. He has been of great assistance in the past.

Clearly, this is not the venue to spew your ill mannered, insulting and idiotic comments. As I said, contact Adobe directly or use the Feature Requests section of this forum if you think you are capable of expressing yourself like a thinking person.

[Edited formatting.]

[Guest]

Considering the strong feelings, I thought it was reasonably polite up to Post #117!

[_Zap_Andersson_]

Somehow the quote "I reject your reality and substitute my own" comes to mind here. :)

No, my explanation of alpha channels is not "wrong" in any way, shape, or form. If it is, please point out my error, and back it up with documentation that it is in error. And the incorrect TGA format documentation that tends to accompany some Adobe products most *certainly* do not count.

The reason Adobe is very stubbornly turning a blind eye to proper premultiplied behaviour, is because they use a straight alpha pipeline internally. This pipeline is *incapable* of representing a color such as premultiplied (1,1,0,0) (the "additive yellow" I mentioned in last post). Due to this complete inability to even represent this, they have a vested interest in defining it out of existence and claiming it "wrong", while it is not. Which I bet accouts for 90% of the stubbornness preceived in this thread.

Adobe products use a "straight" alpha (where alpha is considered more as a "mask" than opacity) because they stem from a paint tool background, where a "mask" is something you create out of pixels that are already there.

With that mindset, saying you have "multiplied with the alpha" in a premultiplied file makes sense.

But from the context of a renderer, this makes no sense. In a renderer, it's about taking samples off some geometry. In the simple case of opaque objects, lets say we take 16 samples in a pixel. 12 of these hit an opaque yellow (1,1,0,1) object, and 4 of these hit the background (0,0,0,0).

When summing up these samples, we get a color that is (0.75,0.75,0,0.75). This is not because we have "multiplied with alpha", it is simply because there was a *coverage* of an entity with an alpha of one in 75% of the pixel.

Now assume instead that the background was red (1,0,0,0), which is quite legal. The downsampled pixel would have a color of (1,0.75,0,0.75). And pixels where NO samples hit would have a color of (1,0,0,0).

While this may not be so useful for traditional compositing, it can be useful for all sorts of other reasons. For example, a 3ds Max render always sets the alpha to 0 in pixels that are "background", even if they are filled with, say, a sky, or anything else. So there is completely legal RGB data there, but with an alpha of zero.

If this is put into an EXR, and loaded by Photoshop, this sky is now lost forever. So data is clearly destroyed in the process.

We furthermore have these luminiscent pixels. Go back to our object-against-a-background issue, but instead of making the object just yellow, lets make it luminiscent yellow, i.e. (1,1,0,0). When this object covers 75% of the pixel, and the background is transparent black (0,0,0,0), the final pixel will be 0.75,0.75,0,0.

This means that when this color is comped on top of the underlying layer, the premultiplied "over" operation, which is...

r = fg + bg * (1 - fg.a)

...will work out to an add, since the alpha is zero, and hence (1-alpha) is (1-0), e.g. 1, so the math boils down to

r = fg + bg

The fact is that photoshop simply cannot do this, because it is "straight alpha" internally.

After Effects is kind of limping along here with the "Luminiscent Premultiplied" mode, but it is still a hack and breaks down any time you add even the slightest effect, since the AE effects pipeline is also, unfortunately, straight alpha.

Straight alpha has a few merits (notably with multiplicative post math and in color correction), but ends up being needlessly complex (compared to the premultiplied math) for lots of other operations like blurs, filtering, etc. The math in the premultiplied case can simply treat the alpha as another channel, and apply the exact same operations to it as it does to RGB, for things like blurs, convolutions, whatnot. A "straight alpha" system has a much much harder time here.

But hey. What do I know? Apparently I've been "defined wrong" in this discussion, even though I have the inventor of the alpha channel on my side.

/Z

[_Dragos_H_Stefan_]

>Personally, I don't give a damn about EXR, Open EXR, "the VFX community" or anything related to video or the film industry.

You're in the wrong thread and off-topic, troll.

If there is one hostile person in this thread, it's Chris Cox.
Mr. Cox, I still want to see some *clear* statement about industries which are bigger users of OpenEXR and would be negatively affected by the inclusion of an option or even of a preference.
You were very specific that changes in the workflow would greatly affect a much larger number of users than the VFX industries yet in the post were you were supposed to clarify this the information was very vague: "some market segments are experimenting", "I don't have hard numbers", "some studios, of which some use it as in the spec and some not","texture artists". And photographers interested in HDR, this being indeed probably a significant community (I'm talking about numbers).
I fail to see how a preference or dialog box on open (and with a checkbox "Don't show again") can affect in a bad way the workflow of anyone you mentioned.

Dragos

[Neil_Keller]

This topic, although quite heated with comments from many first-time visitors, has none-the-less been pretty civilized and articulate until post #117.

It is rare to have someone as directly involved in Photoshop development as Chris Cox appear and respond as he has here. Whether you agree with his viewpoints or not, he's a valuable asset to these forums, particularly to the many who have been regular participants over the years.

For this topic, everyone is welcome to make his or her point so Chris can use them to evaluate any future changes in the app. It's to no one's benefit to make him feel unwelcome.

So please, let's keep the topic to the point and name-calling and accusations out of here. I do not want to be put in the position of closing this long topic.

Thanks!

Neil
Forum Host