DISCUSS: Tone Mapping in Premiere Pro Beta

Thank you for developing this feature! Color management is now usable with Sony LOG HDR footage!

There are many wrongs to the otherwise good intentions of this. I share a bit of the frustration @R Neil Haugen has on it.

The current direction to me feels like a band-aid fix to a much more complex subject. I can already see the youtube videos arise titled: "I Fixed My Footage With This Simple Click!"
Is that the market Adobe is really catering to?

Premiere needs to step up when it comes to managing color, that's for sure. But it needs to be done in a robust and most importantly, open system that allows full control over the pipeline by the user, accompanied by proper terminology. Preferably a framework that will allow potential other colormanage system to be added in the future. So no funny auto conversions and questionable tickboxes.

How do I think Premiere should move forward?

1. Remove all forms of auto conversions from the entire program (by default).

That includes you sRGB! I mean look at this. We don't have control over it. This is a 5x2 grid of identical color squares with same filetype vertically. The top row has no profile embedded and sRGB embedded bottom row. Only .jpg and .png get detected and auto converted whilst AE detects .psd .ai and I think .eps too.

2. Unlock all colorspace options for any filetype and even AE dynamic links.
I get it. You want to make it as simple as possibe for endusers but stacking assumptions on assumptions on locked innerworkings of software only makes things more complicated even for the 'noob' to manage if things don't work as intended. By giving us acess to the full list for any file we can at least ourselves decide what gets converted and what not. By default this should all be set to none in my opinion. We can leave the automatic ones for when the proper full colormanagement system is in place to globally switch them for known media.
It could look something like this:

3. Add all 'popular' spaces including working spaces.
If we want to get into providing true options to manage colors we need all the options and not just a limited set of what Adobe might think the users need. This means all camera colorspaces and transferfunctions but also spaces like DaVinci Wide Gamut/Intermediate, ACES 2065-1, ACEScc, ACEScct, ACEScg, Adobe RGB, ProPhotoRGB, Display P3.

3. Dedicate an extra page in the project settings to Color Management.
Time to move that small color management box on the bottom of general project settings to it's deserved dedicated tab. Here everything should be available. This is what I hope to see at some point.

- Color Managent System dropdown menu.
1 - None (greying out or hiding the other settings)
2 - Adobe (Adobe DRT, Adobe Tone Map whatever name works)
- Use auto color management for known media checkbox (switching all to 'from File' like it does now by default)
- Use Tone Mapping (I don't see the point of having this per sequence so should live here. If neccesary a 'Bypass Tonemapping' checkbox sequence settings instead would make more sense but no one that I know of would need HDR->SDR conversion without it)
- HDR Graphics White settings
3 - OCIO
With option to use the 'Environment Variable', 'Same as Project' or 'Custom' config file path.

Same as Project would require the folder to be named OCIO with the config inside and sit right next to the .pproj file. This would be a great option to include configs into projects for interchange and archive!
An OCIO config should substitute the list of color spaces on the media with what the config holds. The sequence color space options should also reflect the configs Display/Views colorspaces.

There's a lot more you could do with OCIO but I'll leave that for another discussion once it's relevant.
4 - future implementations....ACES 2.0, IPP2...etc.)

Genral settings
- 3D Lut Interpolation math

4. A proper full Adobe DRT
Ultimately the current Tone Map implementation should be replaced with a DRT.
From what I can tell in my short testing is that the current tone map solution is very rudimentary and feels like it's born soley out of necessity to do something about files that contain true HDR encodings like Rec.2100 like the iPhone of which people expect it to 'look normal' when they import it. Similar to the 'highlight burn fix' sliders you used to have in 3d renderers. For Sony material it's not effective enough to handle a medium bright blue display (Left: DaVinciDRT, Right: Adobe Tone Map) Sorry for lack of better example at the moment.

This is not enough to serve as robust color managing pipeline which is a shame for the potential Premiere has going this direction. If we can get a pleasing true display rendering transform including the abilty to have an actual Working Color Space before the Output Color Space we can actually get work done. For example copying grades from REDWideGamut/Log3G10 to Sony SLog3/SGamut3.cine would perceptually look similar aside from sensor differences because the desicions were working color space based.
The DRT should also have it's ways of dealing with bright saturated colors in whatever way Adobe sees fit. Most DRTs apply a certain amount of dechroma towards the highlights which we are familiar to with film and is also the easiest way to perceive colors as relatively more bright within the SDR domain.

This may sound like a huge overhaul but to me it's the only way to truly become professional when it comes to managing color. The simple users can still enable "auto everything" if they want and professionals have the tools to adapt the sotware to their workflows.

5. More ideas...
Premiere Pro is quite robust when it comes to playback and it natively operates in 32bit float if I'm not mistaken (or until such effects are applied). Having a clear proper manage system in place opens up doors for more advanced compositing tasks without having to resort to After Effects.

Add Color Space Transform effect.

This would provide the same list as the media color space list with the added option to linearize in or output just like the Color Profile Converter in AE. With this you could for example use blurs and glows or even exposure in true linear math for much more realistic results! It would also make working with a manual setup much nicer. You could even add the option to convert to display space with Adobe DRT/ToneMap. You could achive similar things with Adobes system this way as you would with OCIO.

Add native OCIO effect
We have a free OCIO plugin and BorisFX Sapphire offers one but they're slow. Perhaps if Adobe engineers get in on it they can make it run much faster with Mercury Engine. And native support is always better!

In the end After Effects should also start following suit. It already has an 'okay' system in place but this icc profile based management where even your OS and monitor profiles are listed is just a disaster if you ask me. It also has Arri and Sony profiles but RED's profiles or any others were never added. There could be so much cleanup. Ideally it's fully in-line with how Premiere works for much better integration. For new users this is also a lot simpler to learn just one.

I really hope this post will provide some solid ideas and help steer Premiere Pro in a direction that benefits both beginners and professionals in the industry. Now is the time to change and now is the time to do it right!

Color management is a complex and complicated "thing" and I realize that most of us have limited time and knowledge of the deeper details. So I'll quickly add a couple things here. First, an explanation, and second, the three tools I think we need for working with this moving forward.

Tonemapping of Luminance Values Versus Color Space/Volume Conversions

Tonemapping is typically ... and as utilized within Premiere from Smit's comments ... involving only remapping the range of luminance values. It doesn't normally change the color values other than the normal variation of perceived saturation with brightness changes.

Color Space/Volume Conversion only concerns changing the outside boundaries of the colors/hues available for mapping to the screen (space), along with the implied changes to the number and variations of hues available for mapping to the screen (volume).

So the new Tonemapping ability can auto-change the apparent brightness range of the image as displayed. Bringing down the far greater range of an HDR image to within the narrower brightness range of an SDR image. But the color space/volume conversion is separate to the tonemapping of brightness. And not 'covered' here.

User Controls for Tonemapping

Smit asks if 'we' would like to have some level of control ... and for me that answer is a definite YES please!

There are three controls I would love to have:

1. Total Brightness
2. Highlight Roll-off Slope
3. "Gamma" Adjustment for the Mid-Range Values

With those three controls, we could get the basic shape usable quickly, and finish the job easily in Lumetri.

1. Total Brightness ... this one is pretty explanatory. Essentially equivalent to the Whites control or the right end-point of an RGB curves tool.

2. Highlight Roll-off Slope ... this one is needed as all HDR media is log-encoded, which means it has a logarithmic fall-off built into the code of the device recording the media. They all vary, mostly dramatically. So sticking the users with one roll-off of the highs is going to cause a HUGE problem when dealing with media captured by several different sources.

Having a Highlight Roll-off tool will help us match the upper end characteristics quickly between cameras/devices ... and be able to be batch applied.

3. Gamma adjustment for the middle values would blend in with the Highlight Roll-off tool to give us the shape to match various cameras again. And these tools would make the job of color correction & grading media SO much faster, more even, and more reliable.

Neil

Smit,

One thing badly needed is a way for Premiere to tell users WHAT media is either CM'd or not-CM'd. As of now, it's a guessing game ... and any attempt to find a list will fail.

So ... you note Sony log is 'color managed' ... but is that all flavors of Sony log files created by all Sony cameras? We've had plenty of issues on the shipping forum where one or another Sony log wasn't CM'd but another was.

And what about log formats recorded in an external recorder? Several external recorders offer various log formats ... will those work as a CM format?

What about the many Canon and Fuji and Arri and Red log formats? Others?

You are leaving your users having to constantly guess, which in professional environments is simply not tenable.

Neil

WHY OH WHY are you splaying out ALL the new color management (CM) tools all over the freaking app?

You are making a NIGHTMARE for users! No other app is built this way!

We now have CM controls BURIED in Project panel/Modify/Interpret Footage, in the Sequence settings, in the Scopes settings ...

And the placement for this option is SO non-sensical to a user.

You modify clip CM in the Project panel, apply Overrides in the project panel, but ... do Tonemapping in the Sequence panel!

Why is tonemapping stripped off and put elsewhere?

Yea, I'm kinda hacked here, Smit. I like that we're finally getting some user CM controls. And for what you've got done, they work well.

But the implementation is about as horrid as I could design in a nightmare. And yea, @Francis-Crossman17221443 , you know my feelings on this! Please, can we have some relief from this crazy design?

Neil