We have decided to take a slightly different approach and moved the conversation over here: https://community.adobe.com/t5/premiere-pro-beta-discussions/discuss-timeline-tonemapping/m-p/133879...
We are excited to announce Tone Mapping in Premiere Pro Beta when using HDR clips in SDR sequences
We are actively seeking feedback on this feature, to make it work for everyone! This feature will also see continuous development in the upcoming days, so please make sure you keep checking it in the new Beta releases.
What is Tone Mapping and why do I need it?
When you create a new sequence using your clips (SDR, HLG, PQ), Premiere pro automatically creates a sequence from the clip’s color space. Let’s say we create a sequence with an iPhone HDR Clip- the sequence’s Working Color Space is now set to Rec 2100 HLG. This is a great way to edit your HDR clips while keeping the full dynamic range and Rec. 2100 colors of the source.
However, many of us want to create a regular SDR sequence (Because many of our friends and family are going to view the video on a regular SDR screen). For that, you would go to Sequence Settings and change the Working Color Space of the Sequence to Rec. 709.
When we change the Sequence’s Working Color Space, Premiere converts the Rec. 2100 colors of the iPhone clip to Rec. 709 colors. But there is a mismatch in the dynamic range between the higher dynamic range clip and lower dynamic range sequence. Because of this, our iPhone clips now look over-exposed with clipped highlights. This is also the case with any HLG/ PQ or color-managed log clips (like Sony log clips).
(Image: iPhone HDR clip in Rec. 709 Sequence. Notice how it looks overbright with clipped highlight details)
(Image: Scopes of the clip in Rec.709 Color space show that most of the highlight details are clipped)
Tone Mapping is critical when converting colors of a media from a larger dynamic range to a smaller dynamic range to obtain a pleasing version and to retain the details of the original media that might have otherwise got clipped in the process of conversion.
Simply put, Tone Map takes all the details from the HDR highlights and preserves them by rolling off into a lower dynamic range, essentially approximating the appearance of HDR content in an SDR sequence.
(Image: iPhone HDR clip using Tone Mapping in Rec.709 Sequence)
(Image: Scopes of above clip when using Tone Mapping in Rec.709 Sequence. Notice how the previously clipped details are now rolled in.)
How does Tonemapping work in Premiere?
With June Beta update (Premiere Pro Beta 22.6) you will see a few changes in Sequence Settings window. When you create a sequence, under Working Color Space you will find a new checkbox called ‘Automatically Tone Map Media’. This is checked ON by default for new Sequences.
With this checkbox ON, whenever you drop an HDR clip (say your iPhone clip) to an SDR Sequence, the clip is automatically Tone Mapped and all your overbright data is brought into Rec. 709 legal range.
(Image: New Tonemapping checkbox under Working Color Space)
Tone Mapping only impacts clips that are of a higher dynamic range than the Working Color Space. SDR clips that are within range are not impacted.
If you check OFF ‘Automatically Tone Map Media’ in Sequence Settings, Tone Mapping will not be applied to any of the clips in the sequence. You can then use LUTs or the Lumetri Color panel to manually dial down the overbright highlights. Be aware, some LUTs may create unwanted artifacts.
Is there a way to modify Tone Mapping applied by Premiere Pro?
Tone Mapping currently is automatically managed by Premiere Pro. With this Beta release, we would love to hear from the community about the following:
Please let us know via comments on this blog post!
What about legacy Premiere Pro files?
Legacy Premiere pro files will open with the Sequence Settings option of ‘Automatically Tone Map Media’ checked OFF. This way, your clips will continue looking as they were in the previous versions of Premiere Pro.
Tone Mapping will not be applied to any clips in the sequence until ‘Automatically Tone Map Media’ in Sequence Settings is checked ON.
Why do some files not get Tone Mapped?
Files which are currently Color-managed in Premiere Pro make use of the Automatic Tone Mapping feature. This includes color spaces of HLG media, PQ media and Sony log in Prores, h.264, HEVC, XAVC codecs.
Some log files are still not color managed in Premiere Pro and will get Color-managed over time. For those clips, older methods of LUT based Tone Mapping will continue working.
When you import and use Non Color-managed clips in a sequence where Tone Mapping is required, Premiere Pro will not Tone Map these clips.
However, If you change the interpretation of the clip and chose to override the Media color space to a Color-managed colorspace, Premiere Pro will automatically start applying Tone Mapping to the clip whenever it is used in a sequence with a dynamic range smaller than that of the clip’s.
WHY OH WHY are you splaying out ALL the new color management (CM) tools all over the freaking app?
You are making a NIGHTMARE for users! No other app is built this way!
We now have CM controls BURIED in Project panel/Modify/Interpret Footage, in the Sequence settings, in the Scopes settings ...
And the placement for this option is SO non-sensical to a user.
You modify clip CM in the Project panel, apply Overrides in the project panel, but ... do Tonemapping in the Sequence panel!
Why is tonemapping stripped off and put elsewhere?
Yea, I'm kinda hacked here, Smit. I like that we're finally getting some user CM controls. And for what you've got done, they work well.
But the implementation is about as horrid as I could design in a nightmare. And yea, @Francis-Crossman , you know my feelings on this! Please, can we have some relief from this crazy design?
An HDR clip inherently does not require Tonemapping. Like the color conversion library, Tonemapping is only required when there is a change of Color spaces, form a higher dynamic range to a lower dynamic range. So to answer the question- How should an HDR media be treated when it comes to a Low dynamic Range sequence? The logic should be goverened by a setting in a Sequence setting, which also happens to be the place where you define the Working color space for the sequence.
However, having said that, you know how much we appreciate your idea of a Color manager panel. We are aware that the color settings are at many multiple places, and we understand the user hassles that come with it. When a Color manager panel is ready, all these settings, including which footage is color managed, and which one's not, should all ideally work from one place.
Thanks for the reply.
And yes, for user sanity let alone working at the speed and precision required in pro workflows, we NEED all CM questions/queries answered with a quick look at ONE panel.
This hunting here and there to find where something might be hidden is so destructive to workflows with deadlines.
One place. Thank you!
One thing badly needed is a way for Premiere to tell users WHAT media is either CM'd or not-CM'd. As of now, it's a guessing game ... and any attempt to find a list will fail.
So ... you note Sony log is 'color managed' ... but is that all flavors of Sony log files created by all Sony cameras? We've had plenty of issues on the shipping forum where one or another Sony log wasn't CM'd but another was.
And what about log formats recorded in an external recorder? Several external recorders offer various log formats ... will those work as a CM format?
What about the many Canon and Fuji and Arri and Red log formats? Others?
You are leaving your users having to constantly guess, which in professional environments is simply not tenable.
Color management is a complex and complicated "thing" and I realize that most of us have limited time and knowledge of the deeper details. So I'll quickly add a couple things here. First, an explanation, and second, the three tools I think we need for working with this moving forward.
Tonemapping of Luminance Values Versus Color Space/Volume Conversions
Tonemapping is typically ... and as utilized within Premiere from Smit's comments ... involving only remapping the range of luminance values. It doesn't normally change the color values other than the normal variation of perceived saturation with brightness changes.
Color Space/Volume Conversion only concerns changing the outside boundaries of the colors/hues available for mapping to the screen (space), along with the implied changes to the number and variations of hues available for mapping to the screen (volume).
So the new Tonemapping ability can auto-change the apparent brightness range of the image as displayed. Bringing down the far greater range of an HDR image to within the narrower brightness range of an SDR image. But the color space/volume conversion is separate to the tonemapping of brightness. And not 'covered' here.
User Controls for Tonemapping
Smit asks if 'we' would like to have some level of control ... and for me that answer is a definite YES please!
There are three controls I would love to have:
With those three controls, we could get the basic shape usable quickly, and finish the job easily in Lumetri.
1. Total Brightness ... this one is pretty explanatory. Essentially equivalent to the Whites control or the right end-point of an RGB curves tool.
2. Highlight Roll-off Slope ... this one is needed as all HDR media is log-encoded, which means it has a logarithmic fall-off built into the code of the device recording the media. They all vary, mostly dramatically. So sticking the users with one roll-off of the highs is going to cause a HUGE problem when dealing with media captured by several different sources.
Having a Highlight Roll-off tool will help us match the upper end characteristics quickly between cameras/devices ... and be able to be batch applied.
3. Gamma adjustment for the middle values would blend in with the Highlight Roll-off tool to give us the shape to match various cameras again. And these tools would make the job of color correction & grading media SO much faster, more even, and more reliable.
There are many wrongs to the otherwise good intentions of this. I share a bit of the frustration @R Neil Haugen has on it.
The current direction to me feels like a band-aid fix to a much more complex subject. I can already see the youtube videos arise titled: "I Fixed My Footage With This Simple Click!"
Is that the market Adobe is really catering to?
Premiere needs to step up when it comes to managing color, that's for sure. But it needs to be done in a robust and most importantly, open system that allows full control over the pipeline by the user, accompanied by proper terminology. Preferably a framework that will allow potential other colormanage system to be added in the future. So no funny auto conversions and questionable tickboxes.
How do I think Premiere should move forward?
1. Remove all forms of auto conversions from the entire program (by default).
That includes you sRGB! I mean look at this. We don't have control over it. This is a 5x2 grid of identical color squares with same filetype vertically. The top row has no profile embedded and sRGB embedded bottom row. Only .jpg and .png get detected and auto converted whilst AE detects .psd .ai and I think .eps too.
2. Unlock all colorspace options for any filetype and even AE dynamic links.
I get it. You want to make it as simple as possibe for endusers but stacking assumptions on assumptions on locked innerworkings of software only makes things more complicated even for the 'noob' to manage if things don't work as intended. By giving us acess to the full list for any file we can at least ourselves decide what gets converted and what not. By default this should all be set to none in my opinion. We can leave the automatic ones for when the proper full colormanagement system is in place to globally switch them for known media.
It could look something like this:
3. Add all 'popular' spaces including working spaces.
If we want to get into providing true options to manage colors we need all the options and not just a limited set of what Adobe might think the users need. This means all camera colorspaces and transferfunctions but also spaces like DaVinci Wide Gamut/Intermediate, ACES 2065-1, ACEScc, ACEScct, ACEScg, Adobe RGB, ProPhotoRGB, Display P3.
3. Dedicate an extra page in the project settings to Color Management.
Time to move that small color management box on the bottom of general project settings to it's deserved dedicated tab. Here everything should be available. This is what I hope to see at some point.
- Color Managent System dropdown menu.
1 - None (greying out or hiding the other settings)
2 - Adobe (Adobe DRT, Adobe Tone Map whatever name works)
- Use auto color management for known media checkbox (switching all to 'from File' like it does now by default)
- Use Tone Mapping (I don't see the point of having this per sequence so should live here. If neccesary a 'Bypass Tonemapping' checkbox sequence settings instead would make more sense but no one that I know of would need HDR->SDR conversion without it)
- HDR Graphics White settings
3 - OCIO
With option to use the 'Environment Variable', 'Same as Project' or 'Custom' config file path.
Same as Project would require the folder to be named OCIO with the config inside and sit right next to the .pproj file. This would be a great option to include configs into projects for interchange and archive!
An OCIO config should substitute the list of color spaces on the media with what the config holds. The sequence color space options should also reflect the configs Display/Views colorspaces.
There's a lot more you could do with OCIO but I'll leave that for another discussion once it's relevant.
4 - future implementations....ACES 2.0, IPP2...etc.)
- 3D Lut Interpolation math
4. A proper full Adobe DRT
Ultimately the current Tone Map implementation should be replaced with a DRT.
From what I can tell in my short testing is that the current tone map solution is very rudimentary and feels like it's born soley out of necessity to do something about files that contain true HDR encodings like Rec.2100 like the iPhone of which people expect it to 'look normal' when they import it. Similar to the 'highlight burn fix' sliders you used to have in 3d renderers. For Sony material it's not effective enough to handle a medium bright blue display (Left: DaVinciDRT, Right: Adobe Tone Map) Sorry for lack of better example at the moment.
This is not enough to serve as robust color managing pipeline which is a shame for the potential Premiere has going this direction. If we can get a pleasing true display rendering transform including the abilty to have an actual Working Color Space before the Output Color Space we can actually get work done. For example copying grades from REDWideGamut/Log3G10 to Sony SLog3/SGamut3.cine would perceptually look similar aside from sensor differences because the desicions were working color space based.
The DRT should also have it's ways of dealing with bright saturated colors in whatever way Adobe sees fit. Most DRTs apply a certain amount of dechroma towards the highlights which we are familiar to with film and is also the easiest way to perceive colors as relatively more bright within the SDR domain.
This may sound like a huge overhaul but to me it's the only way to truly become professional when it comes to managing color. The simple users can still enable "auto everything" if they want and professionals have the tools to adapt the sotware to their workflows.
5. More ideas...
Premiere Pro is quite robust when it comes to playback and it natively operates in 32bit float if I'm not mistaken (or until such effects are applied). Having a clear proper manage system in place opens up doors for more advanced compositing tasks without having to resort to After Effects.
Add Color Space Transform effect.
This would provide the same list as the media color space list with the added option to linearize in or output just like the Color Profile Converter in AE. With this you could for example use blurs and glows or even exposure in true linear math for much more realistic results! It would also make working with a manual setup much nicer. You could even add the option to convert to display space with Adobe DRT/ToneMap. You could achive similar things with Adobes system this way as you would with OCIO.
Add native OCIO effect
We have a free OCIO plugin and BorisFX Sapphire offers one but they're slow. Perhaps if Adobe engineers get in on it they can make it run much faster with Mercury Engine. And native support is always better!
In the end After Effects should also start following suit. It already has an 'okay' system in place but this icc profile based management where even your OS and monitor profiles are listed is just a disaster if you ask me. It also has Arri and Sony profiles but RED's profiles or any others were never added. There could be so much cleanup. Ideally it's fully in-line with how Premiere works for much better integration. For new users this is also a lot simpler to learn just one.
I really hope this post will provide some solid ideas and help steer Premiere Pro in a direction that benefits both beginners and professionals in the industry. Now is the time to change and now is the time to do it right!
At NAB in April, they said there's a lot more color management coming and of course, no date for when we might expect it. Or hints as to what. So it's definitely a "work in process".
Which makes using this now a bit less enjoyable than any of us would like.
I tend to like most of your suggestions, as well ... one would expect ... 😉
Thank you for developing this feature! Color management is now usable with Sony LOG HDR footage!
A development I'd like to see:
When display color management is active, LOG footage in the project panel and source viewer will not have tonemapping when viewed on an SDR screen, but when dropped on an SDR sequence with "automatically tonemap media" selected the tonemapping will correct for exposure.
With display color management inactive, log footage will still look like uncorrected log footage in the source viewer.
I wish there was a project setting that could tonemap outside of a sequence. If I don't have an HDR display I don't want to see the footage in HDR form if I'm looking at it in the project outside of a timeline.