[Released] Timeline tone mapping

[eric escobar]

This feature is now shipping in v23.2 of Premiere Pro. More information is available here.

------------

In this latest public beta, Premiere Pro introduces automated tone mapping for HDR (High Dynamic Range) video to SDR (Standard Dynamic Range) video workflows.

Tone mapping works with iPhone HLG, Sony S-Log, Canon Log, and Panasonic V-Log media, as well as media that is in HLG and PQ color spaces.

TWO STEPS TO TONEMAPPING

Tone mapping is done in Two-Steps. First, you set the clip’s color space to the wide gamut color space it was recorded in (like Sony S-Log) in the Interpret Footage dialogue.

Then you must drag that clip to a Rec. 709 timeline. Tone mapping will automatically be applied, but you can confirm that “Automatically Tone Map Media” is checked in the Sequence Settings:

SOME CLIPS INTERPRET AUTOMATICALLY, SOME REQUIRE YOUR INPUT

iPhone HLG clips are automatically detected and interpreted on import and you don’t have to do anything, just drag and drop them on to a Rec. 709 timeline.

For Sony XAVC S-Log media, you can enable Automatic Detection and interpretation with “Log Color Management” in File > Preferences > General and checking “Log Color Management”:

All other log media must be enabled manually by overriding the color space in the Interpret Footage Dialogue.

COLOR MANAGEMENT GOING FORWARD

Tone mapping currently works with iPhone HLG, HLG, PQ, Sony S-Log, Canon Log and Panasonic V-Log formats.

Currently, Automatic Detection is supported only on iPhone HLG, HLG, PQ and Sony XAVC S-Log media. Other HDR and log formats will be supported in future beta releases.

Tone mapping HDR source media to SDR timelines is one part of a larger color management system in development for Premiere Pro that is being introduced and validated through the beta program.

This workflow offers a LUT-free way to transform your media from HDR to SDR. This is a first step, and more functionality and control will be added.

You can, of course, continue to work with log footage and LUT-based color workflows. This new method is another, alternative color workflow model that exists alongside it.

We want to know what you think.  Please join the conversation below.

[Baffy19]

@eric escobar 

"It is not necessary to work in Rec 709 for HDR delivery".

According to your words, HDR does not need to be interpreted, but only work on the correct sequence of HLG or PQ. And if the HDR material is output to SDR for the end user?
And the second question that worries me the most. When working with the LOG source, can I do without the initial stage of interpretation in the Modify panel to another REC 709 color space? Why am I asking, because I want to work in the original color wide LOG space for more flexible color manipulation, and convert the source itself to REC 709 in the Lumetri panel?

[eric escobar]

The HDR source has to be interpreted as HLG or PQ, which Premiere does automatically. In the beta, you can interpret Sony, Panasonic and Canon log clips and they will display properly in a wide gamut timeline so they are also HDR sources.

What do you mean by "another Rec 709 color space"? Is there some variant of the Rec 709 color space you need for your workflow? What is it?

There's no way, currently work color in log space in Premiere Pro. You can use the existing manual method of working with a log clip values in Rec 709 using Lumetri, and then placing an adjustment layer above the clip and doing the log to rec 709 transform using a camera manufacturers recommended LUT. This way if you want to preserve all your original color data for grading in a separate application like Resolve or Baselight, you can just disable the Adjustment Layer and export the project with all original color data preserved.

Does that help?

[Baffy19]

There's no way, currently work color in log space in Premiere Pro. You can use the existing manual method of working with a log clip values in Rec 709 using Lumetri, and then placing an adjustment layer above the clip and doing the log to rec 709 transform using a camera manufacturers recommended LUT.

By @eric escobar

FINALLY. I got my answer. Thank you very much. That's exactly what I wanted to hear and that's how I work. This process is correct from the point of view of using the full potential of the LOG space before its premature transformation into the REC 709 color space.

[Baffy19]

What do you mean by "another Rec 709 color space"? Is there some variant of the Rec 709 color space you need for your workflow? What is it?

By @eric escobar

As is known from the science of color, the camera of each individual manufacturer takes pictures in its own color profile, whether it is: LOG, HLG, BRAW, ARRI, RED. Hence, everyone has their own color space, some wider, some narrower. As we know REC 709 is the smallest color space. Therefore, I don't want to work with it on the timeline at the initial stage in order to be able to pull colors from a wide color space, unlike REC 709. If we interpret the material at the initial stage, before placing the source on the timeline, then we lose this opportunity and narrow the color space LOG, BRAW, etc. and do not have flexibility in color, since we work in a narrow space REC 709. I hope I explained clearly that there are color spaces different in scale from REC 709. That is why the interpretation must be done manually through the manufacturer's LUT at the last stage in the chain of color correction of the material.

[eric escobar]

got it! I just misunderstood the sentence. Yes, the modern color workflow is a process of manipulating and transforming of wide color spaces of cameras to the narrower color space of rec 709. The goal is always to preserve detail and information in that process. And there are many ways to do it. We are working towards optimizing for that daily!

[Baffy19]

Excellent. Now there is confidence in the correctness of working with color. I would also like to add the OFFSET wheel to the LUMETRI panel. He is sorely missed. Is there any way to take note of this issue?

[Chetan Nanda]

Hi @Baffy19 

I am from Premiere Pro color team, and following the discussion on this thread. 

There are multiple school of thoughts going on here, and we need these kind of inputs from our direct users to build robust, useful and easy color management tools inside Premiere and other Adobe products.

We would be happy to jump over a call to discuss and understand more.

As @eric escobar Eric has already explained the state of color management in Premiere, I would like to shed some more details on inner working. 

Premiere Pro is able to automatically detect the color space of media, this is true for formats like H264/HEVC (wrapped in mp4), ProRes wrapped in .mov and more. We call them color managed media.

On import side, Premiere has two way to interpret and convert media color space to timeline color space.

1. Override color space option in interpret footage : 

• This is not needed if media is tagged correctly and Premiere is able to read the correct color space of media via reading metadata from file.
• If media is tagged wrongly, then user use override option to override media color space to correct one. In this workflow, media will be interpreted as new color space by rest of the application.

Premiere pro will convert from interpreted color space to timeline color space without applying LUT. It internally uses mathematical conversion to convert to 32bit float timeline and no clipping is performed.

2. Input LUT in Interpret footage

This is the alternative workflow, which is LUT based. i.e. if user has their LUTs to convert from say HLG to Rec. 709 and they do not want to use Premiere Pro conversion then they can use Input LUT option to do the conversion.

Working color space:

Along with that Premiere supports following working color spaces:
- Rec. 709 - for SDR production.

- Rec. 2100 HLG - wide color gamut timeline for HDR production.

- Rec. 2100 PQ - wide color gamut timeline for HDR production.

So, When an media imported into Premiere Pro and we directly create a sequence from it, then it assign a working color space to sequence automatically, if the media color space is same as one of the supported timeline color space, then it uses media color space as timeline color space.

If media color space is not from the one of the supported color spaces, then by default Rec. 709 timeline is created.

So, e.g. if log footage imported and then Premiere Pro created Rec. 709 timeline as default, as Log is not one of the supported timeline color space. User can go and change the timeline color space from SequenceSetting dialog.

Again, We would be happy to jump over a call to discuss and understand more.

Thanks,

Chetan Nanda

[Baffy19]

Thank you @Chetan Nanda for joining the conversation.

I just wanted to clarify that when working with LOG, BRAW, ARRI, RED, HLG, I do not use the interpretation of the material, but go to the LUMETRI panel to interpret the source. Why am I doing this. This is because if we interpret the material by automatically redefining the color space from the Modify panel, then we lose the ability to influence the image before it is transferred to the REC 709 space. That is, we move from a larger space to a smaller one. Consequently, we lose flexibility in pulling colors and giving the desired appearance with full dedication. If we interpret the material through the LUMETRI panel, then we have the opportunity to play with the full range of colors and brightness in the frame, because we will be able to do all the primary color correction in a wide range of colors before developing and transferring to the same space. And then, by the third and fourth stage, give the frame the desired appearance and style. Step by step it turns out like this:

1 - we throw the source on the timeline;

2 - we develop the material with the use of the camera manufacturer's LUT in the LUMETRI panel;

3 - use the standard Lumetri effect and move it above the specified development LUT in the Effect Controls panel. Here you can use several Lumetri effects to make primary and secondary color correction (local);

4 - use the new Lumetri effect and put it below the developing LUT in the Effect Controls panel and create the desired appearance;

5 - throw the new Lumetri effect into the Effect Controls panel in the latest and style the material as desired.

It is according to this scheme that we work with a wide range at the initial stage and control the process of all the color correction chains.

[R Neil Haugen]

Glad to see another color dev popping in!

Baffy ... you do NOT get the original media into Lumetri in the workflow you describe within Premiere at this time. Why? Because all sequences have color management set for the sequence.

So if you have a Rec.709 sequence, that media is either transformed prior to landing on it ... or you have data outside the sequence color space that will not be properly handled on export.

Like you, I would prefer a working space separate from display & export. Which if as I've suggested, was P3/D65 in a Rec.2020 container, would 'hold' all currently practical color spaces easily. With good transforms to all smaller ones already available.

But that ain't what Premiere can do ... so thee & me will keep at the devs to give us that! Yep.

In the meantime ... their process does work, you do not lose data. That's been tested thoroughly by a couple colorists who have both the kit & training/experience in CM far past you and me.

Is it the process those colorists would have chosen? No. Not hardly.

But in both testing and theory, can it work to get the larger data into the smaller space? Yes, both in theory and in practice, it does work.

I think the key is mentioned above, which is converting the data to the 32 bit float, as then it can be scaled in high-decimal point math. In testing, everything from RED raw, Arriraw, BRAW, S-log variants in wide-gamut origin ... all come through the conversion to Rec.709 without clipping, crushing, or data loss.

And this is clearly a mathematical process they are applying, not a LUT.

So I go back to what the one said to me ... this does work correctly, in all tests, but ... it's weird how they named things. And he agrees the splaying of the controls all over the frickin' app with names that do not give an idea of what they actually do ... nor any explanatory tool tips! ... is danged confusing.

Even to a pro colorist.

But the process you are trying to use cannot work in Premiere at this time. Physically impossible.

You might be able to apply a change via the Source monitor ... bring a clip up there, and apply a conversion/transform LUT as the Input space (Technical LUT slot) in the Basic tab.

But you would have to then go to Interpret Footage, and set the Override-To option to Rec.709.

At which point, you would be better served by applying the same LUT in the Interpret Footage Input LUT slot. And working within their setup.

Lumetri on a sequence will only see the clip CM as managed by the sequence. As the clip is managed to the sequence color space before Lumetri will "see" it.

Neil

[Chetan Nanda]

Hi @Baffy19 ,

I got the understanding of the workflow you are following.

Just for confirmation, Step 3, as you mentioned above, is where you are doing the adjustments before converting to Rec. 709. Am I understanding it correctly?

Hi @R Neil Haugen ,

What @Baffy19 is following is completely possible in Premiere pro while dealing with Log media.

With 23.1 Release. We have a setting in Preferences

With this setting off, Log media when imported on Rec. 709 timeline will not get converted. Its like the media is treated as Rec. 709 by the application. User can follow the workflow listed down by Baffy.

I am willing to setup a dedicated call to get deeper into the discussion.

Thanks,

[FlyingFourFun]

@Chetan Nanda - is it possible for a 'live stream' on YouTube or something like that, even if its private or a teams/zoom call - Id like to sit in on this call also and maybe ask a question or two possible.  I know this is in Beta still, but there is a great deal of interest in such a vital part of Premiere that I think it would make sense to open the call up a little to others.  Thoughts?

[Chetan Nanda]

@FlyingFourFun Sure I will add you if we have a call.

[Baffy19]

I am ready to consider these issues here. The calls will be based on the same. Therefore, I see no point in discussing it on the phone. I have a lot of questions about the LUMETRI panel in general. Either upgrade the tools, or adjust the color wheels, or add four wheels to use the LOG material. Now it is impossible to work on it in full. It is limited. I can explain why and this will also interest you.

[R Neil Haugen]

I would love to see your explanation of how a fourth wheel would be used in log, might be pretty interesting.

I've been arguing for a RETURN to the fourth wheel we had for a while for HDR work.  The current setup ... is to me, totally a cadge. You need to grade with the scopes set to HDR but with 10-bit scales.

Well, then you can't see what your nits/IRE levels are, can you? So Francis says for that, after grading, scrub after setting the scopes scales to HDR.

But ... the scope scale in HDR goes clear to 10,000 nits/IRE, WAY past any current possible workflow. The highest settings being used anywhere for streaming even are 1,000 nits/IRE.

And ... the four graticules, bottom to top, at the quarter-scale points ... are ... wait for it ... !

• TEN nits/IRE
• ONE HUNDRED nits/IRE
• ONE Thousand nits/IRE
• TEN Thousand nits/IRE.

Currently it is fairly common to have specs that call for 600 or 800 nits peak white, and of course, graphics white at 203.

Right. Try and scrub through and get an accurate feel for where your graphics white & peak values actually sit.

Which is why a fourth color wheel, with an active pivot control! ... is so needed. Use the 'standard' three for the general grade, with the top of the Highlight wheel set for the project graphics white.

And that fourth wheel then allows for setting your peaks where you want them.

AND ... we need to at the least, have a Scopes HDR scale setting that sets the 'graphics white' point of the project at either the mid line or between that and the three-quarter line area. AND allows us to set a project "peak white" value, which would set the TOP point of the scale.

Neil

[Baffy19]

I totally agree.

First of all, you need to add an OFFSET WHEEL. Secondly, you need to add a WHEEL group to control HDR, since they are the ones that give fine-tuning on the influence of white and black. Thirdly, now we are talking about the problem of working with existing 3 WHEEL. They don't work exactly as they should. Therefore, we do not get the maximum benefit from color correction. I have an explanation for this.

[R Neil Haugen]

Will be interested to hear your issues/suggestions for the current wheels.

On my system, they work nearly identically to the LGG ones of Resolve. Very typical ... if you want to push say the shadows only towards one direction, as the actual reaction is it moves the shadows more than the mids/uppers, you need to also move the mids wheel in the opposite direct. Resolve works the same way.

I really miss the "9-way" of SpeedGrade, where when you went into Shadows, Mids, or Highlights mode ... the three wheels worked only that section of the image. With full pivots too! That was very fast, and I didn't miss having a curves tool because I didn't need it.

But again, if in the 'normal' mode 3-wheel mode in Sg, you would need to counter a move in one wheel with the ones on either side. Oh, and Sg was O/G/G ... Offset, Gamma, Gain ... which if I have to drop either Offset or Lift ... I'll drop Lift in a heartbeat.

Neil

[Baffy19]

Why compare with DaVinci. There are 4 WHEELS instead of 3. And to manage individual color ranges, there are 4 WHEELS LOG, which are insanely useful. Look at how subtly they work. And there are also 7 WHEELS for HDR control. And that's great again. Here for Premiere, at least, WHEELS LOG are needed, which, unlike the existing 3 WHEELS in Premiere, do not allow you to adjust and affect only individual parts without affecting other neighboring ones. This is a castrated color correction, forgive me for my statement. If only you use additional plugins, such as Magic Bullet. That's where the very 4 wheels are and they work correctly there, unlike the Premier. Compare how the offset on SCOPES occurs when affecting HIGHLIGHT. See how the effect on MIDTONES occurs. See how the influence on SHADOW happens. This is the correct operation of the color wheels and such a WHEELS group with the LOG index should be added. Then another conversation. You cannot perform color correction in Premiere using standard methods. Either through plugins or in DaVinci.

[R Neil Haugen]

I don't disagree with wanting more tools ... I always want that! ... I've pushed for an Offset in Lumetri since it was first proposed. And for say the ability to 'move' the tabs of Lumetri around ... and even simply having the tabs as effects. Add as many HSL or wheels or curves tab as you want, any order. Among many other suggestions.

Such as having actual pivot controls. The only pivot in Lumetri is in the Creative tab, the Highlights/Shadow color balancing tool has a pivot.

But all that said, I simply cannot agree with the blanket comment that you cannot perform color correction in Premiere "using standard methods".

I've done way too much work pushing Lumetri around. And I can work small areas of the image at need at any time. Is it as easy to do so as in Resolve? No. But possible? Certainly. I do it every project I work on.

I've also got a full ColoristaV workspace saved.  I LOVE the hues tool there ... the two circles where one is sat v sat  ,,, moving a dot around the wheel changes hue, moving in or out changes the saturation of that hue. With the accompanying (and linked) sat v lum circle, where moving the dot out brightens that hue, in darkens it. That is a wondrous tool. Would love that in Lumetri.

But that is really the only thing about the ColoristaV panel I can't do similar/as-well in Lumetri, which works easier with my panel. So for most things, I never bother going to Colorista.

And also, having the full RG Universe, I can use their grain tools, which ... after Adobe removed a number of the ones I used to use in Premiere ... are pretty useful at times.

But working with the various curves panels in Lumetri, I can do most of what I can with that hues tool in ColoristaV. Enough that most people won't notice the difference.

And that last comment is something which noted colorist Walter Volpatto hammers at times. For instance, he's tried layering grain to more completely mimic film grain ... smaller 'dots' for shadows, bigger for highlights ... from grain 'effects' by various plug-in makers, to working with actual film grain "overlays". And then done a "simple" add noise/grain effect in Resolve.

Exported a test project with all the above processes intermixed, and had a Hollywood studio audience watching it projected in a screening room. All of them DPs and directors ... a very visually astute, sensitive, and critical audience. Could anyone in the room, watching on the big silver screen, see the difference? Notice when he changed methods?

Nope. Because the process of transforming the Resolve sequence to the deliverable used for the projector, with the compression that is applied, wiped out those subtle differences. Completely. And he had the same result for streaming, viewed even on a huge TV. You can't tell the difference between a simple grain effect and an elaborate re-creation of exact film grain after viewing the deliverable ... delivered as expected.

So ... he says it's really stupid to knock yourself out on something that cannot be seen in any deliverable. His 'grain' in all his films is the basic Resolve grain vfx option.

Do I want more and better color tools in Premiere? Of course! But in the meantime, can I do decent work in Premiere? Well, yes, really.

Doesn't mean I'm satisfied with the tools in Premiere. But I can work in it, rarely need to go to Resovle. Which ... is certainly nice to have available.

If I were going to do say a full "Hollywood" type film though, you bet I'd wanna be in Resolve or Baselight. Well ... a bright & happy rom-com could be done in Premiere ... not a particularly difficult audience to deal with ... 😉

Neil

[Baffy19]

Remove all the additional tools for color correction and try to work on projects with not squeezed dark, overexposed light ones. I'll see how comfortable you feel. You mention Colorista in your work. Of course, it saves and is necessary for a good effect on ranges. Then answer me the question: WHAT PREVENTS DEVELOPERS OR PEOPLE WHO WORK WITH COLOR OR SUPERVISE THIS DIRECTION FROM MAKING EXCELLENT AND NECESSARY TOOLS FOR FULL-FLEDGED WORK? It's been going on for too long to count, and there's ZERO sense in development.

[R Neil Haugen]

Possible with log media, is the key there. Baffy also mentioned some raw forms, and ... that may or may not work.

And ... as I've mentioned to Lars ... the naming of the new tools is not always indicative of the actual or full effect, and with no explanatory tool tips ... this is rough for users to sort out all the implications.

Because every one of these has numerous implications for differernt workflows.

Neil

[Baffy19]

Exactly before the conversion. I have specially completed the steps today and confirmed it on SCOPES. If we first develop the material, LOG allowed, and then place the Lumetri effect lower down the hierarchy for adjustment and balance, then SCOPES will be cropped, both in HIGHLIGHT's and SHADOW. If we do as I described above, then we will work with the full range without cutting the dynamic range. This method is valid in any version, I think.

[Baffy19]

Here is an example of what if it is wrong to go for color correction.

[Baffy19]

You might be able to apply a change via the Source monitor ... bring a clip up there, and apply a conversion/transform LUT as the Input space (Technical LUT slot) in the Basic tab.

But you would have to then go to Interpret Footage, and set the Override-To option to Rec.709.

Neil

By @R Neil Haugen

Why, after the conversion, the technical LUT should interpret the material in REC 709. You have already developed the material in REC 709 with the help of LUT. What is the process of double conversion?

[Chetan Nanda]

>I have a lot of questions about the LUMETRI panel in general... Now it is impossible to work on it in full. It is ?> >limited. I can explain why and this will also interest you

Hi @Baffy19 , 

As @R Neil Haugen mentioned, it is not the case that Lumetri is not usable at all or we can't work with it.

We do agree that it needs improvements, and Yes,  We would like to understand the issues you are facing  with Lumetri panel and in other color management area in the app.

[Baffy19]

@Chetan Nanda 

Unfortunately, Adobe does not pay me for hints and obvious problems in the operation of the application, so that I ask here on points. If this were possible, believe me, the product would shine with bright colors in the faces of users. And there are a huge number of flaws beyond the limits of the LUMETRI panel.

The worst thing with the panel in LUMETRI Pr v. 23.1 that the program crashes when working with it. There were problems with the Vignette effect. This is the first.

Secondly, circles affect all areas of the image simultaneously (HIGLIGHTS, MIDTONES and SHADOWS). I need to bring color only to the shadows, while they affect the neighboring MIDTONES and, even worse, HIGHLIGHTS. For the hundredth time I write, compare the work of 4 WHEELS in Magic Bullet with Premiere circles. Just look carefully at the SCOPES when you turn the ranges. Carefully. Where competent people look at it and consider it the norm. Or is it beneficial for someone to use third-party plugins in cooperation. I don't understand.
Third, the absence of the OFFSET wheel and LOG WHEELS makes the LUMETRI panel incomplete. OFFSET will allow you to quickly balance the entire frame, without the ability to turn circles and sliders of temperature and hue. LOG WHEELS will allow you to influence the tonal areas without affecting the neighboring ones. This is insanely important.
Fourth, there is a lack of separate image balancing settings for the white and black points, which will allow you to cleanly align the source's tonal map without "dirt".
Fifth, there is no tool to be able to influence the gradient of using tonality for an image through the LUMETRI panel.
Sixth, there is not enough tool to adjust only the brightness in each individual channel without affecting the range of colors.
Seventh, in the CREATIVE tab there are "cropped" two circles for setting the shade for SHADOWS and HIGHLIGHTS. Who came up with this and why he earned the medal.
Eighth, when selecting a LUT on the CREATIVE tab and after applying it, it is impossible to assign a new LUT from viewing the preview window mode, since the countdown returns to the first LUT in the list, which makes it impossible to track and view the following LUT in real time. It's not practical.
Ninth, there is no structure for placing LUT on the CREATIVE tab and looking for a LUT of a certain group of hundreds of LUT is a stupid idea. Here you need to support the folder structure when placing the LUT in the Premiere root folder.
Tenth, there is no way to add frequently used and necessary for everyday work LUT to favorites.
Eleven, all the color settings should be placed here on the LUMETRI panel, and not scattered throughout the program.
These are the main ones.
I hope I no longer have to ask what should be on the LUMETRI panel and how it should help users in their professional work. I really hope that my words will improve the LUMETRI panel and make working with color pleasant and professional.