Upgrade Mac and Premiere but LUTs don't work - NOT color mgment issue

[Afterglow]

Moving from Mac Pro to new Mac Studio and from Premiere 2018 on High Sierra to Premiere 22.5 on OS Monterey.

Have already worked through color management issues. (Bad dog, Adobe, bad, fix this now instead of the tiresome 6 step process for SLOG footage!)

I've re-installed the LUTs I used in the previous Premiere in exactly the same folder (Lib/Application support/Adobe/Common/LUTs/Technical) as recommended in Prem 2022.

When I open ongoing Premiere projects, the effect control space of Lumetri shows the LUT and settings as they were and the LUT changes each clip's look but not as it was or should be. It's a washed out look not too different to the untreated SLOG image. When you scroll down to the 'favourite' LUTs and select exactly the same LUT, presto, the clip looks like it should.

Surely, please, there's some way to avoid re-setting this on every one of the hundreds and hundreds of clips in ongoing documentary projects?? Thanks!

[R Neil Haugen]

I'm not used to working with the Lumetri effect very much via the ECP ... I tend to use it only via the panel as it's quicker to work there, and of course, I use a Tangent Elements panel.

But the "favorites" LUT option ... I've no clue what you mean by that.

There's the Input LUT slot in the Basic tab, which I never ever suggest people use because you can't trim the clip into the LUT ... by adjustments processed prior to the LUT ... for proper LUT usage.

And there's the "Look" slot in the Creative tab, which is where most all LUTs should be applied from.

LUTs being, as most colorists call them, "the dumbest math out there". They're simply a look-up table of "take pixel value of X and make it Y". If that clips or crushes pixels, so be it.

But if you apply a LUT in the Creative tab's Look slot, then use the Basic tab controls to trim exposure/contrast/whites/shadows/sat while viewing and "scoping" the results, you get the proper effect of the LUT without the clipping or crushing or other problems.

And for simplest access, I use and teach the Program-Package file/Adobe/COMMON/LUTs area. Seems simpler folder assignment to get to for my brain. (Of course, NEVER ever put LUTs in the Program/Premiere/Lumetri/LUTs folders ... )

As to the S-log, that shouldn't be that difficult ... select all the s-log clips at one time. Right-click/Modify/Interpret Footage, set the S-log3.cine option or the "Override to Rec.709" option, whichever you need. Done in a batch.

There are times with the new color management tools where say one uses an Input LUT ... and program manager (and former color engineer) Francis Crossman says if you use an Input LUT in the color management options, you might want to also set the Override to ... option to Rec.709 if you're using a Rec.709/SDR sequence. Very similar to setting an input transform and working space transform in Resolve.

That said, they built an entirely new underlying color system for the Pr2022 builds. Caught even me totally off-guard having used the public beta. It seemed like it was just a couple new controls. But when bongo! every default behavior is different, and all log media that isn't "color managed" officially by codec in Premiere is seen as HLG ... I realized this is simply not the old Rec.709 underlying code at all.

Interestingly, even in person at NAB, when I asked staffers about this being a completely new color system, I got ... silence. Until I asked another question. They wouldn't tell me I was wrong ... which I know from long experience they have no trouble doing when I am wrong! ... but apparently, they couldn't tell me I was right.

Fascinating, that. Working for a large company must be an intriguing experience at times.

And at this time, we still have a number of isses with their brand spanking new color system that aren't working as they should be. And the way the new color management controls are spread through a context menu in the Project panel, a 'main men' selection in the Sequence Settings panel, and a right-click context menu again for the Scopes panel ... is just wrong.

We need a unified color management panel ... Color Management Panel

Neil

[Afterglow]

Thanks for reply. Yep, no problems with Adobe introducing new color management system but it needs to fit in with current users' workflow and be communicated as such and I've supported your call for a CMP. At the moment it adds a lot of work by the time you select the camera clips within the folders (sometimes thousands spread across 30 folders and it's no good just selecting the folders) of all the footage, then switch them back to Rec 709.

Re LUT 'favourites', if you look at the ECP in Lumetri effect in Basic Correction Input LUT, to the right there's a drop down menu with what I think of as favourite LUTS which, among those pre-loaded by Adobe, are those installed in the Lib/Application support/Adobe/Common/LUTs/Technical folder.

I'm not sure standard practice is to make basic corrections pre-LUT. Certainly with camera like Sony's FX6 shooting in Slog3 it's much more difficult and ultimately less successful doing things this way, whereas applying something like a Phantom LUT (developed for just this camera) at the Basic Corrections stage gets the best results straightaway. I've tested this again and again and never succeeded.

Anyway, it seems there's no fix for my problem of getting LUTs applied in the older version Premiere projects to automatically work in the newer version other than manually changing each clip.

Thanks again.

[R Neil Haugen]

There are some issues with getting the same image between the last version and this one, which I doubt could be totally fixed ... simply because the underlying color system is completely and TOTALLY different.

All color math before was based on Rec.709, meaning color primaries at sRGB, expected top white of 100 IRE, math in 32 bit float. The limited HDR capabilities was acheived by using what they referred to as "over-range Rec.709 data or values".

That whole system was incapable of transmogrification into a full modern color system, and had to go. The new one I've called "color space agnostic" as that's how it behaves. It can work in either Rec.709/sRGB or Rec.2100 in HLG and PQ. (How well it works is still under discussion of course ... )

However, getting the exact same visual behavior between the two systems, the previous one and this one, is probably quite difficult. Some of my colorist buds simply didn't think the Premiere team would be able to make a 'modern' color system. Well ... it seems they've got the basics of it functional, mostly ... sort of.

But taking projects that were worked in the last version into this one is not ... easy. Nor probably, wise. I do strongly suggest that if you have a project started in 2021 or earlier, work that project and any added work in that project in the 2021 version. Keep 2021installed for all that work. (Which is at least easier than working multiple versions of Resolve.)

And you're right, that there's a lot that isn't working yet in Pr2022, at least as well as it needs to. I expect at this point, we'll get a couple minor fixes between now and sometime in November, when the Adobe MAX event is scheduled. That's where they've been rolling out the "New" versions the last few years.

And we'll have to see if they get more out in Pr2023. Certainly hope so. At the moment, this is easily the diciest for color working since the CC versions began. And yes, I talked about that with the supers of the Adobe video apps at NAB ... and they clearly understood what I was talking about.

And right now, I really, really miss having SpeedGrade as an option!

A note on LUT deployment ... in Resolve, if you apply a LUT on a node, that is processed after all other color controls/effects applied in that node. And I know the grading trees used by a lot of colorists doing manual work where the transforms are either in the first node or frequently second.

If first node, they 'trim' any problem data such as crushed or clipped data in that node. Many prefer a "clean" node tree, so the only thing done in their second node is transforms, and the first node is there for trims if and as needed by the clips.

In several of the discussions I've been through, someone has had a problem in that they're using a color-managed setup (either DCM or Aces) and have troubles with clipped or crushed data in some of the clips. And at that point, the discussion is how to get that auto-managed action reversed or disabled for that clip so that they can trim the clip properly.

Why?

Nearly all tech LUTs are built around 'perfect' setups. Perfect setup for the camera: lighting color, brightness, and contrast range. "Field produced" clips may well not be so perfectly produced. And if one or more of the clips of a project don't fit quite sweetly within the expectations of the LUT, well ... bad things happen to some poor pixels.

Ergo, a need for trimming.

The media a lot of my colorist buds working in Resolve get tends to be pretty well produced in-cam, and they can do a lot of projects mostly working without many trims. Which is awesome if that's the media you're working with. Though even their projects typically have a real clunker or two or three.

A lot of the media I've seen used and discussed 'here' ... well, has perhaps a wider range of camera production values, shall we say? And really needs to be trimmed a lot more than one would hope.

And if the clips you get never need trims well ... congrats! You've got some awesome camera people working ahead of you!

Neil

[Afterglow]

Hi and thanks for the thoughts above.

But I'm still stuck with a number of big projects that cannot be upgraded from PPro 2018 to either Prem 21 or 22 with the LUTs working - even if the LUTs are located in exactly the same place!

It's one of those situations where I could scream or weep at the hours, days and weeks ahead of re-color grading the footage. Why does it have to be like this?

I would love for someone to suggest any kind of fix.

I would also like this to be put to Premiere Pro developers.

[R Neil Haugen]

The underlying color system in Pr2022 is completely new, has nothing in common with 2021 and earlier. It had to be, to move forward and eventually properly work with HDR media.

But with a completely new color system and default behaviors, it doesn't work well with projects from earlier versions.

For those, keep 2020 or 2021 or both installed, and handle any reworked bits iof those versions in their original version.

Neil

[Afterglow]

I've experimented with projects in v2021 but that doesn't work either, as mentioned before. So I guess I'll install 2020. Shame cos dialogue to text and other features of 22 are handy. But of a mess. 

[R Neil Haugen]

Not working in 2021 is the puzzle, that had the same CM as 2020.

Neil

[Afterglow]

It gets crazier.

Installed PPro 2020 in the new Mac Studio OS Monterey, set up LUTs in exactly the same spot in Common folder in exactly the same order of LUTs, opened an ongoing project and the LUT has no effect. 

So when you open the Effect Control Panel for any clip with a LUT in Lumetri, it lists the LUT as being applied even though it has no effect and the picture remains the same washed out SLOG image. Oddly enough, the other settings in the Basic Corrections do continue to have an effect.

Looking at the second image below, while Basic Corrections input LUT reads Neutral Fx6, there's no effect but if you scroll down and select the same LUT from the drop down list, the LUT works. Arrrr, why?

I may be spending a lot of time on this, but fixing this single bug will save me from an extra week of work or more.

[Afterglow]

No one has any ideas for how to fix this? So frustrating that the Input LUT shows the correct LUT - in this case Neutral Fx6 - but there's no effect on the picture. And as mentioned above, the LUTs are exactly in the right spot and can be used in Lumetri if chosen from the drop-down menu.

It's the same in Ppro 21 and 22, having upgraded from 2020.

Anyone? Adobe developers, testers?? At present, I'm having to move back and forth between old and new computers and different versions of Premiere and it's sending me crazy.

Thanks in advance

[R Neil Haugen]

I can't seem to replicate the problem. If I select a lut, it's applied.

Also, I would note ... one of the advantages of using the Lumetri panel is you don't ever actually 'apply' it to an effect ... that happens automatically. So you can work down a sequence without doing any drag/drop from the Effects panel to the timeline.

Neil

[Afterglow]

One thing I've discovered is that almost all my applied LUTs are from presets I've created with pre-made LUTs (like Phantom LUTs, Sony, etc) and stored in the Effects Panels in the Presets folder. These show up as described above in updated Premiere apps BUT DO NOT have an effect on the clip.

If, instead, a LUT is directly applied not through a preset then it works when the project is updated from a later Premiere version to more recent versions.

Why is this? I wish I knew and could fix it. Perhaps the engineers know about this?

In the meantime, I'm regrading all my ongoing projects source footage.

[R Neil Haugen]

Where are the LUTs themselves stored on disc? Not the preset, the actual LUT file.

Neil

[Afterglow]

Hi Neil, thanks, as described earlier, LUTs are stored in "Lib/Application support/Adobe/Common/LUTs/Technical folder." That's in all three Premiere versions I'm using across my cMP and newer Mac Studio.

Recapping, I'm trying to migrate Prem projects mainly from Premiere 2020 (on cMP OS High Sierra) to Premiere 2022 (on Mac Studio's OS Monterey). The footage has been loosely graded with a couple of favourite LUTs and some important adjustments made in the Effects Control Panel. Crucially, the LUTs in Premiere 2020 were applied via Effects Presets I've created with LUTs in them where I drop the preset on to clips in my timeline and then make modifications to Highlights, Shadows, White Balance etc. Easier that way, right?

Well, everything shows up fine in Premiere 2022 but the LUTs themselves have no effect. If I go into the Effects Control Panel and in Basic Corrections (where it shows I'm using a certain LUT which has no effect) then select from the drop down Input LUTs, then the LUT is properly applied and has an effect on the image. It's very complex and frustrating and I can only see this as being a small bug.

[R Neil Haugen]

Was just wanting to check.

If you create a preset in 2022 similar,  does it apply the LUT?

As if it behaves properly then yes, this would be leading to the conclusion of a bug.

And as they completely rebuilt the ENTIRE color handling and processing in Pr2022, we users have had several bugs the devs have had to sort and kill.

Wouldn't be shocked at another one.

And BTW, your process is as I teach colorists, when they must work in Premiere. With one change.

I rarely use the Basic tab to apply LUTs. Colorists call LUTs the dumbest math out there for a good reason. They are normally carefully constructed under perfect conditions for lighting and contrast of scene plus perfect camera settings.

Which is great if every clip you're working with has the identical camera exposure/contrast/color balance. Or is slightly less contrasty than the LUT was built for.

If your clip is slightly under or over exposed, and/or has more contrast, or more saturation than the clips the LUT was built for, you will clip highlights, crush blacks, or get over-saturated "clipped"  color data you can't recover. Why?

Because they process the LUT in the Basic tab before any of the controls.

If you apply the same LUT in the Creative tab, you get the behavior of Resolve, Baselight, any real colorists app.

Use the Basic tab controls to trim the clip into and "through" the LUT.

Your LUT now shapes the media exactly as it is designed to do. And no pixels are harmed in the process.

I've got their chief color scientist considering a change in processing order for the Basic tab. Until they do though,  I suggest using the Creative tab for even tech/normalization LUTs.

Neil

[Afterglow]

Hi Neil, that is interesting about the process you teach re grading and LUT application. Likewise, I'm always surprised when a clip doesn't require individual adjustment besides adding the LUT, esp shooting as I do doco material. Worth noting that the SLOG3 clips I pull off my FX6 seem to defy successful grading without a suitable LUT. The Sony LUTs are pretty bad, the Arri LogC works well and Phantom LUTs are superb.

I'm not sure how pixels might be harmed with the use of a LUT before any adjustments and am open to finding out more if you can point me in the right direction with any links?

On another note, if you can sway Adobe's color scientists to re-jig the Lumetri panel, surely they can re-configure the color management aspects of Ppro 2022? Also, I don't currently know of a workaround for making 'flat' proxies from my clips in the 709 space; they all have that crazy over-saturated color treatment.

[R Neil Haugen]

Among other things, I'm a contributing author over at MixingLight.com ... a pro colorist's subscription teaching site. I've been involved with the color side of video since 2014, back when Adobe's SpeedGrade app (formerly Iridas) was still 'live'.

I did a deep dive into Premiere's color management/handling back for NAB 2019, a 30-minute presentation in the Flanders FSI/MixingLight booth. Speaking between people like Alexis Van Hurkman, and Dolby Lab's house expert on DolbyVision HDR.

At that time, Francis Crossman was a color engineer, and overall digital video head Patrick Palmer assigned him to provide all the information I needed. Francis was the go-between for Lars Borg and himself and me for several hours of talks, tons of emails, and online meetings ... yea, even back pre-pandemic. (Francis now runs Premiere under Patrick Palmer's overal video management.)

Then before that presentation could be recorded and put up on the MixingLight site, I had to work between ML founders Robbie Carman and Patrick Inhofer and Francis and Lars with a TON of highly detailed questions from Robbie and Patrick. Took over a month of several lengthy emails a day.

So yea, I knew Premiere's old color management/handling/defaults/pipellines cold. And ...when the public 2022 was released, I got a huge shock when it clearly wasn't just some changes over the old system ... I was able to establish that the 2022 build had a completely new color system.

After it was released, via emails, online meetings, and especially at NAB in person, I challenged them on this. It seems that for some reason they aren't supposed to say it's brand spanking new and that's why there's been so many hiccups. But ... they very carefully do not say nor imply I'm wrong. Ah yes, experience and proper skills may be needed for the reading of corporate tea leaves ... sigh.

EVERYTHING color changed with the 2022 release. All underlying math, defaults, pipelines, everything. And that's the reason pre-2022 projects with 'typical' Lumetri work often glitch when opened in 2022. As say if you have log images, there's no default to those being Rec.709 anymore.

The 'natural' default of 2022, unless the engineers have written specific code for that specific format/codec, is log encoded media is treated as HLG ... hybrid LOG gamma. HDR!

I realized it was doing this. Then remembered a quote by colorist/teacher Kevin Shaw: "Some Rec.709 is encoded in log, but all HDR is encoded in log."

That's why the "override to Rec.709" needs to be applied in the new Color Management section of the Modify/Interpret Footage dialog for most log-encoded media if you want to use it on a Rec.709 timeline.

Ok ... back to LUTs in general ...  still here?    😉

The whole thing about proper application of LUTs is simply a "Best Practices 101 for LUTs" as taught by colorists. It is so easy to grab nearly any LUT, and get it to 'break' the media by either crushing blacks or clipping whites by giving it a clip just a couple points over/under exposed.

And if you apply that LUT before any controls to modify the clip ... and it clips/crushes/over-saturates the image ... you're hosed. That data cannot be 'brought back'. You've mangled pixels. Not a good thing to do of course.

So that is one of the first things taught when colorists teach colorists how to work. Careful testing and application of LUTs. Including always applying them with a way to massage the image prior to the LUT. To "trim" the image to fit the LUT.

And think about it ... that LUT for 'normalization' of log encoded media is designed for the Perfectly Exposed/Perfectly Shot clip, right? So the 'shape' it gives will only work properly if the clip you have matches the clips that were used to build that LUT.

So ... apply any LUT to a node in Resolve, then go to your Wheels or Curves or whatever tools for image modding, that work is applied within that node prior to processing that LUT. As any colorist would expect. In fact to make it even more obvious, most colorists seem to prefer to apply the LUT in one node, named, and make a node prior to that one, also named, where they do all the trimming of the clip into the LUT.

In Premiere, to match that, you need to apply the LUT in the Creative tab slot. Then use Basic tab to trim the clip to best fit the parameters and 'shape' of the LUT.

The two quickest controls for this in the Basic tab are Exposure ... which is sort of a massive Gain control except it does roll off data at the top. Coupled with Contrast. Contrast in SDR is totally based on the 50IRE middle of scale. Everything goes up or down from that point. Playing the two against each other is really the best way to do most trim work.

As you apply a bit of plus or minus Contrast, to widen or narrow the image contrast, use the Exposure tool to slip the center of the image up or down so the shape of the contrast applied via that LUT is where you want it to be visually.

If some that should be midtones is getting pushed into shadows, lift the Exposure. If some what should be shadows are going into midtones, drop the Exposure a bit.

Then use Highlights/Shadow controls to affect the ends of the image.

Neil