Lost in color management

[Tokumeino]

Hi, as an amateur photographer with many unsorted videos from many sources on my hard drives, I want to gather clips and edit videos to sort them all a bit. No deep edits and no fancy color grading but I don't want to completely mess with colors.

Sources. My (eventually old) clips are shot in various formats and color spaces : sRGB, HLG/BT.2020 and now SLOG3/SGamma.Cine and I want to mix them all in the same timelines.

Exports. Due to the increasing availability of larger color space screens (TVs, phones, computers), I'd like to be able to edit my videos in a larger color space and postpone until export the choice of the output colour space. Does it make sense to work on HDR timelines by default and hope that the export process will export correctly SDR or HDR, at will ?

Display. I'm waiting for a laptop which is advertised to cover DCI-P3. I always calibrate my screens with my good old i1 Display Pro and DisplayCal never asked me for a "target" color space so I assume that regarding the screen, all I'll have to do is to calibrate as usual and let Windows and the apps deal with colors. Am I right ?

I've tried Resolve for a few projects and the nodes thing makes the pipeline immediately visible. For various reasons, I prefer to invlove in Premiere Pro but everything the color things look like black magic. I'm sure it is efficient at the end, but what settings should I pay attention to ?

[R Neil Haugen · Accepted solution]

First, this is easier in the current public beta, which is a "next-series" build, the 24.x builds that will go public probably first day of Adobe MAX in October.

Because in the 24.x builds and forwards, nearly all color settings will be available in one place, the Lumetri panel's new "Settings" tab. I've been asking for a unified CM panel, and we're finally getting it. Thankfully.

For any export to succeed, the clip CM, working space CM, and export CM must match completely in color space settings.

So the clip on a sequence must have the same CM as the sequence and export. So no, you can't mix willy-nilly and get a usable export.

However ... you can duplicate a clip in the Project panel ... doesn't duplicate the clip on disc, it just gives it another 'home' or reference place for the clip metadata Premiere will use for that clip in specific use. This allows for using the same clip in the project for separate sequences and different color spaces.

I'd suggest renaming clips that you dupe, adding the CM to the clip name reference. And probably put in bins for the same color space even.

That way, again, the same clip (on-disc) can be used in both say HLG and SDR/Rec.709 sequences in the same project. With of course completely different effects added to the clip, including color.

Understand ... I work for/with/teach pro colorists, some of whom were among the earliest adopters of full-on DolbyVision HDR for broadcast use. And were hired by Dolby to do the in-house training for colorists on how to work with their new toy. Both in software and hardware.

And I've spent a ton of hours in Resolve as well as Premiere, pushing color data around. Participating in wide-ranging discussions on CM and HDR/SDR workflows. All of that.

So ... a bit of advice ... HDR is a wondrous thing in some ways, but only if the media involved, the full workflow to export, and the display device agree on the details. Which at this time, is definitely iffy. Most screens still do not completely or properly support all HDR forms, the ones that support HDR in X don't in Y ... and besides, most of them don't correctly support what they say they do.

Totally Wild Wild West out there.

That's the first thing. The second is ... do you have a problem say watching Lawrence of Arabia, that it's only been viewed on TVs in SDR/Rec.709? Sound of Music? Doubtful, really. Because even SDR/Rec.709 shows, when well done, are visually beautiful.

The realistic gains of HDR come in really only three ways.

• bigger color gamut or "volume" of colors. (Useful only in a well setup display of course.)
• larger dynamic range ... which actually isn't necessarily doing what most people think.  I'll talk about this below.
• For many colorists, the biggest gain in HDR workflows isn't highlights ... it's more shadow definition.

The first two, bigger color volume, and larger dynamic range dark to bright, are nice but not necessarily massive. As most displays even that say they support say 90% of X color space ... well, that ain't the full space. And after a full-on colorist type calibration and then running a profile to check the calibration ... well, what's claimed and what is actually achieved with a lot of extra work aren't the same thing.

Past that issues, most displays don't handle brightnesses above say 350-450 nits well if at all. Especially the color goes wonky up "there". That's not even close to the 800 nits you really need to have to get wowie HDR shown.

So ... bigger color gamut and brighter images are sort of useful but not really ... um ... accurately used at this point.

The last thing is what trips up amateurs grading HDR, is they tend to try to take the mids and highlight areas too bright, which the displays will dump the details out of ... and don't pay attention to their shadows.

A well-done HDR grade places the "graphics white" ... think a white printer paper in direct sunlight ... only at around 200-203 nits. Which is only 100 nits above the SDR limit, right?

Above that, is simply colored specular bright spots.

But in order to even simply double the DR of SDR work to 200 nits, you have to be very aware of those shadows. If you have well shot media, they can be carefully expanded to show more details and definitions well into the shadows. If you don't ... or aren't very aware and careful ... you get a noisy gunked up mess. Just very obviously so.

Past this, well ... not a lot of SDR shot media handles moving into full-on HDR too well. You get a lot of noise and banding and other artifacts trying to expand both the dynamic range of brightness and the color gamut at the same time.

Just because we have HDR doesn't at all mean that SDR stuff is automagically "bad". But unless 1) the media can support it and 2) you're pretty good at this stuff, trying to push SDR/Rec.709 media into HDR can be a mess.

[Tokumeino]

Thank you so much for taking so much time for such a sound answer ! I'm answering quickly to thank, but I'll have to read it twice more, beeing a newbee in color management and in video in general.

I get your point about screens. The laptop I've ordered is typically advertised as DCI-P3 (I guess it should be some sort of D65-P3 with a more conventional gamma) and it only goes up to 430nits, which is indeed far from plain 1000nits HDR. So it is not actual HDR but it is more than sRGB nevertheless, a kind of in between, typical in the farwest you mention. About the colorists aiming at 200nits in HDR (perhaps for similar reasons as photographers used to set whites at 240 or so and not 255 just to get something a little more subtle), wouldn't 400 nits (probably much less useable as far as accuracy is concerned) be enough to decently work on an HDR timeline ?

But I get that your general recommendation is to stick to Rec.709 for now, right ? Did I get somehow ripped off by buying a DCI-P3 enabled laptop ?

I've just watched this nice video about the Beta you mention, and it is related to mixing clips in various color spaces without making copies and such :

And as far as I undestood, the new "Auto Tone Map Media" and "Auto Detect Log Video Color Space" are precisely intended to map everything to the timeline color space without headaches. Your "for any export to succeed, the clip CM, working space CM, and export CM must match completely in color space settings." is really super underwhelming for me and could be a Premiere no-go for me.

[R Neil Haugen · Accepted solution]

As someone who works in both Resolve and Premiere, PrPro's color setup is a ton more simple to work within.

That thing of matching the CM spaces is really quite simple. You can select say a bin of clips that are HLG, right-click/Modify/Intepret Footage, set the Override-To option to Rec.709. Boom, done. All of them set to work in SDR/Rec.709 in one action.

Then just check the Sequence settings ... you should anyway, you know? ... and see that the CM is working space Rec.709.

Then just use export presets that don't have HLG or PQ in the name. It's that simple.

For HDR, you 'flip' the above essentially. With a few more little goodies, but after doing it twice, you'll be fine.

As to your laptop ... a 400 nit P3 screen is a decent screen, probably. For a good bit of time, not very many screens anyone is watching on will really go past that much in a usable sense.

HDR is simply more complex .... no getting around it. But it is doable.

Take the graphics white point thing, which is not the same as the top of the 100 nit SDR/Rec.709 scale. This is a puzzlement for many people. Because there's two 'tops' of the image to think about. Graphics white, and total specular brightness.

The "graphics white" is simply the brightest value you want detail to show in. That is typically set around 200 nits. And you need to watch the scopes, and figure out how to check that. Which is a bit of time spent, but we can help you with that at need.

But ... you may have specular highlights, like candle flames or lightbulbs, streetlights or stoplights, that go well above that, but it wouldn't normally have any detail or texture. But still have some color to the light.

So in some scenes, you won't even hit the 203 "graphics white" on the scopes, while in others, you may have a few things above it. Takes getting used to judging when you need the extra brightness ... and when not.

[Tokumeino]

Thanks again. I beleive that I understand the graphic whites vs specular highlights. It raises a question that is perhaps dumb... For photography, I'm used to calibrate my current sRGB screens for 120 nits (a touch above 100 nits). If one aims at HDR video, should he calibrate for slightly above 200 nits ? (if OK to burn specular highlights on the display and loose control over it during edit, be the scopes) Unimportant side note : color spaces are somehow abstract to me ; they appear as overlaping triangles in CIE XYZ charts that I don't understand, and I cannot figure out what are the actual colors missing in sRGB. I may be wrong but your answers let me understand that those missing colors are the "fluorescent" colors, which shine not so much on a regular screen than in reality. This way of thinking more about "absolute" brightness levels than contrast ratios (even if they are obvioulsy related) is quite a shift in the way I need to figure out things for video. And this could be the key to understand scopes. As a photographer, I have so much to learn !

And now something completely different (sorry to ask but you seem very competent)... there are creative LUTs available for different color spaces. If I'm using the color management options, and if I apply creative LUTs using the Lumetry panel, then I need to use LUTs designed fot the timeline color space and not the original clip color space, right ? (the pipeline order is no crystal clear to me).

About LUTs vs color space, even if there are technical LUTs to convert between color spaces, I see them as different things. Am I right ? As far as creative LUTs go, my understanding is that they operate within the color space they are designed for and thus, my bet is that they are prone to trim colors. Regarding color spaces, if an editor is coded properly (with 32b floats and such), my guess is that if an edit (like raising exposure) would let colors clip out of the timeline space, then the information is not necessarily lost for future edits in the pipeline (lowering exposure for instance). Am I right ? Sorry for all these "am I right" there are "milestone" concepts that I need to understand correctly if I want to elaborate further and make oppropriate choices.

[R Neil Haugen]

By the way, I came outta a long pro stills career a decade back. Somehow got invited to NAB/Vegas (HUGE trade/training show) the first year, the wife decided I should go ... fell into Life with a number of noted colorists, and well ... now I'm somewhat of a noted head on these things. Paid to teach colorists ... woa.

You asked a good array of questions ... so here goes. Get a good cuppa something first ... 😉

Making Sense of Color Spaces

Most color spaces will have "statements" ... qualifying comments ... for both brightness/dynamic range and color space used.

Dynamic range refers only to the brightness differential, white to black point, of a camera or sequence. There must also be a color space listed as the one being used. For instance, an HLG deliverable spec might read "HLG @1,000 nits in Rec.2100" ... meaning a max brightness alllowed of 1,000 nits, using the Rec.2100 color space.

Note that max brightness allowed does not mean all scenes should have a top brightness of that figure. Clearly, the oudoor middle of the day in Arizona will be the shot that goes to close to 1,000 nits, while the indoor scene in a bedroom may top out at 80 nits. The actual brightness per shot is an aesthetic choice.

One famous and long scene in Game of Thrones ... streamed in HDR ... had a max brightness on a proper system of 12 to 20 nits for around a ten minute segment. Barely visible if you were in a blackened room with a very good TV and the streaming service to your set didn't compress the image too badly. This was an aesthetic choice ... that was so extreme that many users simply couldn't see anything for most of that scene.

Color space is often shown as those CIE triangles, but can also be thought of as color volume or even color gamut. And can be shown as rotating cylinders with colored dots inside, which actually is a more informative way to see the difference in color (hue, technically) between spaces.

In looking at either form, CIE triangles or volume/gamut cylinders, the areas of major change are in the greens and blues available. The red point ... the most saturated (intense) red hue, doesn't change that much. So it's the breadth and depth of greens and blues that changes most notably between color spaces.

For SDR/Rec.709 work, the specs call for the SDR dynamic range, the 100 IRE points max on the old electrical meter volt scale. Note that IRE is now frequently just called NITS, which is technically a monitor brightness measurement. Current head of PrPro Francis Crossman always corrects me if I refer to the left side scope scales as nits, because PrPro is actually set to give the old millivolt IRE output rating. But in practical terms, they are used interchangeably.

For HDR workflows, the dynamic range can vary with the project, and so is specified as the top value for that project. Professionals will have deliverable specs that call for say 1,000 nits or maybe as high as 4,000 nits. For the latter ... well, there are very few professional monitors that can actually show that high a signal. And no TVs I know of. It's sort of a future-proof thing.

Depending on the format of HDR, the color space can also be chosen from among several. Within Premiere, we have really only two choices of HDR format at this time. PQ, perceptual quantization, or HLG, hybrid log-gamma.

PQ is mostly used for professional deliverables, say passing on the project to another video post worker while keeping all brightness/color values. It is also the choice if you need to create HDR10 exports, as the HDR10 setting can be turned on well down in the PQ presets. IF you know where to look, and for what. And how to set the controls, which are static brightness numbers.

Premiere cannot create the HDR10+ format at this time, as that requires dynamic brightness numbers ... in other words, metadata in the export for changing brightnesses in various scenes.

HLG is hybrig log-gamma ... meaning a defined logarithmic curve is applied to the dynamic range (brightness) used, and a color space will also be specified. Quite often Rec.2100, a subset of the Rec.2020 standard. Premiere uses Rec.2100 for the HLG format.

So what is a color space ... or volume ... or gamut? Really?

This is a very naturally confusing concept. And I think largely because the CIE type triangles are confusing. Especially with say six different colored triangles overlapping. That's nearly impossible to even see what the heck they all do.

It's better if you can find ones with only the two or three spaces you're interested in seeing. For Premiere Pro video workflows, realistically at this time, that's probably Rec.709 and Rec.2100, maybe P3. Unless you are using one of the costly Red, Arri, or Sony cameras that does actually use a wider space.

Rec.709 is SDR ... standard requiring a specific max dynamic range, and uses the sRGB color space, and a specific display gamma. The color space is the same space as used in sRGB stills work. This has been used in broadcast for years now.

There still is a separate video form color space similar to sRGB, and apparently a couple stations still list it in their deliverable spec sheets. But it's been replaced by sRGB for nearly all broadcast and streaming use for many years. Always use sRGB as the color space for Rec.709 sequences.

I do recommend researching a bit about color volume and color gamut in video use. They aren't exact equivalents, and tech nerds will call you out if you use them interchangeably. But ... they are rough equivalents for general discussions. And you can often get better graphs and animations showing the different choices available. Such as the animated (revolving) cylinder graph I mentioned above.

So if there's only a bit of added hues possible on the reds, and it's mostly in the blues and greens that more hues are available, what's the big deal about color in HDR?

Actually ... the two parts of HDR work that do really stand out are the dramatically expanded brightness range available, and ... the intensity of the brighter colors. What is this brighter color intensity thing?

In SDR work, as you get to the midtones and brighter, color intensity drops significantly. The visual sensation of saturation of a red with a brightness of about 40 IRE, at maximum allowable sat, is quite a bit more than the same red hue at 85 IRE. The brighter the color, the less saturation is seen. Or perhaps better said, as perceived.

In HDR work, color intensity does not drop with raised brightness, when shown on a very capable screen. I've seen a bit of HDR I've done, back in 2019, on a 3,000 nits Flanders grade 1 reference monitor ... a $30,000 monitor! ... and on a supposedly 400 nit TV.

On the Flanders ... just ... OMG ... wow ... the color intensity was breathtaking.

On the 400 nit TV ... well, it was a bit brighter in color than when in SDR. But that screen did not have the full color volume coverage in brighter values that the specs allow for. So yea, it was a bit more colorful, I suppose, than an SDR signal, but ... not that much, really.

In 2019, Flanders had a new screen type that they were expecting to take over the industry for that $30G monitor. But the company owning that factory couldn't sell enough working screens to pay for the creation of the factory, and went belly up. Taking that tech down with it.

And we've not had a replacement until just a couple months ago. Flanders now has a new monitor tech coming out of China I think. And has started delivering a 55" grade 1 reference monitor using quantom dot OLED ... QD-OLED ... with perhaps the biggest color volume yet and 2,000 nits peak brightness. The XMP550.

At only $19,995, it's a steal, right? Ha. But the real high quality QD-OLED screens will be coming out for TVs probably over the next year or two. And with the inevitable price drops for quantity production, they will become more prevalent in homes and shops. And maybe you and I will be able to justify an HDR monitor we can actually grade with.

As ... all TVs and monitors out now do limit the brightness and saturation to protect the screen from burn-in.  So when grading you can't "park" on a frame for more than a few seconds, before the image will subtly drop both brightness and sat. You probably don't even notice it.

But ... that the TV or monitor is always changing the image, at least at times, is Hades to try and actually grade to, reliably. It's why the colorists using larger LG TVs as HDR monitors get the technician's remote so they can go into specialized menus and override ... well, most ... of that behavior. As not even all working pro colorists can justify $20,000 to $30,000 monitors as a business expense. When HDR deliverables are going to be at most 10% of their output. If that.

And at this time, the majority of working pro colorists have yet to deliver a single paid HDR gig.

But what about using scopes with HDR work? It's confusing!

Yup. Totally agree. Set the Lumetri Scopes panel in Premiere to its HDR setting, and ... the total brightness at the top of the box is 10,000 nits, which NO monitor can produce, and realistically, ain't gonna happen for YEARS.

The middle graticle (dividing or section line) is 1,000 nits. The one at the bottom quarter is 100 nits. The actual area of the scope you use is so compressed it's really unusable. The top half of the box is simply wasted space. Which I've complained about, of course. But ... engineers gotta be engineers, right? Sigh.

So I agree with comments made awhile back by Francis Crossman ... for HDR work, set the scopes to 10 bit but not HDR. Grade as you think visually usable, paying attention to the Waverform and/or RGB Parade 10-bit scales, then at some point, switch to the HDR setting and scrub through seeing what IRE/nit levels you are hitting.

Of course, the Vectorscope should always be "up" ... and if the sequence is set to say Rec.2100, then the scopes show the signal as sent to the display space. It seems ... correct.

And as you go through your trim scrub with the scopes set as HDR, adjust the image as necessary ... and over a bit of practice, you'll get better at setting your contrast/white/highlight values.

I'm not happy with that advice, but ... it is the only practical way to work at this time.

[Tokumeino]

Thank so much, again. I now beleive that I get a basic understanding, enough to drive my experiments. Your input is so interresting and fascinating that the nerd inside me really wants to practice ! 

This thread is so full of keywords that I'm sure that your efforts will also help many visitors in the furure.

[R Neil Haugen · Accepted solution]

Color fascinates my mind. Now, sitting and making the 5,000 decisions a minute that actually cutting a sequence together freezes my executive functions.

But ... color decisions ... don't.

Brains are fascinating if weird devices.