Very problematic greenscreen - how to fix edges/halo?

[Serega_M]

Hi, guys,

This is the first time I came to such a problematic greenscreen source that I can't key it properly to my (and my client's) satisfaction, so I hope that some real pro(s) out there will help me with this or tell me what I am doing wrong or what should I do to fix it.

I prefer to do all my keying directly in Premiere Pro (Windows) but if this situation can only be fixed in After Effects I am all ears too, although the best would be to do it properly in Premiere.

I have 4K footage shot on a not very good greenscreen stage and I’m supposed to use the lector in parts of the composite video as full figure in "master" shots and only his upper half in "close ups", where the keying problems are much more obvious.

The 15 sec sample of original 4k greenscreen footage I am working with is available to download here: https://www.dropbox.com/s/6kd81tbpdyosbv8/problematic%20greenscreen%20sample.mp4?dl=0

if anybody wants to try to do the magic 🙂

Background I use for close up is here: https://www.dropbox.com/s/30ryluwmocedzxc/problematic%20greenscreen%20sample%20bg.png?dl=0

Even though the video source is 4k, the final project is 1080p. So, the problem is that not only the greenscreen stage is unevenly lit and has different color parts in it (especially the part where his legs are on the floor, see this pic: http://joxi.ru/v29n1W6HzWlw5A), but all his body has a distinct "halo" or edge, which I can not properly eliminate. For all of my keying I routinely use Premiere Pro's Ultra Key which almost always gives a good result from the get go (in Agressive), but not in this case - see the pic: http://joxi.ru/n2YV1k3cZO6BMA). Paradoxically, the standard Ultra Key gives me much better result from the start than the supposedly much more superior and flexible Boris' Primatte Studio, which gives me MUCH more halo before the matte tweaks: http://joxi.ru/8An9w6eS7VGQX2 . I have even tried Red Giant's supposedly Hollywood quality keyer and it gives me even worse results than Boris.

So, after heavy tweaking I came to this result with Boris' key: http://joxi.ru/gmv9wq4SvzbEXr
which is kinda ok for "masters" where the lector's figure is relatively small, but very obvious for the close ups, for which I finally decided to use the heavily tweaked Ultra Key's result, which is not ideal either, cause to eliminate the halo I have cranked up the matte's transparency, among other things, see the pic: http://joxi.ru/n2YV1k3cZO6QMA
It introduces its own set of problems and is not looking good on light backgrounds which are also present in some places in the video and still there are some grey artifacts present, especially under his arms.

I've spent hours trying to solve this problem and watching tutorials and stuff, but still I can't pull this problematic greenscreen off to a level of quality I want (i.e. no halo or visible greenscreen artifacts on the composite), yet Boris' webinar tutorial on Primatte Studio (this one - https://www.youtube.com/watch?v=-FcfjqRTpYg&t=1519s) makes it seem like piece of cake supposedly on any, even problematic greenscreens.

So, any suggestions or maybe even direct solutions to my problem is welcome if anybody dealt with situation like this before. Thanks in advance!

[Meg The Dog]

Pulling a key from a green screen source is never ever a single button job.

This key is pulled with 3 Ultra Keys applied to the source clip. Each keyer was restricted to a specific area of the shot (by using the keyer's masking) and adjusted to pull the best key for that area.

1 is from above his head to about his knees, another from above his knees to below his feet, and the third is used as a garbage matte. The background is placed on the track above the key and masks the grid that is shown at the top of frame.

MtD

[Serega_M]

Hi, Meg,

Thanks for your reply. I am routinely using several Ultra Keys for various areas if the need arise, as you've suggested, and before results were good, but not in this case. Was you abe to completely eliminate the keying artifacts under his arms in the upper torse mask? Your picture is too low of resolution to see, and also, you have used the light background of the sky, while my background is dark where those artifacts are much more visible 😞

But if you have no visible artifacts even on the dark background (on completely black video, for example), can you please make the screenshot of your UltraKey parameters so I will try them here as well? Thanks!

P.S. Salvo, care to elaborate about After Effects? Cause I have tried to do the same with AE keyer, and it gave me no better result, unfortunately 😞

[Meg The Dog]

Are these the problems you are concerned about?

MtD

[Serega_M]

Yeap, exactly.

[Meg The Dog]

Looking at the source clip you provided, I see these artifacts in that file. I assume these are also in the original camera masters?

If so, and if this were my project, I would take the clip(s) into After Effects and rotoscope the artifacts out, then take thouse corrected files and make the key, which except for these issues, looks pretty good.

MtD

[salvo34]

I've been following this thread and am very impressed with the progress made toward a solution. Thank you ! I'm learning a lot !

At some point a step by step 'tutorial' on settings ( to get a decent clean key to start with ) and then how to get rid of the little 'holes' in the key will be needed ).

I'm looking forward to that process so I can learn how to do THAT TOO !  Thanks so much !

🙂

[Serega_M]

By rotoscoping out artifacts you mean create the fine masks over these areas, then track them and fill with something (green?) or make them transparent?

And please, can you give me a screenshot of your upper torso of this guy's key options? 😉 I need to edit three more videos with him shot in the same studio so would like to see what you did in comparison to my tweaks. Thanks in advance!

[salvo34]

poster still didn't answer if source material is in fact the same as the sample you got via link. Some miscommunication there... he's not paying attention and just wants you to fix it dog. And send all the info so he can duplicate it without understanding what he's doing ( the process ). Good luck with you new friend.

[Serega_M]

Salvo,

To understand exactly what he did I've asked for a screen of his UltraKey's options cause I did tweak it myself (gosh, it is in my original text even!) and I want to see what is different between his and my key options. 

And yes, the source and the sample are the same, why would I bother to make an aritifcially worse sample than my source? It is just 15 sec of 6 minutes source video I am working with with no tweaks from my side whatsoever.

[Meg The Dog]

My inclination would be to paint in additional deep brown/black to cover the artifacts and take the pressure off the keyer:

since you are going over a dark background anyway. You could also cover the whitish areas with green if you wanted.

Here are the settings for the upper torso, sorry to say no magic here:

Out of curiosity, what camera shot this and what codec was the original camera master in?

MtD

[Meg The Dog]

Also - took me a minute to find this - you probably don't have time right now but I highly recommend this 5 part series on keying in After Effects by Chris Zwar -

"If you’re wondering what I think the single most important thing is when it comes to keying, then it’s not something technical. It’s not to do with the camera or lighting. It’s not a particular plugin or other piece of software. It’s not a secret technique hidden away in After Effects that only 3 people know about. It’s nothing like that.

It’s attitude."

https://www.provideocoalition.com/advanced-keying-with-after-effects-part-1/ 

MtD

[salvo34]

there's a big difference between 'attitude' and 'dreaming'. Adobe is dreaming by stuffing more and more stuff into a 5 pound bag, programmed at 8 bit originally.. and now up to 64 bit.

Thank God this poster has you as his tutor now.

My 'attitude' is to get stuff done cause I already know how it's done. It's called " years of working on stuff that got done right "

Not some stupid program that keeps automatiing stuff and over designing the UI so nobody knows what's where anymore in the menus.

The mantra " work around " for adobe is very old ... and Anne was probably the most innovative and helpful person to do that for this program.

I hope she is OK in that stupid country she lives in... ( oh, sorry, it's a 'global economy' now)

good luck shooting and editing anything that pays now, cause the film biz in the entire world is dead in the water right now.

[Serega_M]

Meg,

Thanks for screenshot, you have a very different options indeed in Matte Generation and Matte Cleanup section than mine, so I see why the difference 🙂 And thanks for the link to tutorial, I will definitely check it out tomorrow, although I do prefer to do all my work in Premiere (and nowadays you can do A LOT inside Premiere directly, which is good :)) with going to AE when absolutely necessary.

As about the video source, I dunno about the camera, as I was just handed the file to work with, but Mediainfo shows this: http://joxi.net/zANjLDgc6gpdeA

[salvo34]

AE might give better results

[salvo34]

If a green screen ( or blue) is lit properly you can do stuff with one click. Normally that involves stage cyc ( cove sweeps etc. ) rather than vertical wall meets horizontal floor ( sharp line )..

Pan Am was green. Boardwalk Empire was blue outside ( atlantic city ) and green indoors on stage.

https://www.fxguide.com/fxfeatured/pan-am-retro-green-screen-world/ 

doing people ( singles ) is vertical... people stand vertical and don't normally talk and act while on the ground like a snake. A vertical frame would therefore capture MORE of the pertinent image, rather than a horizontal frame. After the key is done on vertical you just rotate it back to horizontal and you have your alpha channel done.

This is especially true if shooting models ( like some movie has models showing off clothing and accessories like black plastic handbags that might reflect some green )...  

anyway, if it's not lit right you might as well just consider it a bunch of MASKS rather than a green screen, cause what's the purpose of using green otherwise ??

[salvo34]

there are some interesting facts concerning the 'article' about how things were done for pan am.

There were targets on the grid ( exactly place via lasers from floor to grid so distance between targets and height were exact ) Took a couple weeks to do on lifts. Those were the 'camera match move targets'... similar to when you put markers on green or blue screens ). It was shot at Steiner studios ( brooklyn navy yard ) and not grumman. AE was NOT live in composites on set ( additional reflections and refractions on glass etc. ).  People do the best they can to write stories about stuff so it's basically a good link to what was going on.. but the still shots were not anywhere close to what the camera was shooting ( f stop and so on ). Kinda hard to explain... but if you were there you would see it.

There were cameras mounted ON TOP of the arri cameras to shoot the TARGETS on grid.. so the match move could be done from that live stream.

🙂

[salvo34]

P.S.

the most amusing thing about that article is the comments below it. They have absolutely no clue what they are talking about re: rotoscoping and so on. It's like kids trying to guess what was going on ... similar to this forom in many ways.  Just guessing, without a clue what really happens on sets and in post houses. I hope to heck this gets better here in the forum... and it stops diving to the lowest common denominator of moronic automation for iphones and go pro cameras.