A few questions on rotoscoping efficiency

[peterg2036972]

I read that to speed up rotoscoping 4k footage it can be helpful to just crop to the region of interest.  And also apply some contrast adjustments to better define edges

If I apply some color correction effects onto the footage (such as contrast adjustment), does the rotoscoping brush work off the effected image, or the original footage regardless of any applied effects?

If I wanted to rotoscope footage with contrast effects applied, do I have to make the footage with the effect into a pre-comp?  And if so, how do I use the Rotoscope Brush on a pre-comp?  Clicking the Pre-Comp just opens it up to the original footage but does not give me access to the window that allows me to paint with the Roto Brush.

Using a Proxy – I know this is generally not advised for rotoscoping but in my instance the lower res 2k version still has good clarity.  Will the rotoscoped outline be automatically resized when I substitute the proxy back for the original 4k footage?

[Warren Heaton]

***If I apply some color correction effects onto the footage (such as contrast adjustment), does the rotoscoping brush work off the effected image, or the original footage regardless of any applied effects?

Effects render in the order that they are applied; however, with since we work with the Roto Brush in the Layer panel where individual effects can be idolated, I'd suspect that AE works with the un-effected layer source and go to the trouble of nesting.

***If I wanted to rotoscope footage with contrast effects applied, do I have to make the footage with the effect into a pre-comp?  And if so, how do I use the Rotoscope Brush on a pre-comp?  Clicking the Pre-Comp just opens it up to the original footage but does not give me access to the window that allows me to paint with the Roto Brush.

Open the nesteed Comp in the layer panel by right-clicking and choosing Open Layer.  Also, command double-click does this on Mac (it should be control double-click on Windows).

***Using a Proxy – I know this is generally not advised for rotoscoping but in my instance the lower res 2k version still has good clarity.  Will the rotoscoped outline be automatically resized when I substitute the proxy back for the original 4k footage?

For best results with the Roto Brush, your footage should be full-frame, full-motion with the Comp Resolution setting set to Full.

If you set 2K source as the Proxy for 4K source, you'll be working at 4K the entire time without the detail or accuracy of your 4K.  Until you use the Freeze button, you'll even see the pink roto brush lines shift a little as you disable/enable the Proxy.

A general note on the Roto Brush:  I find it best for doing rough roto work to quickly work out a composite and then follow up with the stronger tools (Mocha for AE, Mocha Pro, and/or Silhouette FX). 

[Rick Gerard]

Efficient roto, no matter what toolset you use, starts with Shot prep. If it is not shot with roto in mind you are going to have a harder time producing the matte. The goal of all roto is to produce a matte. More than 90% of the time I do not just roto on a footage layer, even using Rotobrush. I will almost always do some kind of color correction on footage that I intend to roto, even if it is only temporary. For Rotobrush, this usually means duplicating the footage, adding some color correction and maybe even masking to the footage, pre-composing, then opening the pre-comp in the layer panel* and running Rotobrush only on the frames and the part of the image that needs the mask. I hope you followed that. I can't tell you how many times I have helped people with their roto projects and seen an entire actor's body cut out when the only part of a 15-second shot that needs roto is the 20 frames of the actor's shoulder.  

* You can always open a nested comp in the Layer Panel by using the Layer>Open Layer Menu or by a right-click in the timeline.

Let me show you a recent example:

Here's the shot. It is about 12-seconds long. The actor walks to the phone and answers, focus follows the actor. Only about the last 20 frames have the actor in front of the TV. This is the final composite showing an animated shape layer (turned on so you can see through the layer and see the two paths I had to keyframe manually.

This was actually my second attempt at generating a matte. My first idea was to give Rotobrush a try. The original layer was duplicated, trimmed to the first frame where the actor's shoulder needed a mask, a mask was added, some color correction was applied to try and generate a good line. This is 6K Red footage by the way. You can't get a much better image than that. It looks promising.

I used curves and Change Color to try and improve the edge between the TV screen and the actor. The entire thing was pre-composed and then I opened the nested comp in the Layer Panel and tried Rotobrush. This is what the matte looked like after just 12 frames.:

When you open up the nested comp in the layer panel you will see all of the footage but Rotobrush will be limited to just the area inside the mask. After about 20 seconds of propagation, I could tell that the change in focus was going to be something that I had to handle with some hand roto and Camera Lens Blur.

So that is one example of how to efficiently roto something. If I had handed this shot to a half dozen students most of them would have not trimmed the roto to just the 20 frames, a few of them would have tried to mask out the entire actor, and a few of them would not have even thought of using an animated shape layer as a track matte for the screen replacement. 

So I guess that is at least part of the answer to the question you asked. Efficient roto requires you to analyze the shot, make a good try at what you hope will be the fastest solution, then change the workflow when you run into trouble so you can get things done. The most important thing for you to learn is how to figure out how to do roto with the least amount of effort. If it was not for the change in focus and the continuously softening edge of the actor's shoulder Rotobrush would have been able to create the matte that would be used as a track matte for the TV screen in about 10 minutes. Creating the matte by animating a shape layer path took me about 12 minutes. It was the shortest part of the entire project.

[peterg2036972]

Thanks for the feedback.  My footage is just of a rotating bottle from some product photography.  Roto Brush was giving very poor results but I managed do get a much better mask using Mocha Lite (which I completely forgot was included with AE).  Mocha seems to be much more responsive (I am guessing it is less processor intensive than Roto Brush – maybe due to planar tracking?).

I only needed to sperate foreground and background for different color correction and effects.  The Mocha mask was still somewhat problematic with a fair amount of jerking – but I was able to fix that by inverting the masking to work on the background instead of the foreground object and so the jerking of the outlines were not noticeable.

Anyway – thanks again for all the advice.