Skip to main content
A
A.I.1
Legend
March 26, 2019
Question

Is there a good method/software for determining best rotoscoping method & estimating roto time?

  • March 26, 2019
  • 2 replies
  • 1286 views

Different rotoscoping tasks can be easy or complex and time consuming, and a lot can depend on the contents of the shot and which parts need rotoscoping or otherwise separating from the background.

One thing that could affect time or complexity of rotoscoping could be the amount of complex motion in the shot (though it won't always be the case as there might be complex motion in a shot but a client may only want something rotoscoped in it that doesn't have the complex motion).

Is there any available software/quick methods of calculating the complexity of motion in a shot that would give a figure, ie. complexity of motion vectors (like something that could give it on a scale of complexity) such as 0-100 motion vector complexity, that could be used in estimating rotoscoping complexity/time of a video/shot?

Though motion vector complexity doesn't necessarily impact roto complexity (eg. it depends which things need to be removed from the background, I assume motion vector complexity could also be affected by noise or similar but maybe some pre-blurring could be used), but I think on average it would be a good indicator of roto complexity.

-

Another question would be is there a simple way to determine the best (quickest & easiest) way to remove the background from a shot (whether by rotoscoping with masks, roto with rotobrush, or keying etc. or is it just best to try a few?). Are there cases where some use of difference mattes could be used or software that can sometimes automatically/semi automatically remove the background from particular shots (eg. such as a shot with horses charging at the camera with a slight camera move at the end and the horses completely exit at the end of the shot, and the grass would need to be removed and the trees and sky in the background removed. Is there a way of using difference mattes that could work for that (I tried but couldn't get that to work. There's also some movement in the grass)? Is there any current software that could almost automatically do that (separate the horses & riders from the background)?

This topic has been closed for replies.

2 replies

R
Community Expert
Rick GerardCommunity Expert
Community Expert
March 26, 2019

I have been doing roto for so long that I can tell you in about two minutes if the shot is going to take me 10 minutes or 10 hours. The more experience you gain the easier it it is going to be to just look at a shot and know how long it’s going to take.

The most common mistake or someone new to this particular job is to try to do too many things with a single mask. If I’m trying to cut out a person there will usually be eight or 10 masks. One mask for the torso with maybe three or four keep frames, a mask for each arm with maybe seven or eight keyframes, and a mask for the head with maybe 10 or 12 keyframes. I see a lot of newbies putting keyftames for every five or six frames on a mask that  has 40 vertices and includes an entire person. You would have to have an extraordinarily complicated piece of software to analyze a shot and make an intelligent  decision on which parts to include in each mask. A couple of days with a good instructor Is all that you would need to be able to look at a shot and figure out how long it’s going to take to do the roto and be accurate with your prediction.

Another thing that helps an awful lot is knowing when to motion stabilize or even do a modified corner pen track before you do the Roto.

I was teaching a class not too long ago and we needed to separate an actor from the background for about 15 frames while he walked in front of the sign. Almost every student tried to include the entire body for the entire length of the clip.  All that was necessary was to motion stabilize the shot so that the sign didn’t move. Create a mask that covers only the actors upper torso for 15 or 20 frames, then put the motion back in the shot. The entire job took less than 10 minutes for a few but some were trying to Roto the entire 300 frames in the shot And they never finished.

I personally don’t think that running a clip through some kind of analysis software is going to be anywhere near as efficient as a little experience and just taking a look at the shot.

A
A.I.1Author
Legend
March 26, 2019

I agree that with many vertices on a mask it can be difficult. Sometimes it needs a lot of vertices for when it's closeup but really needs less when it's in the distance (though if possible it's probably best to split the clip into sections I assume for that).

I agree that where possible more masks with fewer vertices per mask should be simpler to work with than a mask with many vertices that covers multiple joints since that should make the joint rotations a lot easier. Though it can also mean you end up with many masks and many masks on screen simultaneously. I think other software can hide masks where After Effects doesn't have that option (an alternative for fewer masks shown could be to duplicate the footage I suppose (and have some masks on different copies of the clip). But Adobe could make it easier to quickly organise masks (eg. so you can have parented masks for simpler organisation and finding the right mask to change.  In theory rotoing things like people or horses etc. could be done using a joint type system for the masks. That should make it faster.

I personally don’t think that running a clip through some kind of analysis software is going to be anywhere near as efficient as a little experience and just taking a look at the shot.

If the auto-analysis type software isn't best software could still help in giving a good estimation based on a few things you typed in or selected (maybe a spreadsheet calc could be used instead). Sometimes a quote is needed for a video (or maybe multiple videos) before you see the video and only know a few things (such as the approx duration and the resolution). Maybe such software (or spreadsheet calc) could give a series of quotes/expected times based on different assumed complexities of the (as yet unseen) video(s) eg. "if the video contains x/y/z the quote would be X and time Y". But such software could also be used to give a fast, more accurate quote/estimated time when you have seen the video (even without the motion vector complexity value, even though I assume that would help), just by selecting/typing a few values for it.  Though it probably isn't a simple calculation. Maybe neural network software could determine which factors (about the videos) affected roto time and complexity and how much, or maybe data/statistical analysis could, based on past work.

But those factors that affect it and approx how much should be relatively common to all roto artists (eg. it should be common info that could be used if we can determine that data, so we shouldn't have to wait to have worked on a similar shot before before giving as accurate quotes/times as possible for it).

R
Community Expert
Rick GerardCommunity Expert
Community Expert
March 26, 2019

I think you are way over thinking this. Roto work is almost always different for each shot. Each setup requires a different approach. In all my years in the visual effects and movie making business, I've never even considered taking a talking head or a presenter and removing the background from several minutes of video. It's just not practical.

Rotobrush can greatly speed up some types of roto work, but most of it, nearly 90% of the roto work that I do is done to fix some kind of overlay. I've done some very complicated moving actor roto. One project had a general mask for most of the face, two masks on a hat, about 5 masks on the hair, and a ton of motion blur and feathering going on, and I think I even had a separate mask for the actors nose, but that kind of roto job is usually just a few frames. If you needed more than three or four seconds of that kind of roto work then you probably need to reshoot or ask for a couple more days of production time in the budget.

The key to successfully completing any masking or keying job is in planning the shot and executing it well. No client can afford to have you just take a camera and shoot something and then decide they need to remove the background. I don't think we are anywhere near having software that can look at a scene, decide what needs to be done, and give you a time estimate or set a budget. All the motion prediction in the world isn't going to be able to figure out which motion is the actor and which is the background, and what parts need to go and what parts need to stay. Software can figure out is where edges are (mask tracking can seriously speed up Roto by the way) but it can't figure out what you need to do to make the shot work.

Dave_LaRonde
Dave_LaRonde
Inspiring
March 26, 2019

You're not going to like this answer:  experience. 

There's nothing automatic about rotoscoping.

A
A.I.1Author
Legend
March 26, 2019

But surely if there is software that gave a complexity level of motion vectors that could be used in a calculation to help determine roto time. Or maybe it could be used as an indicator of maximum roto time.

edit: Maybe the motion vector info already stored in a H264 video is shown by some software or could be used in some calculation to give an average motion vector complexity figure for the entire video which could then be used in a calculation for estimated rotoscoping time.

There was one program years ago that could remove an object from video by just placing a rough mask on the object (though I think it depended on the shot). Maybe it would need a trained AI program to auto-separate particular objects and they aren't really available yet. But maybe there could be a best procedure (if not software) that could give the likely best method for a particular shot.

You're not going to like this answer:  experience.

Part of that could be from knowing what a particular shot contained and how long it took with particular methods. In theory some of that info could be put into an algorithm/calculation to give an approx roto time as well as maybe a best method for a particular shot.

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices