Known Participant

Question

Automatic rotoscoping based on movement?

Forum|Forum|6 years ago
December 4, 2019
5 replies
6274 views

I'd like to have After Effects automatically rotoscope anything that moves in a scene.

Can this be done?

What I'm trying to do is this: I have footage of people performing against green screen, but the green screen has already been replaced with a static background image. I want to replace the static image with a new image, so theoretically if AE can just detect any pixels that change, then it should be able to automatically rotoscope for me.

How to

This topic has been closed for replies.

L

Lee JamesAuthor

Known Participant

Update: Just for the record, the feature I was asking about here has been implemented in DaVinci Resolve 17 - it's called the "Magic mask" tool.

So those here saying such a feature couldn't be done were wrong, frankly.

D

doesntdoot

Participant

Yeah, I was just looking for this because I saw someone do this in AE in TikTok, at least I could have sworn they did this in AE. I would HOPE AE has it, because even basic webcams are starting to have this feature

xxxgreg

Participant

This - why hasn't this happened yet in AE?

Nathan Lovell_52

Inspiring

My background is writing scripts and extensions for After Effects, as well as recently learning a lot about plugins. What you are describing would be best solved by what others here have suggested, or by some sort of hybrid approach using masks or the rotobrush. You will need human interaction in order to do what you want accurately.

In terms of what you are actually asking for, it requires machine learning, vision, and most certainly a GPU. This does not mean it is impossible, it would definitely be possible to create some sort of edge recognising plugin that could that make the pixels outside of a radius transparent. But creating a plugin like this would be an extremely costly and difficult process, which in the end would only benefit a handful or people who have incredibly powerful systems capable of handling such a processing load.

xxxgreg

Participant

I second the OP's original question - the AE project team appear to have dropped the ball on this feature. Photoshop is frequently improving automatic object isolation, AE has hardly advanced its auto roto in quite a while - by now I would have expected the AE team to incorporate Photoshop's object identification AI together coupled with scene movement to be able to automatically roto moving objects in a scene. I'm sure it's not just me - there will be TONS of footage from MANY PEOPLE that need background replacement. It's a completely missed opportunity.

Regarding suggestions of creating difference mattes - these only work well under optimal conditions - I was running difference mattes about 15 years ago when necessity required it, I would have thought AI would have moved the needle in that time. Diference mattes are also difficult to impossible to properly implement after the fact.

Here is a listing of the latest updates the May 2020 release for AE - basically Adobe is only doing maintenance releases of AE nowadays:

1. Tapered shape strokes
2. Concentric shape repeaters
3. ProRes RAW import support
4. Automatically update audio devices (macOS only)
5. Copy media to shared location
6. Enable collaboration on cloud documents

Come on Adobe, you're dropping the ball with AE - give it some much-needed attention and TLC.

R

Rick Gerard

Community Expert

If the difference key doesn't fix it, and it can't if you don't have a clean frame with the replaced background, then you are out of luck. That's the only tool that AE comes with that might possibly offer a solution. If you showed us a screenshot of the video, or better yet, if we could see the video we may be able to offer some other suggestions.

L

Lee JamesAuthor

Known Participant

Thank you both. I don't have the original green screen footage, and there's no point me showing an example as I wanted to use it for all manner of different purposes.

I tried the difference key, thanks. It does a basic job of what I wanted, and I'll use it, but it's obviously not the tool I wanted.

I want a tool that simply identifies movement in the video and rotoscopes it for you. I don't see why After Effects doesn't have such a tool. It would be incredibly useful. It would mean you could do "green screen" without needing a green screen, you just shoot against a static background and After Effects would cut you out automatically.

I think I'm going to have to request this feature.

Dave_LaRonde

Inspiring

"I want a tool that simply identifies movement in the video and rotoscopes it for you. I don't see why After Effects doesn't have such a tool."

Well, that still exists only in the realm of magic.

Dave_LaRonde

Inspiring

You can try using a difference key, but I can't guarantee success. It's an extremely picky keyer. Personally, I'd do everything I could to get the original green screen footage.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded