Feature Focus: Roto Brush 2

Report · Jun 23, 2020

Hello Beta Testers!

We are pleased to inform you that Roto Brush 2 is now available for beta testing in the 17.1.2x32 build of After Effects (Beta). This new propagation engine incorporates the technology behind Project Fast Mask from MAX 2018 to streamline rotoscoping/isolating a subject or object from a background.

The amazing notiansans from Learn How to Edit Stuff has created a great tutorial video for Roto Brush 2 in After Effects (Beta) showing the basic workflow as well as a comparison with original Roto Brush.

What We Need From You

We are looking primarily for feedback on how the new propagation engine is performing with real-world footage, though all feedback is welcome. Your testing efforts and input are highly appreciated.

If you are willing and able to share footage with the After Effects team where Roto Brush 2 is having difficulty isolating a subject when propagating, please follow these steps to submit the footage and project:

In the Project panel, select the composition that contains the footage with Roto Brush applied.
Go to File > Dependencies > Collect Files
In the Collect Files dialog:
- Set "Collect Source Files" to "For Selected Comps"
- Enabled the "Reduce Project" checkbox
- Click "Collect" and select a location to collect the project file and footage
Zip up the folder containing the newly-collected project and footage.
Add the .zip to your Creative Cloud storage ( or Dropbox, OneDrive, etc ) and email a link with a brief description of what you were experiencing to rotobeta@adobe.com. All footage submitted will be kept confidential and will not be shared with third-parties.

Getting Started with Roto Brush 2

The application of Roto Brush with the new propagation engine is largely the same as in previous After Effects versions:

Select the Roto Brush Tool in the toolbar ( Alt/Option + W ).
Double-click on an existing layer in the Composition panel. This will open up the layer in the Layer panel.
Paint a green foreground stroke on the subject, object, or region you are attempting to roto. Avoid painting across edges as this might confuse the selection. This creates a base frame ( visible as a blue highlight in the Time Ruler of the Layer panel. )
Refine the selection by painting additional foreground strokes, or paint background strokes by holding Alt/Option to remove portions of the selection.
Foreground stroke, adding to selection:

Background stroke, removing from selection:

Once you are happy with the selection on the base frame, press the Spacebar to start propagating the selection through the footage. This will trigger the propagation of the matte from the base frame to the new frame. You can also use Cmd/Ctrl + Left/Right arrows to propagate frame-by-frame either forward or backward, if you would like to go slower and evaluate the propagation more closely. This might be more ideal for footage which requires more correction strokes.
After you have propagated the matte through all of the video frames you need and you are happy with the result, you can freeze the propagation using the Freeze button at the bottom of the Layer panel. After freezing, you can still make adjustments to the matte itself in the Roto Brush effect without re-propagating. If you need to add or remove from a selection after freezing, you can unfreeze the propagation with the same button.

Correcting the Roto Brush matte:
As Roto Brush propagates the matte from frame to frame, the selection may start to include unwanted regions or "lose" portions of the original selection. To correct this, paint a background stroke ( holding Alt/Option ) on the first frame where the unwanted change occurs to remove unwanted regions, or paint a new foreground stroke to include anything that was missed. As with the base frame, avoid painting across the edge of the selection as much as possible.

Adjusting and Refining the final matte:
The result of the propagation can be further refined and improved via the controls in the "Roto Brush Matte" group in the Roto Brush effect, especially with the "Reduce Chatter" property. Additionally, the Refine Edge Tool can be used for highly-detailed edges like hair and further controlled in the "Refine Edge Matte" property group. There are also options that compensate for motion blur and decontaminating edge colors.

Use the toggles at the bottom of the Layer panel to view the matte results, or open the Composition panel to view the result in-context with other layers.

Toggles in Layer panel for viewing alpha, alpha boundary, and alpha overlay

Additional Resources

For more details on the Roto Brush workflow, below you'll find links to both Adobe documentation as well as tutorials from Roto Brush users.

Please note: Due to the enhancements in the new version, some of the tutorials may be outdated or encourage practices which may not yield the best results. For example, painting very close to an edge may result in a poor result if any pixels from the background are accidentally included in a selection.

Thanks very much for trying out After Effects (Beta) and Roto Brush 2!

Report · Jun 27, 2020

I'm having trouble testing Roto Brush 2. It loses the selection from 2 frames, but when I switch to Roto brush 1.0 everything is OK... I use Mac Pro (2013 ) OSX Moajve 10.14.6.

Report · Jul 14, 2020

Hi woytekz,

I'm sorry to hear that Roto Brush 2 loses the selection so quickly on your footage. Would it be possible to send a collected project with the footage to rotobeta@adobe.com? A screen recording on the behavior would be helpful as well so the team can see how the new Roto Brush propagation engine is falling down. Additionally, what GPU is being used in the 2013 Mac Pro?

Thanks very much for any further information,

- John, After Effects Engineering

Report · Jul 20, 2020

Hi John,

I use Mac Pro 3Ghz 8-core Xeon E5 with 64GB.

Everything works OK, with Rotobrush 1.0, but when I switch it to 2.0, the selection loses

Cheers,

Woytek

Report · Aug 12, 2020

Finally, this feature works for Mac Pro 2013!
I checked it out on version 17.1.4 (build 13 )

Cheers,

Wojtek

Report · Jun 30, 2020

Must say I'm impressed so far. Big improvement over the first roto brush, that's for sure. Though, it still has difficulties with concave details which it tends to smooth a lot, like between nose and mouth on a profile shot. Using "Best" quality helps a bit for this matter, but it still could be improved. It also tends to include background elements too much to my taste during propagation, especially with straight elements like wall corners or grids. Like it propagates well for the first say 20 frames, and at 21 BAM! It decides to include something in the background out of nowhere. But again, a lot better than before, it is a huge time saver for the project I'm working on. Shots are still, but lighting is flickering and it handles that pretty well. It only happened two times on about 20 shots that when the light was flickering a bit too much, the refine edge brush strokes would get crazy.

Report · Aug 10, 2020

Hi,

This might be a temporary solution but try to get better edges using color correction methods.
After applying the masks just remove the color corrections and the masks should stay 🙂

(not sure if this still works but it used to).

Cheers

Report · Aug 16, 2020

when ROTO BRUSH 2 WILL BE RELEASED OFFICIALLY

Report · Aug 17, 2020

Our new public beta program allows us to get feedback (including crash reports) from users to help us make that decision. So the answer is: when we have the evidence based on feedback that we’ve fixed all important bugs and the feature is working well in the real world.

Report · Aug 18, 2020

We are not allowed to share footage but I did try rotobrush on various clips and while better it still wont cut it for our needs. If you think about red Carpet footage where celebrites are standing in front of busy backgrounds, people moving around in the background, lighting is poor, camera flashes going off, camera zooms. Really the worst conditions for rotoing. was hoping Roto 2 would be able to better handle these situations.

Report · Aug 18, 2020

Hi Janud, we’re always looking for ways to improve the segmentation results when using Roto Brush 2. One way we do this is to "train" the machine learning models by feeding it problematic footage. We'd very much like to analyze the footage you are referencing. If you can get permission for us to only use the footage in-house for development use, then please send to rotobeta@adobe.com

Report · Sep 03, 2020

I've tried the rotobrush 2 in the beta version of AE, but it simply does not work! Everytime I get an error: After Effects (beta) warning unknown

Report · Sep 03, 2020

It's working again, in build 17.5x010. Follow this forum thread for deails as we update the post:

https://community.adobe.com/t5/after-effects-beta/rotobrush-2-no-longer-working/td-p/11403445?page=1

Thanks.

Report · Sep 04, 2020

Have been using Rotobrush infrequently for about a year, and now Beta 2.0. It's much better. However, it's still frustrating when a previously straight line loses its straightness over time, probably due to a lack of contrast between foreground and background. 1. Is there any way to add a keyboard command to teach Rotobrush that a straight line never changes over time? 2. Is it possible to change the contrast between foreground and background to help Rotobrush distinguish? Thanks.

Report · Sep 04, 2020

Is anyone else confused by this, from the above instructions: "After freezing, you can still make adjustments to the matte itself in the Roto Brush effect without re-propagating. If you need to add or remove from a selection after freezing, you can unfreeze the propagation with the same button." My confusion is this: What adjustments can you make to the matte without unfreezing? Actually, the whole propagation and freeze thing is confusing. When I freeze, it starts to propagate (or possibly render, or save or ?? and it takes ages. Then unfreezing takes ages. Then clicking anywhere begins propagation again, taking ages. 3 days later on a 7 second, admittedly complex, clip and counting.......

Report · Sep 06, 2020

I'm having a similar issue with propagation. It just runs and runs and runs and runs and runs....
I used to be able to let it run and it was done in like a minute or two. I just give up when its at Propagating: 200 of 4000.

One of the recent updates seem to have broke the engine; its not acting at all like it did in the previous versions. I can't believe no one else is seeing this (I posted in AE Beta forum).

Report · Sep 06, 2020

This update is awesome !

Unfortunly the beta version of After Effect is really unstable and there are a lot of bugs, that not allowed me to continue my composition...

When the rotobrush 2 will be included in After Effect CC (not beta) ?

Report · Sep 17, 2020

Wow! The speed of propagation is kinda mind-blowing. One thing -- not sure if this is a bug or a documentation error: When I hit the spacebar to start propagating, things freeze -- if I just move the playhead to the end of the clip, success.

Report · Sep 21, 2020

Yes, this is a known issue, AE stops responding when previewing in the Layer panel. A fix should be available soon. In the meantime, continue moving the CTI to start propagation. Thanks

Report · Sep 18, 2020

Hi,

When it will be available on production version? I thought that because it was available on beta version 17.1.4, with the official release of the same version that will bring the roto brush 2.0.

Thanks

Report · Sep 21, 2020

We ship when the feature is ready for public release. We’re getting close. We make that decision partially based on feedback from our Beta users. So thanks for contributing.

Report · Sep 23, 2020

Hi James,

Thanks for your reply. Good to know that you are getting close to one of your most revolutionary tools so far, for rotoscoping. The roto brush 2 is fantastic!

Cheers

Report · Sep 23, 2020

OK, new bug.

When I roto out my object, and press play to auto-propagate, it crashes AE. I have to click through frame-by-frame.

This has happened for about a week, now, even with the latest update.

Report · Sep 23, 2020

Yes, that's correct. Previewing in general is broken in Layer and Footage panel views. Issue is not specific to Roto Brush but affects ability to view propagation in real-time. We're actively working on fix. In the mean-time-you can work around the problem by moving the CTI to your target frame to induce propagation.

Report · Sep 23, 2020

OK. Wasn't sure if anyone brought my issue up yet.
I'm good with going frame by frame for now but I'll also give your suggestion a shot.

Thanks!