Now in Premiere Pro Beta: New Masking Tools

[Francis-Crossman]

We’re excited to push an early development preview of new masking workflows to Premiere Pro (beta).  Whether you are color grading, applying effects, or cutting objects out from the background, these new masking tools are built to help editors work faster, smarter, and more creatively without needing to go to After Effects for simple tasks.

Note: This beta is an early look at upcoming Premiere Pro workflows and is not intended to reflect general availability performance or polish. Please contact us with bugs, feedback, and feature requests to improve these features.

• Object Mask – Automatically identify objects and people in your footage using AI and isolate them with a single click. Plus mask refinement tools and fast tracking throughout clips.
• Redesigned Shape Masks – the traditional shape mask tools (ellipse, rectangle, pen) have been rebuilt from the ground up with many performance and UI improvements.
• Faster Shape Mask Tracking – the tracking algorithm has been completely rebuilt too.  It’s significantly faster than before, more accurate, and offers more precise control for correction.
• Compositing tools – blend any combination of Object and Shape Masks to create complex masks with new blend modes: add, subtract, intersect

To get started

New masking workflows are only functional in beta projects that support an upcoming major release of Premiere Pro. This means your beta projects will not open in Premiere Pro 25 or earlier. We're excited to give you early access with these steps. 

1. Update to Premiere Beta version 25.6 build 48 or later.”
2. If you see this dialog, choose “Enable new masking”
   Launch dialog
   – OR – if you do not see the dialog at launch, or you skipped it, do the following…
3. Open or create any project
4. Enable “New Masking” via the beta menu in the OS menu bar at the top of the screen. Premiere Pro (Beta) will need to relaunch.
   Beta menu
5. From now on, Premiere Pro (Beta) will create projects in version 26.0 format. 

WARNING: these projects are NOT compatible with the current shipping version of Premiere Pro and will not open.

Quick Start guide

Follow these steps to get started with New Masking.

1. Make sure you have enabled New Masking (see above).
2. Select a clip on the timeline.
3. Find the new Mask Tool group in the toolbar. Long-press to see all the masking options. Choose the Object Mask tool.
   Tools
4. Hover the mouse over the program monitor, watch as objects are identified, and click. This will draw a static mask around the object or person.
5. Track the motion. Find the new “Object Mask” in the “Unassigned Masks” section of the Effect Controls Panel. Click on the forward and back button.
   
   Tip: the Effects panel is a different panel from the Effect Controls Panel. Find it in the Window menu.

6. Add an effect. Find the Gaussian blur effect in the Effect Panel. Drop it on the clip in the timeline. The mask will automatically be applied to the effect.

7. Congratulations, you just used Object Mask to blur an object!
8. Now, try to use the other tools in the group in a similar manner.
   

Visit here for the complete documentation. https://helpx.adobe.com/premiere-pro/using/object-masking.html

We want to know what you think.  Join the conversation below.

[FlyingFourFun]

This is legit exciting....  if you end up with a how too video, please post it - I would love to watch one to get into the object mask properly exercised.   I opened a project to work on, and this will now have me side tracked and my project will be delayed, but I am legit excited for object masking in prem pro....

[FlyingFourFun]

I had a chance to try this, and it is impressive, but the selection tool and its refinement needs some seriose work.  I was tryign to track a horse, and it would select everythign fine, and when tracking started, it would pickup other parts of the video no where near even the horse.  

When you removed a section, and retracked,  it would make things worse.  When you added a section that was prev workign fine, it would remove other perfectly tracked sections.

The tool needs a brush tool, and maybe a way fro multiple objects to have there own track (maybe multi object tracks are already there, but I have not found it yet).

Over all, its a really solid start, but the unexpected object selctions, lost tracking and the random like changes when adding in something later removes something that was workign fine before.  I assume the AI training used can/will be improved,  so this is still very exciting.

But please add a resizable brush to it, and muti object tracks that can be locked down when they are good.

I assume at some point you will also be able to select obects that are not auto detected?  so far it seems to be focused on forground objects it detects, and not background objects.

Compliments to the team working on this, it shows significant promise.

[FlyingFourFun]

it seems like you can create multiple objects and rename them if after tracking them, you convert it to opacity mask for example, and then they show up as different tracks:

and then each object you tracked shows up as a seperate tracked item:

I dont belive these are doing oclusion based tracking (meaning that a object already tracked can be cut out of the object/ignored for the new item beign tracked liek you can with Mocha for example).

Again, impressive - and I'm sure the more time you have with this feature and its tweaking it will get much better too....

A full how to video may reveal some things about how to use it that I have not discovered yet, hopefully someone is working on that for us 🙂

As an aside,   if you would like I an willing to provide a short clip of some horses and wagons with people etc if this helps the testing process, just tell mw where to upload it to.

As an aside,  I tried this with a less complex clip, where I had someone talking in front of a barn door, and the track was really good.  There is still considerable jittery edges, and a setting to reduce the edge jitters would be ideal, but actualy very impressive and fast.  It was doign 12 frames per second on the person,  and about 6 frames with tracking a horse with a person on it.

[Arnold Nitin · Accepted solution]

@FlyingFourFun Thanks for your detailed feedback.

I will reach out to you over direct message regarding how to share the videos you were referring to. 

We have a helpx page that might help with some of the questions you have raised. 
https://helpx.adobe.com/premiere-pro/using/object-masking.html

Blend mode of mask can be used to combine the masks together. You can use "Add", "Subtract" or "Intersect" to combine masks. Any number of Object Masks and Vector Masks can be combined to create complex masks. The Blend Mode of a mask operates on the full result of all the masks that came above it in the stack.  

For Object mask we have different modalities to select an object  - Hover & Click or Draw a rectange or lasso around an object to select an object. If an object you want isn't automatically highlighted, you can use the Rectangle or Lasso tool to roughly draw around it. The model will then detect the object within the drawn area and select it for you. 

[FlyingFourFun]

@Arnold Nitin thanks for the feedback and response.  I have shared with you the sample footage with the horse and wagons so you may use it to test with also.

Apologies, I think I missed that helpx link in the original post - blew right past it to start testing I guess!  It was a helpful post, it confirmed a few things for me.  Also great to see that you expressly state your store the object data in its own file/folder and not inside the project file (thank you so much for making that decision, it will help with load times, and file corruption issues a great deal!).

As for the object selection tool, I think that needs to be expanded better or maybe I'm just thinking of how other tools worked too much and taking a second to switch my thinking.  allowing the object tool to detect other objects in whole might be something worth exploring, such as click on it, and tries to determine the object.  I will admit, this doesn't work great in another tool I am using to create garbage matte with, and it may turn out that using the pen to limit the auto detect object area is more ideal.  Could be worth looking at it closer, but its very usable with the pen, and circle selection tool modalities also.

I started using the rectangle tool, not the object selection, and then tried the pen tool - thinking that's what you meant - was about to write that didn't work - then after reading the the word "modalities" in the post, I tried the object selection tool again, but treated it like a rectangle, and wow!  it works as you described...  It took me a minute to understand that you could use the object tool, but treat it like a pen/rectangle....   not sure if there is a better way to denote that for users so we know it has those multiple ways of selection.

I used it to clean up and add to faulty masks - I am not enjoying the working in the program monitor - maybe you are thinking of a way to bring out the clip and allow more precise control etc , with a object mask workspace, or something - but working inside the program monitor is a little frustrating when you need detailed work - I hope you have some ideas around this.

there are a few other little bits such as not being able to move the viewport inside the program monitor while the tracking is happening to follow an object and zoom in and see the work its doing - it seems that when tracking starts all other processes are locked out.

It would be nice to see a stabilize feature to allow the program monitor stay locked on to the tracked object the whole time its processing also, and still retain the ability to zoom in and out while tracking.

I really hope you find a way to allow object track to be used a video stabilize feature,   I have not played with it, maybe that's already possible - but it can be very powerful to track and object and stabilize video against it (foreground and background).

I'm going to play with this somemore,  but so far its a very strong first (public) appearance of the feature.

[BrianLevin]

Will test the masks but in the meantime can I please reiterate that the shortcut key for Toggle Direct Manipulation needs work no matter what panel you have active. Right now you must have the Program panel active, and so it makes it so that playback stops if you previously had it active, but can't disable it without clicking into the Program Monitor to do so. This used to work amazing prior to the new Toggle Direct Manipulation settings from 25.0. Since you are already messing with that part of the program would you please please restore this function.

[Tom_edit]

it does, you're just encountering an issue with premiere's keyboard shortcut system where if they change the scope of a hotkey in an update, you lose the ability to change that hotkey within the GUI, which causes the issue you're describing.
You can manually go into the hotkey xml and remove the problem hotkey yourself, otherwise you need to create a fresh hotkey file. Early 25.x versions of premiere changed a whole bunch of hotkeys that results in similar issues

[BrianLevin]

Oh wow, well that solves like half the bug reports I posted today. Basically I just need to make a fresh new keyboard shortcut file to solve some of my issues because I just tested with cleared keyboard preferences and sure enough the things that are really bugging me aren't problems after starting from scratch. Guess that's an hour I get to waste tomorrow morning fixing this.

By chance what else do you know of that got changed that is having these types of issues because so far it's Toggle Direct Manipulation that I've been complaining about and on 25.4 the hotkey for Zoom Program Monitor to Fit not working in the Program Monitor (it does if I follow your advice and start with fresh keyboard shortcuts).

[Tom_edit]

Those two are the main two I remember off the top of my head. Pretty sure I encountered one or two others but I don't think I ever noted them anywhere

[BrianLevin]

Out of curiosity, do you know of a way for me to indicate in my existing KYS file what has been user modified vs what is unchanged from the Premiere Defaults? I'm thinking I might just fully rebuild my keyboard shortcuts, but I've made a lot of modifcations over the years and don't even know where to begin comparing it against the defaults.

[FlyingFourFun]

Found something else;  trying to remove something that is selected in error; and while holding Alt to remove to it the following error comes up:

The part I am trying to remove from the mask is the little bit of grass in the top right corner of this picture.

Trying to add a missing section of this lamp post also generated the same error, and there was no way for me to add it to the mask to be included:

Trying to select this missing bit of the other lamp post always selects the windows and never the missing bit of lamp post:

These would all be a good use case for a brush tool I suspect where I can manually select the area.  It would link to the other detected masks, and use those to stay in sync if it was having a hard time tracking it (use relative planar data to stay locked on)

Another thing that is a workflow issue, is there needs to be markers for where the mask is drawn, and what part of the clip is tracked and not, and you should be able to use those to find your way back to the correct frame where the mask was created, and to skip to untracked portions of the clip.   Just found out the hard way by hitting the spacebar in error and it started playing the video, how hard it was to come back to where i was building the mask.

[MyerPj]

Very cool feature. I like this idea, AE type features that split the difference between having them locally and switching to AE.

I cut a clip to 2 seconds to try it, and the tracking was done in an instant. Real cool, great new feature.

[Stanley Arthur]

Go and try it for yourself. It is unusable for anything except solid color shapes with no drop shadow or even slightly fuzzy occlusion. It tracks well but the result is garbage. Hair? Forget it. No amount of feathering or expansion helps. I was all set to do a tutorial on this but it's too embarrassing. It would have ended up as an Adobe hit piece. Stick with AE on this one, folks. Once again, Adobe, your competitors are years ahead of you. It's hard to believe you put this out there.

[Ishan Y]

Hey Stanley,

Sorry for the trouble. I've moved this post to the Premiere Pro Beta community. Could you share any screenshots or screen recordings? That will help the engineering team better understand the problem. Thank you for taking the time to give your feedback.

Thanks,

Ishan

Please tag me (@) in your replies so that it notifies me and helps me respond promptly.

[Stanley Arthur]

Thanks, but no. I don't have the time. My advice is to scrap it and start over. Runway did this better three years ago. Even Resolve Magic Mask is better.

[FlyingFourFun]

@Stanley Arthur Its way too soon to present this to the public the way it is, and it seems you are also thinking that's the case.   For something that's just being shown now, its actually not bad, and has uses cases even in the current state.

What I think the issue is that the AI model they are are using to track/trace the object is too small.  I think they are building it for a 8gb GOU base hoping it works - because it seems there is very little GPU memory use when the process is selecting/tracking each frame.  They should consider offering a larger better trained model for those with more GPU memory, and explicitly tell people they need more memory.   One of the failing Adobe seems to have is trying to set really low hardware targets - and I think this is a mistake.

With that said,  I also think that this feature can work better than it currently is with even a 8GB AI model  -  I have provided some sample footage to them as I didn't think it was too complex for the AI to handle, but it was.  However, how it failed was not in how I would have expected it to - and at the same time, it would do other things like track a growing shadow coming into frame  from my selection that was rather small properly.

It was doing a good job of tracking a horse with a person on it, and not grabbing the shadow when I left it out (tracking separate).

I don't see any evidence that the AI model they are using uses any other tracked data to build a new track.  In mocha for example, I can track a easy part of the video frame, turn it off, and other tracks seem to do better because there is that one solid track already.  But Mocha is more for paper doll roto style, and honestly their ML track is also not great right now.

I am hopeful that Adobe puts the work into this and gets something very usable - it feels like its close with tweaking of the training models - with that said, and maybe I am wrong, Adobe needs to stop catering to 'non ideal' hardware and for these advanced features needs to call out exactly what they want/need - trying to get it to work on anything/everything is going to cause them reputation damage. (building for the lowest common thing is always bad news for the Professionals).

if Adobe wants I can test (and I am sure many others will) models up to 24gb GPU support right now (and I hope to get a 5090FE some day for even more memory) - I am convinced that a larger model and more varied training data will be helpful - but finding that training data will be difficult.  

I think Runway had the luxury of starting it early and maybe (in my opnion) everything people uploaded was used to train their models, I'm not sure with the coverage Adobe got on AI model training they have had the courage to try something like that if only to train the tools - aka only train these tools and not for generative uses.

Overall, this tool shows promise, and its only in Beta,  I'm inclined to be open to where this might go - but of course they need to feel the pressure of clients and competition like to be be motivated to spend blood, sweat and treasure to develop it. 

[Stanley Arthur]

Oh, so you want to see how bad it is? Be sure to watch this on a big screen.

[_nicdean]

Day 1,557 of requesting "Shift" to lock mask points moving horizontally/vertically.

https://community.adobe.com/t5/premiere-pro-discussions/hold-shift-to-move-mask-points-vertically-ho...

[Francis-Crossman]

@_nicdean, thanks for pointing this out.  You are absolutely right.  We should have this.  We discussed this in our team meeting today.  Stay tuned for any updates.

[_nicdean]

@Francis-Crossman glad to hear! would love to see this fixed

[FlyingFourFun]

@_nicdean fingers crossed for you and all of us!  This is a legit request...  BTW I tried to upvote, but upvotes are really hit and miss - if Adobe is looking for upvotes to prioritize these sorts of things, I think the inconsistent action of the voting button will make it hard to get a real sense of the interest in anything.

@Francis-Crossman Thank you for taking it up, and getting an project manager assigned to get this coded 🙂  I am looking forward to testing it.

[DAG82]

Hi, this is a great addition to Premiere Pro Beta. I have applied it and it's slection is very acccurate. I might be missing something but how do I apply the mask to the subject, For example when I want to run graphics behind the masked subject? So duplicating the masked layered on top, running the graphics or text behind it and havingn the original layer below.

[kes_akalaonu · Accepted solution]

Make sure to right click on the tracked mask in the Effect Controls and select Change to Opacity Mask. Duplicate your video and remove the tracked opacity mask and placed that on V1 and your masked layer on V3 or above. Place your graphics on V2 and it should show up.

[Kirsi21888295ykon]

Clicking object mask causes Premiere to quit. On my Macbook Pro with same media files it works, not sure about the quality though.

PC specs:

Ryzen threadripper 7970X

128GB

Nvidia 5090 RTX