How to rotoscope 360 footage

[valeriyn63141698]

Hello,

I want to separate the subject in my 360 video from the background, and then substitute the background with a clean plate so that I can make the subject appear into the 360 scene with some interesting effect, some time after the viewer starts watching.

I tried a 2D VR edit, a 3D VR edit, VR converter and a few other things, but if I use the rotobrush tool to extract the subject from the background, that works only inside the Edit. As soon as I go to the Output, the background that was supposed to be transparent returns, and I can't overlay the rotoscoped subject over anything else because all the transparency is gone. If I forego the entire VR workflow and work directly on my equirectangular footage (i.e. rotobrush), then as soon as I apply a VR effect like VR Digital Glitch the background returns again and the transparency is lost. I also tried precomps in case the VR effects are not reading the right material but it made no difference. After spending a few weeks on this I am starting to run out of time, and not sure what else I can try. I couldn't find any tutorials, only the ones for clone-stamping the camera rig out (not useful in this case).

Anyone know how to achieve this?

Thank you.

[Rick Gerard]

Masked footage is masked footage. There is something wrong with your workflow, but we can't see what you are doing because you didn't include a screenshot that shows the modified properties of the layer that is giving you problems. Just press the U key twice, PrintScreen, and Paste....

After you complete the roto work enable the transparency grid to verify in the comp by dropping a solid below the layer.

[valeriyn63141698]

Sorry for the delay. The screenshot you asked for:

Here's my workflow:

1. New project, bring in clean plate and subject clips (equirectangular footage)

2. New 2D VR edit (see Fig. 1)

3. Select rotobrush tool, double-click on "subject...Precomp1" layer in VR Edit 1 comp that opens

4. Rotobrush some frames (quick and dirty for demo purposes)

5. Click Toggle Transparency Grid (see Fig. 2) - transparency works

6. Save edit, open Output/Render - transparency gone

7. Create a new comp from clean plate, drag in VR Output - no transparency, new layer covers existing layer

Here's the VR Edit with a solid put behind it:

Here's the VR Edit rotobrush layer with checker:

Here's what it looks like without the solid...

Expected result: VR Output for rotoscoped VR Edit has transparent background that blends just the subject with the clean plate.

Observed result: VR Output for rotoscoped VR Edit has opaque background from the original subject clip before rotoscope. "subject (VR2 Output)" layer does not have transparent background that blends with clean plate, instead it replaces the clean plate, "plate.MP4".

(I'm guessing my assumptions here are wrong)

After your tip with the solid behind the rotoscoped VR Edit I noticed one funny thing. If I leave the solid in, it actually shows it on the background as expected, distorted to fit equirectangular view:

...but if I take the solid back out, the transparency is gone again.

[Rick Gerard]

I'm confused. What is wrong with this screenshot:

It looks to me like you have placed your actor in the scene.

[valeriyn63141698]

The clean plate and the actor were shot from a tripod at exactly the same location and orientation to make it easier to roto-brush (i.e. if the brush includes too much of the background you couldn't tell).

In the above screenshot you re-posted, "subject (VR2 Output)" has lost its transparency and became opaque. It's covering the "plate.MP4" which is the untouched clean plate. If you look at the checkered or pink solid view again, the roto-brush was very rough so the fists are hanging in space with wrists missing. In this screenshot, the actor is intact with no artifacts because there is no transparency.

[P.M.B]

valeriyn63141698  wrote

(I'm guessing my assumptions here are wrong)

I'm voting this my favorite statement of day.

Of course because of the inherent paradox that makes the statement both true & false at the same time.

[Rick Gerard]

Your rotobrush work is not complete or very good. That has to be perfect before you go any further.

I don't see a screenshot that shows the "subject (VR2 Output)" comp so I don't know what I'm looking at. The only comp that I see is the Plate comp. You have a layer below the nested "subject (VR2 Output)" comp so that should show through behind the actor and it looks like it does.

If you are trying to put the actor in a scene, you have done that. If you want a transparent background you need to get rid of the bottom layer.

[valeriyn63141698 · Accepted solution]

I finally found what was wrong. The VR Comp Editor creates a comp called "<originalcomp>(VR Precomp Combined)". That has VR Edit X overlaid on the original. That's why rotoscoping inside of VR Edit only has the effect within the Edit, and as soon as you go to VR Output the transparency is gone. It's overlaying a clip with a transparent background on top of exactly the same clip, which has the effect of ignoring any changes to transparency.

The solution is to open VR Precomp Combined for the target VR Edit and hide the layer with the original comp. I feel like that was more of a "convenience" feature from Skybox developers to make editing in VR feel seamless, but it was interfering with my workflow since it made the assumption that i would want to compose all edits on top of the original being edited.

Thank you for trying to help Rick, but it wouldn't have hurt you to try using the VR comp editor before replying. You might be getting more VR questions soon, since many new cameras are on the market.

[brandona30816091]

Are you making a 2D Edit when you use Rotobrush 2 in a 360 environment? 

I can't get it to work in 3D which isn't a big deal for this project, but I'm trying to figure out if it's possible.

[valeriyn63141698]

Yes, it's a 2D VR Edit. Please see the earlier post where all the steps have been described:

https://community.adobe.com/t5/after-effects-discussions/how-to-rotoscope-360-footage/m-p/9910890#M6...

The modified steps with the solution that makes it work are:

1. New project, bring in clean plate and subject clips (equirectangular footage)

2. New 2D VR edit

3. Select rotobrush tool, double-click on "subject...Precomp1" layer in VR Edit 1 comp that opens

4. Rotobrush some frames so that the background is removed and becomes transparent

5. Click Toggle Transparency Grid to verify transparency works

6. >> Open "<originalcomp>(VR Precomp Combined)" comp for the target VR Edit and hide the bottom layer with the original comp. This way the subject with background removed will not be composed on the original source, which would have the effect of canceling the background removal.

7. Save edit, open VR Output and verify that the background is transparent

8. Create a new comp from clean plate source clip, drag in VR Output - this will overlay the VR Output which contains the subject with background removed over the clean plate