Tracking surfaces in AND out of frame

[GeneralServe]

Hello all,

I have a design project that I need help on. So my footage is an Enscape rendering of a 3-D modeled environment of a drive to a destination from point A to B that I created in SketchUp. I added high-contrasting textures to specific walls within the model itself that I want to track and overlay TouchDesigner effects onto. I have tried tracking the camera, Mocha AE, and Mocha Pro to no avail. For some reason, I cannot get either the native tracker or the Mocha tracker to register the walls as they come in and as they leave the frame. 

I have attached the footage I am trying to use, as well as the TouchDesigner effects I want to overlay. 

Links to tutorials, advice, or any point in the right direction is greatly appreciated.

Thanks!!!

Clip 

Touch Designer Effect 

[Mylenium]

That workflow makes absolutely no sense and your scen e is completely unsuitable for tracking. That's simply not how this stuff works. For one, the setup is "implausible" as far as the underlying math of a tracker goes due to it having been generated artificially. Two, the motion is way too fast for any tracker to even latch on. Three, there are no persistent details visible that would even allow to be tracked consistently for a reasonable duration. You are even starting out zoomed in to a white wall. That makes no sense to any tracker. Four, the contrast of the whole thing overall is completely insufficient and, kind of a 4.5, there are no discernible patterns even on the marked walls, so e.g. mocha's planar tracker can't even determine whether there is a plane. This will never work. Your only hope is to export actual 3D data using scripts or a standard format like Collada and using other scripts and plug-ins or e.g. Cinema 4D/ Cineware to transfer it to AE. Otherwise you need to completely rethink your strategy.

Mylenium

[GeneralServe]

Thanks for letting me know. I am entirely new to any sort of VFX/Editing. Do you have a better workflow/tutorial you can lead me toward? 

[ShiveringCactus · Accepted solution]

Mylenium is right in everything he is saying, ideally you want to get that model exported out of sketchup and bring it into AE.  You should be able export the sketch up model (then convert in Blender or possibly directly) import into C4D Lite.  That will give you a model in 3D space which you can then add a camera too and animate.  This avoids all tracking issues as you can place your texture layers in the same 3D space and line them up on the walls (or if you are in C4D Lite, you can add the textures into the model).

But just in case that is not an option, render out your animation at 1/4 speed.  This will provide a lot more frames and then I wouldn't bother using a tracking program.  These are designed for real world cameras and the lack of information doesn't work too well with 3D animations.  Instead, just use the Power Pin effect and manually keyframe your textures (you'd be doing a lot of work on the corners anyway, so you won't lose any time doing it manually).  Find a podcast, use the Page Down key and set keyframes every 3 - 5 frames and you'll be done in no time.

And once you're done, drop your slow comp into another comp and speed it up to the speed you want.

[GeneralServe · Accepted solution]

Hey Cactus,

I am unfamiliar with all of this. Could you point me in the direction of a tutorial or how to do what you're saying here in this reply? 

Background on my data:

I originally took GIS map data from ArcGIS in Illustrator, exported only the building outlines and brought them into Rhino. Then, I ran a random-extrusion GrassHopper script to get the varying building heights. I then brought that 3D model into sketchup to render with. 

[ShiveringCactus · Accepted solution]

I don't know about any of the stuff pre-Sketchup, but from that program you can export an OBJ file.  You could either import this directly into the latest version of After Effects, but I think you'd be better importing it into C4D Lite.

To do this:

• In After Effects, create a new Cinema4D file.
• C4D Lite opens, and here you can import your OBJ from sketchup.
• In C4D's materials panel, you should be able to identify the texture you want to replace.
• Import your image file into this texture.
• Save everything and return to After Effects
• Create a 3D camera and with your C4D file as a layer in a comp, fly your camera around.

The reason I'm suggesting using C4D Lite is that the OBJ import to AE is so new, I've not seen how to go about changing textures / materials yet.

The other approach I mentioned is it to use Power Pin effect.  Jake in Motion has a good quick video on that:

https://www.youtube.com/watch?v=NeUrbL0oadc

[Mylenium]

Rhino generates clean UVs, so you could extract whatever faces require your color drop texture to separate objects/ a separate object. Then it's either merely a matter of importing that into Cinema 4D and applying the texture or using a technique like this:

https://www.youtube.com/watch?v=5s6Hf68TE34

Of course you can do all of that just the same in Rhino or Sketchup right away without any detours. Doing it in 3D would have the advantage of not having to deal with obscuration/ overlap issues, which otherwise would require generating extra passes for track mattes and masks. Point in case: You basically already are paying for your tools that can do all that is required, but it seems to me you never ventured beyond rendering stills for client approvals and may need to spend some time learning more techniques. There's basically zero reason to involve AE just for getting your wallpapers pinned to the walls.

Mylenium

[ShiveringCactus]

That's a nice tutorial video showing a great use case for UV mapping.

[GeneralServe]

Hey Mylenium, 

There's a slight misunderstanding about what I am trying to accomplish. 

I am trying to projection map a video onto these blank canvases that happen to be buildings; this is for a college design studio class.

[GeneralServe]

Thank you both for your input!! I appreciate the help, you all, and your thorough explanations. I appreciate you all being kind and helping someone in need! ❤️

[Mylenium]

If you need to project the textures on the whole scene, it's even more reason to stay in 3D land. Simply use planar texture projections scaled to the full scene and aligned with the camera or specific faces.

Mylenium