AI-Enhanced 3D Camera Tracker (Adobe Sensei Integration)

Hi Adobe team,

I’d love to see a major update to the 3D Camera Tracker in After Effects — something that brings it up to modern AI standards and makes it a truly intelligent scene analysis tool instead of just a point solver.

Right now, the 3D Camera Tracker works, but it still feels very manual and outdated comparatively. We get tracking points and a solved camera, but we still have to figure out ground planes, scale,         object anchoring, occlusion, and scene layout ourselves.

What I want to see is an AI-enhanced version of the 3D Camera Tracker powered by Sensei.   Something that understands the scene, not just tracks it.

What This Could Look Like

1. Context-Aware Auto Camera Solve

• AI-assisted solve that distinguishes between static and dynamic objects

• Better drift prevention by ignoring moving people, cars, etc.

• Real-time confidence feedback so we know if a solve is solid

2. Automatic Ground & Room Detection

• Auto floor detection and proper world orientation

• Basic room building (floor + walls from detected planes)

• Scene scale estimation

This alone would save a ton of manual setup time.

3. Full Scene Depth & Dense Reconstruction

• Optional full-scene Z-depth scan

• AI-generated depth map layer

• Dense point cloud or lightweight mesh generation

• Even experimental Gaussian splat-style scanning for better object placement

This would dramatically improve occlusion and 3D compositing inside AE.

4. Object-Aware Tracking
Instead of just random track points:

• Detect and label objects (person, desk, chair, vehicle, etc.)

• Allow tracking or stabilizing specific objects

• Generate 3D nulls attached to recognized objects

• Separate dynamic vs static elements automatically

5. Scene Graph / Structured Data

• Build a basic scene hierarchy (camera, ground, walls, objects)

• Allow us to attach elements to real-world objects instead of guessing track points

6. Lens Detection & Distortion Handling

• Auto-detect focal length

• Detect and correct lens distortion

• Match CG lens settings automatically

7. Modern Export Options

• Export camera, point cloud, depth maps, object tracks

• Clean export to C4D, Blender, Unreal, etc.

• Include semantic scene data

Why This Matters

As a video creator, I spend a lot of time:

• Hunting for good track points

• Manually defining ground planes

• Fixing drift caused by moving objects

• Creating fake occlusion masks

• Guessing scale

An AI-enhanced tracker would turn this into:
Drop footage → Solve → Get a usable 3D scene.

It would make After Effects feel years ahead of other compositing tools and remove a lot of friction from 3D integration.

After Effects is already incredibly powerful. Updating the 3D Camera Tracker with Adobe Sensei would make it feel modern again and massively speed up real-world workflows.

I’d love to see Adobe explore this direction.

Thanks for considering it.