Modern 3D Realtime Engine (GPU) for VFX and Motion Graphics

[pat pro]

After Effects finally needs a competitive 3D Realtime Engine (GPU) for 3D compositing and virtual workflows of the future. The Link to Cinema is too slow, complicated and unstable for simple 3D compositing tasks. So it needs something like "3D Elements" on steroids and native in AE.

Dranging some 3D Assets into AE, add some effects, tweak and light them, is a must have in 2022. Compositing, Motion Graphics and VFX are not only 2D/2,5D! - they happen now also in 3D and Realtime!

What this new real time 3D Engine needs:
- 3D Geometry Support (import/export)
- PBR Shaders
- Link to Substance Painter/Designer
- Primitives' and very basics modeling (box, spline) and deformer Tools
- Raytrace Lighting, Reflections (GPU)
- Support of 3D Volumes and Point Clouds (vdb.)
- Camera Mapping Tools
- Advanced Camera Tracker (more manuell control, constrains, import/export)
- Smart 3D Particles with Physics (node editor)
- VR HMD Support

thx

PS: I wirte this since years. please don't delete my requests again.

[autokroma]

Why do you need After Effects exactly ? Couldn't you do everything in your favorite 3D engine editor such as Unreal Engine ? Please contact us to tell us about it ! https://www.autokroma.com/contact

[nubnubbud]

@Bruno Quintin more like 15 seconds per frame! XD

[Patrick25495387jqda]

@autokroma: To mix 3D with 2D or Video Footage.
It makes absolutely no sense to switch for every little 3D-task, like 3D camera mapping, insert a existing 3D object or vdb. particle volume, every time to Maya/Max/Houdini. If I like to animate or simulate a photorealistic, burning dragon with dynamic feathers who is jumping in a river this extra mile makes totally sense!!!, but not for simple 3D compositing tasks. ...further, this Link-Tools from AE to 3D packages (except to C4D) do not exist, or are very slow and not stable. I think, the Maya-Link isn't working over two years now!!! And there are no simple and streamlined workflows to bring your 3D tracking data fast to Unreal, Max, Houdini and the 3D multi pass renderings without troubles back. It's too complicated/slow that nearly no one works productive that way.

[Patrick25495387jqda]

@Bruno Quintin: Which 3D Engine in AE??? You mean that 25-year-old 2,5D thing? 😄 😄 😄

[Bruno Quintin]

@PatRick Proier
nick@atozmedia.com Dorn

😄
https://youtube.com/clip/UgkxsilWrA2HrqNaVdAWL6TWV1EreMV6cLuY

[Bruno Quintin]

There are 3D engines that render photorealistic images at 60 frames per second and there is After Effects that does 15 frames per second to move a solid in the viewer. And that drops to 10 frames per second in the new Properties Panel ¯\_(ツ)_/¯

[autokroma]

Patrick I agree with you but as the same time we could try to stay in our 3D software all the time and never use After Effects -- that is of course only if all features are available the the 3D software

[pat pro]

@ autocroma: At the moment i don't know any 3D package that can handle VFX Compositing tasks like Keying, Roto, Retouche, Tracking, Stabilizing, In painting, DeNoise, Color Grading and and and ...like Composting Tools like AE or Nuke.
Blender is in my opinion the closest of all 3D Packages but still far away.

Your Strategie makes only sense if you work mainly in 3D and add only a video BG or some 2D Plates into your 3D Scene.

After Effects should go the way of Blender but only the other way around: Focus on Film/Video Compositing/Motion Graphics/Animation and add some Importen real time 3D functions and native import/export options of modern 3D/VFX standards like fbx, alembic, usd, vdb, bvh...

[Matt Tillman]

The fact that AE still can't natively import FBX camera data is astounding to me.


Regarding a 3d engine, I completely agree. The ability to pull in 3d models for simple tasks like plate reprojection or light projection templates is incredibly useful. Both Fusion and Nuke have supported this for years.

No one is saying AE should be able to handle 3d in a way that rivals an actual dedicated 3d package, however there are so many compositing tasks that can be trivialised by the ability to import some cameras and meshes and do a simple openGL render.

[TheFosterHouse]

something like Element 3D built into AE would be amazing. Just for its super fast rendering of a 3D working space alone would be enough. Element 3D is VERY fast especially compared to the sluggish AE 3D working space. If the team at Video Copilot figured out how to make it work and work that fast, maybe the devs should have made some calls by now. Its original release was July 10th of 2012. Speed is important.

an improved camera tracking workflow is essential
https://adobe-video.uservoice.com/forums/911311-after-effects/suggestions/44600286-3d-camera-tracker...

Camera mapping tools: absolutely.

Anything we can do to not have to wait for another 3D render, but we need to be able to work faster than it would take to render out a new pass. I've had issues with EXRs with imbedded Cryptomattes literally taking 20-30 seconds to load PER FRAME. no other effects, just the file with cryptomatte selections. I've done side by side comparisons with AE and Nuke and Nuke can select all the parts of a crtyptomatte needed and render out a matte faster than AE could just select 2-3 crypto matte areas. That's the speed issue I've been talking about.

In the end, AE is 2 things: a compositing tool and a motion graphics tool. It's not a CG rendering tool. we don't have Redshift or Vray built in, nor do I want to deal with that.

[Szalam]

For what it's worth, there is a GPU-accelereated, real-time 3d renderer in AE - and it's been there for a few versions! If you drop into draft 3d mode now, you're using it!

That engine has been extended to the latest version of the public beta which released yesterday and now allows you to import 3d objects into AE natively.

[pat pro]

Yes, I played already around with it. Good to see 3D native in AE!!!
... but now comes the hard part! Make it usable and ready for VFX!!!

- Fbx, alambic, usd. vdb. Support (import/export)
- Support of animated Meshes
- Simple/basic rigging tool (like in Mixamo to use Mocap data)
- Shader Editor (PBR)
 - Link to Substance Painter/Designer
- Raytrace Lighting, Shadows, Reflections (GPU)
- IBL (HDRI)
- New Lights (raytracing support) and more parameters like texture projection.
- New Camera with simple camera mapping support
- Camera mapping tools
- Lens distortion tools
- Focal estimation tools
- Primitives' and very basics modeling and Deformer-Tools (to models simple meshes for Projections)
- Shadow Matte
- Support of 3D Volumes and Point Clouds (vdb.)
- Smart 3D Particles with Physics (node editor)
- Advanced Camera Tracker (more manual control, constrains, import/export)
....
These are only the basic things 😄

[nubnubbud]

honestly, that's a lot of the issue many people have with AE, and most adobe products. Things aren't cleanly integrated, and there's no real style guide for how any infividual UI should work. I still remember the time the preview glitched and I saw all the hidden windows shoved beneath the preview display when it flashed.
the issue is, yes, it seems to exist. did I know itt existed? not really, do I know now? sorta? is there any part of the UI that makes it clear that 3D is even a part of the program? dubious at best, given a 32x32 pixel icon that can dissapear in some modes is how a new user first accesses it.

does it have industry standard... anything? no. no it doesn't. it's a little standalone novelty trinket.

AE needs to be remade from the ground up. You can keep the current versions! it can remain for a good while, but something modern, able to use ALL my CPU and ALL my GPU but NOT ALL my RAM, so it can accelerate things using the extra. you know, like everyone else gets. 

I currently only use AE to convert to prores, and do masks. AE's layer system does masking really well, does tracking really well, does motion graphics really well, does blending really well, and IS INCREDIBLE at matte painting. Those should be the focus. screw 3D. we have a million programs, including free ones for that. AE's 3D tracking is freaking useless because it can't interface with any standard, or be sent to any other program.

[PFuerst]

I think Adobe should look at the Foundry and see what they are doing with adding USD. To me it looks like a format that combines the best from FBX and Alembic with options to overwrite parts if necessary.