Feature focus: Extract cameras and lights from 3D models

[TimKurkoski]

Howdy to everybody in AE Land! In today's build of After Effects Beta, v23.4 build 18, you can now extract cameras and lights from GLB and GLTF models in a comp.

To extract cameras and lights, select the 3D model layer in the comp and then choose:

• Layer > Camera > Create Cameras from 3D Model
• Layer > Light > Create Lights from 3D Model

After Effects Beta will create new camera and light layers based on the camera and light data in the 3D scene. If the cameras or lights are animated, After Effects Beta will create keyframes.

The goal with extracting cameras and lights is so that if you have spent the time to set up a scene in your 3D modeling app, you can quickly recreate the same scene in After Effects.

We want to hear your feedback on this, so please give a try and add a comment below with your reaction. Tell us if you encounter any problems or unexpected results, or if you just love it!

Some things to keep in mind:

• Support varies between 3D modeling apps for saving cameras and lights into GLB/GLTF files. You may need to change how the file is saved, and some features may not be supported. (Cinema 4D, for example, does not currently have an option to save lights into GLB/GLTF files.)
• Similarly, there are differences in how 3D scene data is represented between different 3D apps and the 3D file formats themselves. After Effects Beta does its best to translate the scene data available in the file into its own camera and light layers. Light power, for example, can be expressed in several different ways by a 3D app. If you see a case where something seems obviously wrong, let us know so we can investigate.
• Keyframes will be created for every frame of animation. This is the best extraction model of the animation data in GLB/GLTF.
• OBJ format does not support camera and light scene data.
• Extraction is a one-way process. If you update the scene in your 3D modeling app and re-save the file, After Effects Beta will not automatically update cameras or lights that have already been extracted. You will need to re-extract the updated cameras or lights.

Known Issues:

• Focal length of extracted cameras may be slightly wider than in Blender.
• An unneeded keyframe is added to the first frame of an animated property, if the animation does not start on the first frame of the layer.
• Some properties like focal length, focus distance, f-stop, are missing when extracting cameras, if those properties have animation.

[Taylor Jon Peters]

Amazing! Speaking of focal lengths. Would it ever be possible to natively support isometric cameras? 

[TimKurkoski]

> Would it ever be possible to natively support isometric cameras? 

Possible, yes. It's a request we've heard many times over the years and is something we've considered as part of the 3D work we're currently doing, but it's not high on the priority list compared to the current development of importing and rendering 3D models.

Please do suggest this as an idea so it can get voted on and we can see how popular it will be!

[ricardor99448640]

Why not put all this effort in some industry standard like USD or Alembic, instead of a format most of the people in the industry barely or even never use it, and not even C4D has full support?

[TimKurkoski]

USD is on our roadmap. Alembic is also on our radar, but we're not to the stage of being ready to build that out yet.

Please mind that 3D model import is a work-in-progress in After Effects. This is why it's only in After Effects Beta right now. Once we complete the first iteration and move it to public release, that doesn't mean we'll be complete with the work, our plans include future iterations. As always, please let us know what you'd like to see by posting to the Ideas forum.

We've started with support for OBJ and GLTF/GLB for a couple of different reasons. OBJ because it's one of the most common model formats out there.

GLTF/GLB is a widely supported standard with more capabilities than OBJ, but more importantly it is natively supported by the rendering engine that backs the Mercury 3D renderer. This engine is shared with other Adobe tools like Adobe Aero and Adobe Dimension. Using this shared technology and working with the team behind that engine has given After Effects a much faster start than building our own render engine or attempting to integrate a third-party engine.

We're looking forward to adding additional formats and capabilities to 3D in After Effects, but we have to make this walk before it can run.

[n1ckfg]

Yeah the USD-vs.-GLTF debate seems to be happening everywhere! I'm happy with GLTF as the inaugural format supported here--even though my personal first choice would be Alembic--because GLTF is a direct, open substitute for Autodesk's FBX, probably the 3D format with the most use cases and broadest support. It's also afaik one of the only formats with an option for built-in compression (Draco), which can be a huge help on lower-end hardware.

[n1ckfg]

Speaking of which, there doesn't seem to be Draco GLTF support yet. Adding the Draco libraries...should?...be easier than implementing a whole new format, and a lot of the GLTF stuff on the web that people might want to experiment with is Draco'd.

[TimKurkoski]

Interesting. This is probably a matter of the code libraries used by the engine that underlies Mercury 3D in After Effects. They probably need to be updated to this yet. I'll ask the team responsible for the engine if they have plans for this.

[TimKurkoski]

Quick update: the engine team confirmed my suspicion that the version of the underlying GLTF library we have integrated probably doesn't support Draco compression. It looks like there's a newer version of the library available which does support Draco, so it may only be a matter of integrating the updated library. I'll caution that this isn't always simple or quick; library updates need thorough testing and thus aren't "free". Whether Draco support comes to After Effects in the future will likely depend on the necesity of supporting Draco or any other features or bug fixes provided by the updated GLTF library.

(I've been the tester on similar types of library integrations and can attest to the frequency of breakage even when the updated libraries are assumed to be "safe".)

[colleens41028206]

Does this not support objects exported from Adobe Dimension? My create light from 3D object is greyed out for both .glb and .gltf formats 😞

[VictoriaNece]

I'm not sure if Dimension can export objects with embedded cameras and lights. It's no longer being actively developed, so we haven't tested those workflows.