AE and 3D Sound with Tracker?

[AntDX316]

AE and 3D Sound with Tracker?  Is this possible to do, I haven't looked into it.  If you attach it to a Null object, can the sound be in 3D space?

[Mrtn Ritter]

You cannot place a sound source in space in AE, unlike Unity. But you can use a Null to drive balance and volume using expressions.

*Martin

[ShiveringCactus]

@Mrtn Ritter is right, but... you could use the length expression to control the volume of the sound source by the proximty to the camera.  Something like:

VolumeControl = 10;
vol = value[0]-length(thisComp.layer("Camera 1").transform.position, thisComp.layer("Null 1").transform.position)/VolumeControl;
[vol,vol]

This takes the value of the audio file, and minuses the distance between the camera and null object.  The volume control value divides this length to drop the numbers down which flattens the waveform.

[Mathias Moehl]

That's interesting, but only a mono solution right? For stereo you would also need to take the direction and not just the distance into account and factor this in somehow.

[ShiveringCactus]

Yeah, I think you're right.  This was "off the cuff" really.  but there's also the LookAt expression, which would give you the angle of the null to camera.  I'm sure with a bit of thinking it would be possible to control the left and right levels instead of setting both levels to the same.

[Mathias Moehl]

Yes. To keep it simple, you could also use two nulls a bit left and right of the camera instead of the camera itself and calculate the distance to the left null for the left channel and the distance to the right null for the right channel. But this would be a very rough approximation, so calculating the angle would definitely the better approach.

[Mylenium]

Sound is not processed linearly, so aside from the geometric layout the expression would have to account for volume based on a logarithmic scale plus you need to figure in doppler effects, indirect echoes and a number of other things. None of this exists in AE and you'd have to re-create it yourself more or less. Otherwise, most of such stuff is still mixed traditionally for movies and even games do not necessarily produce environmental sounds, but rather just use audio triggers to mix conventional tracks. Even going through the trouble of setting up such a complex system in AE may not get anywhere near producing production-ready, "realistic" sound. I'm pretty sure you'd be rather disappointed how dull and boring the result might turn out.

Mylenium

[Rick Gerard]

The best way to generate 3D sound effects is to render the comp, add it to Premiere Pro, then edit the sequence in Adobe Audition. You have the option in Audition to accurately position sound in several different scenarios. There is even a workspace dedicated to that kind of stereo mixing.

If you just want to control balance and volume based on 3D layer position and camera position it's going to take some vector math and some interpolation methods. I have not tested the sample expression that ShiveringCactus posted but it looks close to volume. For balance left and right you would calculate the angle from the camera and the distance. I don't have time to try that right now, and I'm not good enough to do write that kind of code without having AE open but I'll bet Dan Ebberts could do it.

[AntDX316]

They should just make it where this is possible because it has the 3D/4D perspectives already understood.  This has been out since before the 2000s with Aureal 3D and Sound Blaster.  Being able to have a proper 3D/4D effect in sound can make the watcher have a better understanding of what is trying to be understood.  Right now everything is usually just mono or stereo.  Just because the top youtubers and other organizations of influence "don't do it" doesn't mean it shouldn't exist again.

https://www.youtube.com/watch?v=02ycCtTYDjg

https://www.youtube.com/watch?v=vX2xICHnivw

https://www.youtube.com/watch?v=o9n9fE_DeTE

[AntDX316]

Do any plug-ins exist to make this happen easily like Unity does natively but obviously that is not AE and if it were to work you would have to compress more than once.  I assume you could probably do it through Blender 3.0 but I haven't looked into it.  Can someone try these solutions and let us know?

[AntDX316]

Nothing having an automated system for sound position is as bad as trying to mask every single pixel manually instead of rotoscoping, camera track, track motion, or any other easy automated frame by frame pixel selection.

[AntDX316]

It should have all kinds of adjustable parameters too like you can with Adobe Audition effects. (wide, narrow).