Help with creating colorful visuals for music video editing?

[sirebesq]

Hello all. I'm a musician with something called synesthesia, which is basically when 2 senses cross in your brain. The most noted form of this is when people "see" colors with numbers and letters (A is red, B is brown, C is yellow, etc). For me I "see" colors while listening to music, as in each instrument is a different color and texture moving around to the beat of the song. Kind of like a music visualizer on audio softwares. For years I've wanted to create videos to emulate exactly what I experience with certain songs but I've never been able to figure out how to do this. I have Adobe Premiere and After Effects. What add-ons and plugins would I require for something like this? Are there phone apps that could be used as well? Anybody have any advice on how to do this?? I've been attempting to do this forever but nobody has been able to point me in the right direction. Literally any help would be much appreciated.

[Rick Gerard]

You can use the Keyframe Assistant to turn audio levels into keyframes. That works with levels only. 

You can use Trapcode Sound Keys to do basically the same thing but you have the option of specifying frequency ranges, not just levels.

There is no plug-in that will isolate instruments. You either need to get a recording with different instruments on different tracks or you need to take your audio into a program like Audition and try and isolate instruments sounds so you can import different audio tracks for each sound you want to turn into colors.

The output of both Audio to Keyframes and Sound Keys is a string of keyframes. You then take that data and make it drive any property you want. There are expressions that use the interpolation methods to change one set of values to another so you can drive any property you can animate with numbers. Color is a 4 digit array from 0 to 1. If you wanted to animate a red color fill from a red value of .5 to 1 and the other colors were zero, and your audio keyframes went from 4 to 19 you would point to set up your expression like this:

t = thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider");
red = linear(t, 9, 19, .5, 1);
[red, 0, 0, 1]

 That would give you a red color changed from 100% red to dark red as the audio changed.

Any property could be animated. You could apply audio information to glows, layer fils, Fractal Noise, almost any color correction effect, and experiment with shapes, distortions, and colors all you wanted. The only real problem you will have is separating the different instruments to a different audio track. Trapcode Sound Keys will hep with the frequencies, but for really fine adjustments you'll need to do some custom EQ in audition.

[Mathias Moehl]

If your objects move with the beats, our extension BeatEdit will be very helpful. There is both a version for Premiere Pro and After Effects:

https://aescripts.com/beatedit-bundle/

Here is an example of how you can animate layers in After Effects with the beat:

[jampff]

You may love magicmusicvisuals.com (I do).

After Effects can create keyframes to animate anything in the comp... but you have to put it there for AE to animate it. MMV actually creates the visuals, based on nodes/parameters you define, and then animates them to your music (various properties of the music, not just volume). Incredibly powerful. No, I don't work for them, nor am I sponsored in any way. I just think it's amazing software. Probably perfect for someone like you. Check out the site and the demos for far more details than I can provide.