Skip to main content
M
maaaxataftph
Participant
March 9, 2020
Answered

How to Make Audio Amplitude Match Timeline of Video Clip?

  • March 9, 2020
  • 2 replies
  • 5936 views

Hey everyone,

 

I am asking for help about how to achieve a certain effect for a school project, and my professor does not know how to make it happen. She showed the class how to use the Keyframe Assistant tool on a music layer to generate an Audio Amplitude "null" layer, which has a "Both Channels" Slider Effect that matches the amplitude, or volume(?) of a song automatically. Then, I can take the Scale pick-whip of a text layer or other Vector object that I create in After Effects, and connect it to the Both Channels Slider on the Audio Amplitude layer. After some adjustments of the peaks and valleys of the graph of the audio amplitude, one can create the effect of having an object or text layer scale up and down, in sync with the tone or beat of the song. So, if the song gets very loud for a second, it would cause the text layer to increase in size, scale up, and then when the song gets quiet again, it would cause the text layer to decrease in size again, back to its original size, and then proceed to scale up and down for the rest of the song.

 

My question is a bit different, and I think that I might need to write a custome script for my project, and I don't know if I will have time to learn how to do that before the project is due. Here is my question: in the same way that you can have the scale (or opacity or rotation) of a Vector object match the amplitude that is automatically generated with the Keyframe Assistant for a song, is there a way that I can match the timeline of a video clip to the amplitude of a song? I will attach a few pictures of the video clip I am talking about. Basically, my video clip is 2 seconds long, and it is a clip of a few dozen colored blocks and rectangles that extrude towards the audience. I created the animation in Maya (3D animation software). The first picture I will attach is the beginning of the video clip, the second one is somewhere in the middle of the clip, and the last picture is the end of the clip. What I want is to have the final video to appear as if the height of the rectangular cubes match the amplitude of the song, as if they were a 2D object layer whose Scale I would match (using a pick-whip) to the amplitude keyframes of a song. Of course, the trouble is that I want to match a video clip to the song, not a 2D object's scale, and I don't know how to modulate, or create the appearance that the height of the colored, rectangular cubes, match the amplitude of the song.

 

Am I right in thinking that I need to create a script to do this? Or is there a built-in feature to After Effects that I'm just not aware of? Let's say that the song's amplitude has 1 keyframe at 10 decibels, and then a second keyframe at 50 decibels. The range of the height of the keyframes in the song go from 0 decibels to 100 decibels. So, when the song plays, and hits 10 decibels (10% amplitude), I want the computer to match up the location on the video clip's timeline to 10% of its length. On a 2-second video clip, this would place the timeline slider at .2 seconds. Then, when the song reaches the next keyframe, at 50 decibels (50% volume), I want the computer to seek in the video clip to halfway through the clip, which would be 1 second in. This would make it appear that, when the song is quiet for a second, the rectangular blocks are short, at 10% of their appeared height, and then when the song gets louder to 50 decibels, they appear at half their height. And, of course, I would want the computer to include all the keyframes in-between .2 seconds and 1 second, so that the blocks appear to extrude toward the viewer, from 10% height to 50% height, and then proceed to stay in sync with the rest of the song.

 

I know my question seems complicated, but I would really appreciate it if anyone could tell me whether there is a built-in feature that I can use to do this, or whether I need to spend some time and learn scripting. I just want the height of the rectangle blocks in the video appear to modulate and change, in sync with the volume of the song, similar to the way many music videos on YouTube have a 2D logo or picture change in size, opacity, or rotation, in sync with the music. If you do not understand my question, I will provide more clarification.


Thank you,

 

Saxazax

    This topic has been closed for replies.
    Correct answer Rick Gerard

    If I start with your balloon example, a 2 second shot of a balloon blowing up all you have to do is apply time remapping to the shot and add a linear expression. I recreated that scenario using a pre-composed animated shape layer to be the balloon. Here's the project file: 

    https://www.dropbox.com/s/09pbio8brzu5fei/balloon.aep?dl=0

    All you have to do is replace the audio and add your animated movie where the balloon pre-comp is. 

    This is the expression:

    t = thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider");
ease(t, 10, 30, 0, 2)

    10 is the Both Channels Slider level where I want the balloon to start to inflate, 30 is the value where the balloon is fully inflated, 0 is the start time of the movie in seconds, 2 is the last second of the movie where the balloon is visible and fully inflated.

    2 replies

    Warren Heaton
    Community Expert
    Warren HeatonCommunity Expert
    Community Expert
    March 11, 2020

    Red Giant Trapcode Sound Keys for After Effects

    https://www.redgiant.com/products/trapcode-sound-keys/?gclid=EAIaIQobChMIzM3f9puT6AIVjbbICh2IUgDjEAAYASAAEgI1c_D_BwE

     

    They have a trial versoin that you can download and test.  If you decide to purchase it, there's a 50% academic discount.

     

     

    -Warren

    M
    maaaxataftphAuthor
    Participant
    March 15, 2020

    Thanks for the reply Warren, but I was able to figure it out following the advice Rick gave me.

    R
    Community Expert
    Rick GerardCommunity Expert
    Community Expert
    March 10, 2020

    Your graphic shows a bunch of blocks that have a different extrude value so you will need an expression for each of the extrusion values in your project. If you create those rectangles as shape layers you're going to need about 100 3D shape layers. You can cut that number down by putt all of the layers that you want to have the same height on the same layer. Some of your rectangles have blue faces facing the camera and different colored sides. You can do that with a single-pixel stroke. If you are careful you might be able to get the number of shape layers down to 25 or 30.  Every one of those shape layers is going to have an extrude value. That's where you put the expression. 

     

    Let's say that your Audio Amplitude>Both Channels slider returned values between 0 and 30 and you wanted the extrusion to start when the audio level hit 10 and you want all of the rectangles on Shape Layer 1 to modulate between 25 and 200 pixels all you have to do is add the following expression to the Extrude:

     

    t = thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider");
tMin = 10;
tMax = 20;
ease(t, tMin, tMax, 25, 200)

     

    That's all there is to it. Extrusion only happens in positive Z space so you will have to flip your layers 180º in X or Y to move the faces toward the camera. I only see one problem with the project. The C4D rendering engine is a huge resource hog and you are going to need an extremely powerful system to handle the rendering and the preview if you want to have 20 or 30 renders. My iMac Pro - the very top of the line iMac Pro with max memory and the fastest processor, will barely handle 5 shape layers. 

     

    I hope this helps. The project would look like this:

    The other problem you are going to have is that the C4D rendering engine does not support motion blur so you are going to have to put your final comp or a rendered production master of your project into a standard comp and add Pixel Motion Blur or CC Force Motion Blur to clean up the final animation and make it look good. Here is a sample project: 

    https://www.dropbox.com/s/gl7mvlpqz7e4qpt/audio%20to%20extrude%20auto-save%201.aep?dl=0

     

      M
      maaaxataftphAuthor
      Participant
      March 11, 2020

      Dear Rick,

       

      I really appreciate you taking the time to answer my question. However, the answer you have given me was not really what I'm looking for. You recommend that I handle everything totally in After Effects: creating the block shapes, extruding them, and matching their Z height with the amplitude of the song. However, what I was originally asking was if there is a way to get this project done by writing a script for After Effects to match the amplitude of a song to the timeline of a video clip. I have already made the extrusion animation video clip in Maya, and I imported that video clip into my After Effects composition. Let's say, for the sake of argument, that I had a video clip, which was 2 seconds long, and it was of a balloon being inflated. The clip begins at 00:00, with balloon at a small size, and the clip ends at 02:00, with the baloon being totally inflated. This would obviously look silly, but how could I write a script that would match the 2 second clip of the balloon being inflated to the amplitude of the song? In other words, if the song got loud, the computer would play the video clip toward the end, and the balloon would be big. Then, the song gets quiet, and the computer plays the video clip backwards, quickly, and the balloon shrinks down in size, to match the quietness of the song. So, the computer would be playing the video clip backwards and forwards, calculating and matching the amplitude of the song, and the balloon would inflate and deflate in sync.

       

      I use this example to hopefully explain my question a little better. I'm not asking about how to recreate the blocks in After Effects, and then tie their Z extrusions to the amplitude of the song. I understand that that is a way to approach the problem. However, one problem that you pointed out, is that the graphical requirements of my computer will have to be extremely high in order to achieve the effect I am looking for, and neither me nor you have a computer that can handle 100+ rectangular blocks. So, to apply the example I gave here to my issue, again, I have already created the 2-second video clip of colored blocks extruding. I made the blocks and animated the extrusion effect in Maya, put that animation through the Media Encoder pipeline, and brought that clip into After Effects. What I'm asking is how I might write a script that would have After Effects go backwards and forwards through the clip, very quickly, as the song changes volume quickly, like many songs, and create the effect of the rectangular blocks that I created in Maya, extruding and de-extruding, in sync with the song. I do not want to re-create the blocks in After Effects.

       

      I apologize if I was not clear the first time I wrote. What I wish I could do is pick-whip my video clip to the Both Channels part of the Audio Amplitude layer, and have the computer seek backwards and forwards through the video, matching the location of the video clip timeline (100% would be 2 seconds, 50% would be 1 second; 100% amplitude would be 100 decibels, 50% amplitude would be 50 decibels) to the amplitude of the song. Sorry, I know I'm repeating myself a lot here. I'm wondering whether I could write a script that would treat the location in the video clip timeline, in the same way that it would treat the Scale of a 2D vector object, or its opacity or rotation, and then pick-whip that to the Audio Amplitude - Both Channels layer, causing the 2D object to modulate in size, in sync with the song.

       

      Here is a video tutorial that I found of someone doing something similar to what I want, however, he is using a $40 plugin, which I would rather not purchase for this school project. https://www.youtube.com/watch?v=yN-vYEnnR8M . In that video, he has the timeline of the video clip playing forwards and backwards, in sync with song. I hope I have made my question more understandable. Thank you again, Sax

      R
      Community Expert
      Rick GerardCommunity ExpertCorrect answer
      Community Expert
      March 12, 2020

      If I start with your balloon example, a 2 second shot of a balloon blowing up all you have to do is apply time remapping to the shot and add a linear expression. I recreated that scenario using a pre-composed animated shape layer to be the balloon. Here's the project file: 

      https://www.dropbox.com/s/09pbio8brzu5fei/balloon.aep?dl=0

      All you have to do is replace the audio and add your animated movie where the balloon pre-comp is. 

      This is the expression:

      t = thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider");
ease(t, 10, 30, 0, 2)

      10 is the Both Channels Slider level where I want the balloon to start to inflate, 30 is the value where the balloon is fully inflated, 0 is the start time of the movie in seconds, 2 is the last second of the movie where the balloon is visible and fully inflated.

      Terms & ConditionsAccessibility statement
      Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices