Skip to main content
larsa85501086
Participating Frequently
November 6, 2022
質問

How do I switch images to the beat of a music piece?

  • November 6, 2022
  • 返信数 3.
  • 1318 ビュー

How do I switch images to the beat of a music piece? Please take a look at the video before reading on. I'm trying to animate this choir of famous people, into singing the ode 'To joy'. The last part of Beethovens 9th. symphony. I've inserted a drawing of a mouth from a lipsync sheet, over the face of the guy in the middle of the video. And then swithed it out with another as the music changes. I need to set it up so it follows the music. To make it look like he's singing the song. The style of animation is deliberately primitive. It's supposed to look that way. The video was too big so I made a screenshot recording of it and uploaded that instead. This made the delay between the mouth movements and the music even worse. But it's just there to show you what this is about.
I've tried using several lipshapes, but come to the conclusion that I only need to use 2 'open mouth' shapes, 1 'closed' lips and one 'smile'
I've tried watching some lipsync videos on YouTube, but that is not what I'm trying to do with the choir. Using an 'open mouth' and a 'closed mouth' looks better than trying to use several different mouthes to fit with different words in the lyrics.
From watching YouTube videos, I've managed to open up the Waveform of the music file. I've then inserted the 'open mouth' at the highpoints and the 'closed mouth' at the low points of the graph. Didn't work out very well.
YouTube also taught me to create an Audio Amplitude layer from the audio file. I opened it up, selcted the keyframes in the slider and hit 'Graph editor'. This gave me an even better visual on where to put the 'open mouth' image and the 'closed mouth' image. But not a better result. From what I understand the 'Graph editor' in the 'Audio Amplitude' layer, shows the high and low point of the volume. And I'm not sure this is where I need to switch between the mouths, in order to make it look like they are singing the song.
The problem I have with YouTube is that the videos I find, shows me how to animate an image to the music. Change its size for instance. But I don't need to animate the mouths. I just need to switch between mouths to the beat of the music.
Thirdly I've been looking at a metronome plugin for After Effects. It costs 25£ though and I'm not sure this is the best way to do this.
As you can probably tell I'm not well versed in After effects or editing/animation at all. I hoped this project would be a simple matter of connecting the mouth images to the music file with the Track Matte pick whip. But that's not possible. I'm sure there's a pretty simple solution to this though.
The video: I've tried doing it "manually" on the face in the middle of the choir. Simply by inserting the mouths and switching between them based on my own judgement. The plan is to have all the faces "sing" like that. Maybe using a few different set of mouths. And then copy them and fit them to the others.


So how do I switch between the different mouth images in the beat to the music?

 

I know this is as low-level as it gets. I hope you will help me anyway.

 

every help is appreciated. Thanks in advance.

このトピックへの返信は締め切られました。

返信数 3

Mylenium
Legend
November 7, 2022

Please refer to your other post. I'm locking this one to avoid confusion.

 

Mylenium

Mylenium
Legend
November 6, 2022

It's simply meticulous manual work like character animators have been doing for decades. Any automated technique like e.g. using audio threshold expressions (see here: http://www.motionscript.com/design-guide/audio-trigger.html ) simply won't work because classical music doesn't produce distinct peaks liek Hip-Hop or other musical styles would. You simply have to sit there with a cup of coffe and animate the on/ off states with a bazillion keyframes.

 

Mylenium

Participating Frequently
November 7, 2022

if I could make a composition with only the music file and then make it so that when played it, it would be the timeline itself that moved and not the red 'current time indicator'. Then I could place the mouse-arrow on the timeline, start the music file, close my eyes and hit left-mouse-button to the beat of the music. Every click would make a keynote on the music file layer and then I could place the open mouth image after those keynotes. Do you understand what I men? Is this possible?

Mylenium
Legend
November 6, 2022

The simple answer is phonemes, meaning how mouth shapes look when producing sounds. A basic principle of any character animation on which you can find tons of information out there. They have nothing to do with sound volume, beat of the music or frequencies, they are just how you literally from sounds by controlling your facila musculature. As for how realistic you want it, is entirely up to you, but only using two shapes is always going to create a basic jibber-jabber animation which bears no relation to the sound. You could animate it with a wiggle() expression and it would look just as terrible. Creating a bunch of different mouths could easily be done by doing them in pre-compositions and then duplicating them over your characters.

 

Mylenium

Participating Frequently
November 6, 2022

Thanks for replying but that's not what I'm asking. I need to know how to switch between "jibber(closed lips)" and "jabber(Open mouth)" so that it fits with the singing. Open mouth on every syllable, followed by closed lips. 

 

Freu-de, schö-ner Göt-ter-fun-ken,

Toch-ter aus Eee-lyyyy-si-um.

 

Jab-jab, jab-jab Jab-jab-jab-jab,

Jab-jab jab Jaaab-jaaaab-jab-jab.

 

but in tune to the music. So that the mouths open on the exact moment a syllable is being sung. That's what I'm trying to find out. How do I do that?