Layer picker or Cycle layers via Audio Input

[cuckets]

I've been trying to hunt for the video explaining this, which I followed step by step but could not get to function.

Essentially, it was to have Loudness(db) via audio input either trigger a cycle layers or layer picker, the example I'm remembering had to do with arm movement, where someone was raising their arm based on how loud their voice was.

What I would like to eventually set up, is laughing/yelling mouth layers to activate upon loud audio input.

Any help appreciated, thanks!

[alank99101739]

I did not know that was possible! Adobe Character Animator Help | Control puppets using behaviors lists two “Audio input” behaviors - Layer Picker and Jawbone (mouth movements). If you find the video again, please share here!

Reading the manual it seems like you create a layer with children layers then add the “Layers” behavior to the parent. Then you set the audio sensitivity to control how to map volume to child layer to trigger. Child layers can be cycle layers (e.g. as noise gets louder, bird flaps its wings harder or something). I would try having

- Mouth Group

   + Mouth (standand mouth with all the viseme children)

   - Big Mouth

   - Very Big Mouth

So normal volume picks the first child (“Mouth”), and louder volumes stopped using the normal mouth and picked the Big or Very Big mouths. I guess each could be a complete set of mouths perhaps, with its own Mouth behavior added if necessary.

Alternatively, you could have the Mouth group with each viseme child layer being a group with Layers behavior added and different artwork underneath so you can have different sized mouths for individual visemes....

- Mouth

   - Neutral (no child layers as no sound)

   - Uh (with Layer Picker behavior)

      - Uh quiet

      - Uh medium

      - Uh loud

   - Ah (with Layer Picker behavior)

      - Ah quiet

      - Ah yell

   - Smlie (would not use a LAyer Picker since smiles are triggered by webcam, not sound)

Etc.  But I have not tried it.

[cuckets]

I will give that a try later tonight, thanks!!

[DanTull]

Clever idea. Cycle layers doesn't respond to audio amplitude, but layer picker does. If the early frames of the layer picker were the same as the default but there was a group with cycle layers further into the group to which layer picker was applied, it should trigger the cycle only with sufficent audio volume.

I'll see if I can get an example working and post it. I vaguely recall the last time I used layer picker it didn't work right away for non-obvious reasons.

DT

[DanTull]

I can get the layer picker to trigger a cycle layers animation, but it doesn't seem to honor the option to let the cycle finish when the audio volume drops again. Depending on what you're trying to do, that limitation may or may not be problematic. I'm going to ask internally to confirm if that's a bug.

One other option given that limitation would be to have the later layers in the layer picker correspond to layers that make a nice forward/reverse animation. Then for low volumes it'd do nothing, but oscillations in the higher volume range would trigger some movement/laughing animation/etc.

Attached for reference in case it is useful.

DT

[DanTull]

Another variation on this is to tag arms with a Jaw tag and use the rotation option of nutcracker jaw. If it gets too annoying, you could also put the tagged variant in a separate group and use a swap set to toggle the audio volume based arms on/off.

DT

[alank99101739]

I was hesitating to say something, but I must admit a little fear about using audio level effectiveness. I thought a common audio mixing technique was to use compressors and level normalization to avoid too much variation in dB levels. I just worry for professionally mixed sound whether dB level is going to be reliable.

But it looks like Layer Picker cannot be hooked up to a MIDI slider? That would have been nice. That would have allowed a foot pedal (e.g. volume or damper pedal) to move between layers. Looks like need to use software to convert the MIDI pedal levels to individual MIDI note trigger events for the different layers.

[DanTull]

I don't have a midi slider handy, but I think you can hook layer picker up to a slider by binding it to the index or percentage offset parameters.

DT

[alank99101739]

Okay... Doh! I just realized my mistake. I forgot to drag the percentage value over to the control panel! For some reason i was thinking it would just appear in the control panel like triggers. All good! So I should be able to hook the vive controllers that way as well. In particular, use the trigger distance pulled to feed into a layer picker for different hand positions. I was hard coding it as a single trigger before, which felt like a waste.

[alank99101739]

I dropped a Layer Picker on a Body of a puppet, then dragged the Percentage control to the Control Panel. Dragging the percentage control cycled through the arms and body. Have you tried adjusting the percentage value? Or just the audio? Trying the audio quietly just now (kids gone to bed!) I did manage to get it to change layers by turning up the audio sensitivity (after arming the audio input control)

[alank99101739]

Okay, I am going to try a quick livestream demo on https://www.youtube.com/channel/UCGZCBwxBMCm79lxvkE8OvvA/live in a few minutes if you want to join me there, otherwise I will post a link to the little video I create here afterwards. (I have not tried a livestream with someone actually watching... which is rather sad when you think about it... lol!)

[alank99101739]

And of course my audio did not work and I had to reboot to get it working again... but I got there in the end (thanks for being patient). The little video link is here: Layer Picker - YouTube  - But Dave's video (the link you supplied) had it all. I really just showed it worked on my computer too.

[cuckets]

Thanks!

I realized what my issue may have been thinking about this earlier, and it was that I had kept my Laughing Group hidden, as I only wanted the layers to appear when i was laughing or yelling into the mic.

After seeing your video, it helped me realize that I needed to have 1. a "mask" under each laugh mouth, 2. to have the laugh folder be ABOVE the mouth group(to hide them), and 3. adjust so the sensitivity isn't constantly hiding my regular speaking mouth.

like so:

here's a gif after a few aesthetic changes:

Screen capture - 054627df627d9db05d38740676a3a17f - Gyazo

the only thing I cannot seem to figure out now is how to get it to go through the other layers, as for right now, it only goes through layer 1(empty layer) and layer 2(mid mouth laugh) and not to the other "bigger" laughs

[cuckets]

ah, I'd like to see what you put together, but it doesn't seem my app is up to date, with no updates in sight either

[alank99101739]

I wonder if Dan has an internal build of 2.1 (build 22 of the next release?) not 2.0 like us mere mortals.

[DanTull]

Oops. Yeah, I wasn't thinking and created it with an internal development build. Sounds like you got it sorted out, so I won't worry about it now.

DT

[cuckets]

okay, I've found it after hunting through all of my search history in the past month haha

so here in this video(timestamped 21:30)

Character Animator New Features (Part 2: Physics & Everything Else) - YouTube

I tested this myself with a group of layers, but got no response out of it, not sure what I did wrong