Char Anim's "Compute Lipsync Take From Audio and Transcript" can't detect w's

I got curious and recorded a bit of audio saying "Who, what, when, where, and why?" and indeed it only used a W viseme on the first one and that's actually not for the W, it's for the "Oo" at the end of the word. I tried a bunch of other W words and noticed the same thing.

So here's a fun peek under the hood. Inside the app resources the (plain text) file `lex-phonealign/model/dictionary` has the full list of words that the transcript based aligner recognizes directly along with their phonetic decompositions (multiple for words like tomato for toe-may-toe and toe-mah-toe). There's another bit of code that handles words not in the dictionary so that things like names and nonsense words (I used Jabberwocky to test it) can still work. You'll note that there are more different codes than there are visemes, so there's a lookup table in the Lua code that describes how those get converted to visemes.

Of all those phonetic codes, only 3 map to the W-Oo viseme and all the uses I see are for the Oo and not the W.

Also, you're correct that the W is treated as silence (the only letter that is, actually). I'd have to ask internally about how that lookup table from phoneme to viseme was decided. I suspect it is because the Oo viseme tends to be really dramatic and for a lot of words, it'd look weird to have a strong "Oo" for a brief W that immediately turns into some other sound. I haven't tried this, but you may be able to experiment and get what you want by tweaking the dictionary. I unfortunately don't have an easy way to tweak the phoneme to viseme map currently.

Hopefully that helps.

Dan Tull
Adobe Character Animator Team