Skip to main content
D
donjojohannes
Known Participant
January 9, 2017
Question

Languages and vismemes

  • January 9, 2017
  • 3 replies
  • 379 views

First off: Great work. I just come from binge watching all of Dave's tutorials and updates. I also spent a few hours playing with my first puppet.

Now, my question has been asked in a specific way (regarding Korean), but I wanted to ask in a general way and provide feedback for languages more closely related to English.

Here is what I found:
English - as expected - works very well. In fact while exploring CH I ran the tutorials on a second screen, so the mic picked up the sound and had the puppet (with its own mouth design) rattling away. It looked quite good. When I tried a pre-recorded German track the result was mixed. The TH shape came up a lot for some strange reason, though the sound does not exist in German. On the whole it felt less accurate. In Italian I fared a little better with a sample I recorded. Now, I know tweaking can be done by altering the source shapes, inserting specific vismemes manually etc.

My question is: Are there plans to work on the audio processing (auto lip synch) feature to make audio processing more versatile (by pre-selecting languages/sounds CH listens for, maybe?). Phonetics differ quite considerably in languages and - as it would appear from my tests also how the mouths are processed.

But again: Thanks to the Team for working on this great addition to the CS and Dave for the Tutorials.

    This topic is closed to new replies. Start a new post to keep the conversation going.

    3 replies

    oksamurai
    oksamurai
    Legend
    January 9, 2017

    Glad the tutorials are helpful!

    One thing we are always grateful for are specific feedback examples if something doesn't work well in your language. Audio files are the absolute best for this. So if that Th shape keeps showing up when speaking German, if you could share that audio for us and say what the issue(s) are, then that's something we can try to fix head on. If you post it here or DM me I can share it with the team.

    I don't know all the details under the hood, but I do know each version adds more and more data (including languages), so hopefully with every release this will improve. Ideally, yes - we would love to have native support for more languages, and/or more customization options for the lip sync, and you list some great ideas there. I sincerely hope we get there sooner rather than later, and I think we will given how critical lip sync is to a great animated piece.

    D
    donjojohannesAuthor
    Known Participant
    January 10, 2017

    Thanks Dave,

    I gave German another shot using an audio book. Deep clear voice. Results seemed considerably better than with my own speech (deep mumbling through the thicket of my beard - a reason why I cannot really use the live tacking feature, but I'm fine with that). So, it could be me (too fast?) - or maybe it has to do with mic quality - a logitech mic used for skype and things. On the other hand Italian worked better... Anyhow, I will set up my condenser mic and try again. I'll let you know how I fare and provide samples if I cannot get rid of the Th mouth appearances.

    oksamurai
    oksamurai
    Legend
    January 10, 2017

    Cool thanks. Yeah, the louder and clearer the better with lip sync. We do run a hard limiter behind the scenes to try to boost your signal and get the best results even in low conditions, but in general try to get that green audio bar as full as possible without clipping into the red.

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices