Talking Character Animation: Lip Sync Tools and Methods Compared
By the Charmotion team · Updated · How we write guides

To animate a talking character, draw a set of mouth shapes and let software pick one for each sound in your audio. Character Animator and Cartoon Animator compute lip sync from a recording, free Rhubarb Lip Sync exports mouth timings, and AI tools such as Hedra Character 3 turn one image plus audio into a talking video.
Key takeaways
- There are four approaches: volume-based mouth-swap, viseme lip sync from audio, live face tracking and AI-generated talking video.
- Rhubarb Lip Sync is a free command-line tool that maps audio to 6 basic mouth shapes plus 3 optional ones.
- Character Animator uses 11 audio-based mouth shapes plus Neutral, Smile and Surprised.
- Hedra Character 3 takes a start image and audio; Hedra lists API pricing from 2.5 cents per second at 540p as of November 2026.
- For live streams, PNGtuber apps such as veadotube mini simply open the mouth when your microphone passes a set volume.
Already have a character or logo? See examples in real app screens: splash, onboarding, loading and empty-state screens, with the animation that plays in each.
What are the ways to make a character talk?
| Method | How it works | Tools | Keeps your exact design? |
|---|---|---|---|
| Mouth-swap by volume | Open mouth when the audio is loud, closed when quiet | veadotube mini, PNGTuber Plus | Yes |
| Viseme lip sync from audio | Speech analysis picks a mouth shape per sound | Adobe Character Animator, Cartoon Animator, Rhubarb Lip Sync | Yes |
| Live face tracking | Your face and voice drive a puppet in real time | Character Animator, Cartoon Animator, Live2D with VTube Studio | Yes |
| AI talking video from an image | A model generates every frame from a still and audio | Hedra Character 3 | Mostly; details can drift |
| AI lip sync on a video | A model re-times the mouth in existing footage | Kling Lip Sync | Mostly |
| Performance transfer | A filmed performance drives a character image | Runway Act-Two | Mostly |
For a live app avatar the trade-off is different; see lip sync vs amplitude for talking avatars.
Which mouth shapes do I need to draw for lip sync?
Fewer than you might think. Rhubarb Lip Sync uses six basic shapes and three optional ones:
- A: closed, for P, B and M.
- B: slightly open with teeth together, for most consonants and "EE".
- C: open, for vowels like "EH" and "AE".
- D: wide open, for "AA".
- E: slightly rounded, for "AO" and "ER".
- F: puckered, for "UW", "OW" and "W".
- G, H, X (optional): upper teeth on lower lip for F and V, tongue up for long L, and a rest mouth for pauses.
Adobe Character Animator recognises 11 audio-based mouth layers inside a Mouth group, plus Neutral, Smile and Surprised for silent moments. For a mouth-swap PNGtuber, two shapes (closed and open) are enough.
Draw every mouth on the same face so only the mouth changes, and keep the shapes in the character's own style.
How do I lip sync a 2D character for free?
Rhubarb Lip Sync runs on Windows, macOS and Linux, reads WAV or Ogg Vorbis audio, and exports TSV, XML or JSON, plus .dat files for Moho and OpenToonz. Its page lists integrations for After Effects, Spine, Vegas Pro and Visionaire Studio.
- Record a clean WAV of the dialogue.
- Save the script as a text file; giving Rhubarb the dialogue improves recognition.
- Run it:
rhubarb -f json --dialogFile script.txt -o mouth.json voice.wav
- Import the timings into your animation tool, or read the JSON in your own code, and show the matching mouth drawing for each time range.
Adobe Character Animator's free Starter mode works with predesigned puppets and real-time face tracking; building your own puppet uses the Pro features included in Creative Cloud Pro.
How do I make my own character talk to a voiceover?
Adobe Character Animator: build a layered Photoshop or Illustrator puppet with the mouth shapes named in a Mouth group, import your audio, drag it into the scene, select the puppet and choose Timeline, Compute Lip Sync Take from Scene Audio. You can then edit individual visemes on the timeline.
Reallusion Cartoon Animator 5: generates lip-sync animation from audio and adds facial and 2D motion capture; a 30-day free trial is available.
Both suit recurring characters on YouTube or in courses: you build the puppet once and every new voiceover takes minutes.
Can AI make a cartoon character talk from one image?
Yes. Hedra Character 3 takes a start image and an audio track (text direction optional) and returns video up to 10 minutes long, in aspect ratios from 9:21 to 21:9. Its examples include talking animals and 2D-illustrated and 3D-animated hosts. As of November 2026 Hedra lists API pricing of 2.5 cents per second at 540p, 5 cents at 720p and 6.25 cents at 1080p.
Kling's Lip Sync adds uploaded or text-to-speech audio to an existing Kling character video showing a clear face; Kling says it supports realistic, 3D and 2D human characters. Runway's Act-Two transfers a filmed performance onto a character image.
The trade-off: these models generate new frames, so line work, colours and small details can shift. Test with your own character before committing a series to it.
How do I make a character talk live on stream or in a call?
- Simplest: a PNGtuber app. veadotube mini opens the mouth when your mic passes a threshold you set; see how to make a PNGtuber.
- Face-driven: Character Animator's live mode uses your webcam and microphone for lip sync and face tracking; a Live2D model in VTube Studio does the same for VTubers.
- In an app or voice agent: drive a mouth parameter from audio volume or from viseme events your speech engine provides.
How do I make lip sync look natural?
- Start with clean audio. A dry, noise-free recording gives speech analysis far fewer wrong guesses than a track with music under it. Add the music after the lip sync is done.
- Give the tool your script. Rhubarb accepts a dialogue file, and Adobe's team has described transcript-based lip sync as more accurate than audio alone.
- Get the closed mouths right. P, B and M must fully close the lips; viewers notice a missed closure more than any other error.
- Do not change shape every frame. Holding each mouth for at least two frames reads as smoother speech.
- Animate more than the mouth. Blinks, small head nods on stressed words and eyebrow raises on questions carry a lot of the performance.
- Rest between sentences. Return to a neutral or rest mouth in pauses instead of freezing on the last vowel.
Can Charmotion make my character talk?
Disclosure: Charmotion is our product. We list it where it fits and say where other tools are better; every other fact links to its source.
No. Charmotion has no talking, lip-sync or mouth animation. It animates one finished character image in its own artwork into idle, launch, happy, wave and loading loops for apps, websites and videos. If your character needs to speak, use one of the tools above.
The two can work together: a lip-sync tool for dialogue, and Charmotion loops (transparent WebM or MOV) for the moments in between, such as a wave at the end or a happy reaction. See what the loops look like.
Frequently asked questions
What is a viseme?
A viseme is the visible mouth shape for a group of speech sounds. P, B and M all look like a closed mouth, so they share one viseme. Lip-sync tools map audio to visemes, which is why a handful of drawings can cover all speech.
How many mouth shapes does a talking cartoon need?
Two (closed and open) for a PNGtuber style, about six to nine for convincing lip sync (Rhubarb's set), and around a dozen in Character Animator, which also uses Neutral, Smile and Surprised.
Why does my lip sync look late or mushy?
Common causes are noisy audio, no script for the analyser, and mouths that change on every frame. Clean the recording, supply the dialogue text, and hold each mouth for at least two frames.
Can I make a non-human character talk?
Yes. Drawn mouth shapes work on any character with a mouth. Hedra's examples include talking animals. Kling's documentation mentions human characters only, so test a non-human mascot before relying on it.
Quick answers from this guide
Sources
- Rhubarb Lip Sync, Daniel Wolf on GitHub
- Lip sync mouth shapes, Adobe
- Feature focus: transcript-based lip sync, Adobe Community
- Adobe Character Animator, Adobe
- Cartoon Animator, Reallusion
- Hedra Character 3, Hedra
- Best lip sync video AI, Kling AI
- Performance Capture with Act-Two, Runway
- VTube Studio on Steam, Steam
- How Charmotion works, Charmotion