An AI-powered and automatically generated Lip-sync animation brings a smoother and more efficient way to animate and edit facial expressions. AccuLIPS detects and extracts text and viseme from audio, or imports a prepared script for precise lipsyncing. Create natural speak with the co-articulation design, and further fine-tune every word’s visemes and strength level. AccuLIPS contains an English dictionary with 200,000 default words, that are easily customized by adding new words, or changing others.
The iClone AccuLIPS system can automatically generate accurate text data, and correct viseme timing from an imported voice. This design gives users a highly manageable approach to professional lip-sync animation. This method lets you generate text from audio, while automatically aligning words to their corresponding voice sections. If the system is further aided with a text script, then you can get a more accurate text-to-voice alignment.
AccuLIPS allows users to add new words with their corresponding lip shapes defined in the Lips Track, or modify existing words from the main dictionary ( 200,000 words). Users are also able to save and load custom dictionaries.
Enhanced Mouth Shapes
With enhanced mouth shapes for each viseme, the lipsync result is more precise and natural for both realistic and stylized characters.
Co-Articulation
AccuLIPS is designed to simulate real human speech behavior. For true-to-life talking animations, each mouth shape takes on the qualities of other mouth shapes that precede or follow them. i.e - articulators either anticipate the next sound or carryover qualities from the prior sound.
Schwa
Schwa(/ə/) is the weak vowel sound in some syllables that is not emphasized in words.
Slightly Open
Mouth shape would be slightly opened when there’s a break between words and sentences.
Phoneme Pair
Mouth movements are governed by pairing 8 lip-dental shapes, 7 tongue positions, and mouth width; all blended with varied time curves to represent more natural transitions between a total of 240 possible phoneme combinations.
Mouth Shape Affected by the Succeeding Phonemes
When we pronounce words, the preceding phonemes will be affected by the succeeding phonemes, therefore the visemes of the preceding phonemes may be different, like KEY/COOL/CAR.
Mouth Shape Affected by the Preceding Phonemes
Same emphasis on the succeeding phonemes - the phonemes that follow will also be affected by the preceding phonemes, like CHIPS/KICK/LICK.
Though it’s good to see all the detected visemes correctly align to each word, highly frequent lips and jaw movements can cause an unwanted mechanical appearance. The smart downsampling tool can effectively alleviate the jitters by reducing less prominent lip keys, just like how humans tend to slur in a effortless manner.
Phoneme pair design is the foundation of iClone’s Lipsync without abrupt transitions between certain viseme pairings, and with optimized transition curves for smoother lip blends, especially when talking fast.
The strength of lips and jaw movement varies due to when and how people speak. It can be influenced by speed or situation, such as whispering is different from yelling, mumbling is different from enunciating. The Talking Style Presets give you the quick setup over lips, jaw and viseme strengths optimized for different talking occasions.
iClone Lip-sync technology realizes the character lip-sync animation with an easy and intuitive process. The system will align the visemes of phonemes to what the character says, and allow users to further fine-tune. Multiple ways to import the voice resources for characters, permits the lip-sync editing to be more flexible.
Record Voice
Record the voice directly in iClone, so that the voice clip will immediately show.
TTS
Text to Speech. Through the text in the panel, the default voice will be generated according to what is typed.
Audio File
Import an audio file, so that the system can generate their corresponding visemes in the Timeline.
When importing voice files, the system will auto generate a voice track with corresponding timeline visemes (with AccuLIPS there will also be a text track). Users can intuitively adjust the voice and animation.
Adjust the selected viseme with the Lips Editor; including exchanging it for another, deleting it or adding new ones, and editing its Expressiveness strength.
Use Lip Options to adjust the Smooth Options and Clip Strength of a selected clip. You can also separately fine-tune smooth level for Lips, Tongue and Jaw.
MULTI-LANGUAGES SUPPORT
By cleverly bypassing voice-to-text recognition, AccuLIPS can extend its capabilities to other languages beyond English. Lip detection and accuracy can be further improved with additional text scripts while free tools are readily available to convert non-English audio files into their romanized counterparts.
CAPTURE YOUR VOICE AND FACIAL ANIMATION IN ONCE WITH ACCUFACE
AccuFACE plugin for iClone is an AI facial tracker powered by NVIDIA RTX GPU, capable of capturing facial expressions from both webcams and conventional video files. Smoothly synchronize facial movements with voice input. It also fine-tunes lip and tongue articulation using proprietary AccuLIPS technology.
