Base class for lip synchronization. More...
Classes | |
| enum | Viseme |
| Visemes. More... | |
| class | VisemeResult |
| Viseme result. More... | |
Public Member Functions | |
| void | forEachVisemeResult (Block ablock) |
| Visit viseme results with block. | |
| void | forEachVisemeSubsetResult (Block ablock) |
| Visit viseme subset results with block. | |
| float | getJawOpeness () |
| Get jaw openess in range 0 to 1. | |
| OneEuroFilter | getJawOpenessFilter () |
| Jaw openess filter. | |
| float | getJawOpenessSmooth () |
| Smoothed jaw openess in range 0 to 1. | |
| float | getUpdateInterval () |
| Update interval in seconds. | |
| VisemeResult | getVisemeResult (Viseme viseme) |
| Viseme result. | |
| LipSync | new () |
| Create lip sync. | |
| void | setJawOpeness (float value) |
| Set jaw openess in range 0 to 1. | |
| void | setJawOpenessSmooth (float value) |
| Set smoothed jaw openess in range 0 to 1. | |
| void | setNoiseFloor (float noiseFloor) |
| Set noise floor in RMS (root mean square) amplitude. | |
| void | setUpdateInterval (float updateInterval) |
| Set update interval in seconds. | |
Public Attributes | |
| AudioAnalyzer | pAudioAnalyzer |
| float | pJawOpeness |
| OneEuroFilter | pJawOpenessFilter |
| float | pNoiseFloor |
| Dictionary | pVisemeResults |
Protected Member Functions | |
| void | normalizeVisemes () |
| Normalize viseme weights. | |
| void | setJawOpenessFromVisemeSubset () |
| Set jaw openess from viseme subset results. | |
| void | setVisemeFullFromSubset () |
| Set full viseme results from viseme subset results. | |
| void | setVisemeSubsetFromFull () |
| Set viseme subset results from full viseme results. | |
| void | setVisemeSubsetFromJawOpeness () |
| Set viseme subset from jaw openess. | |
Detailed Description
Base class for lip synchronization.
- Version
- 1.32
Lip synchronization analyzes user voice audio and provides either a mouth openess value for jaw flapping or viseme weights for more detailed mouth shapes. See subclass documentation to know which result data they provide. All subclasses are required to fill in all possible values so the user can choose which category of lip sync he wants to use.
- Jaw flapping
This is the most basic form of lip synchronization best used for characters lacking complex mouth pose abilities. Typical example of this are beast type characters with elongated snouts. In this case designing convincing mouth shapes can be difficult unless the character is of a cartoon type where mouth shapes are often very squishy. Apply the #getJawOpenessSmooth() to the character jaw bone.
- Viseme subset (4 visemes)
This subset represents basic mouth shapes that can be easily distinguished. These viseme are provided:
- Viseme::sil: Mouth closed. Covers Viseme::silence and Viseme::pp.
- Viseme::aa: Open mouth. Covers Viseme::aa.
- Viseme::o: Round open mouth. Covers Viseme::o, Viseme::u, Viseme::ff, Viseme::ch and Viseme::kk.
- Viseme::e: Spread open mouth. Covers Viseme::e, Viseme::i, Viseme::ss, Viseme::th, Viseme::dd, Viseme::nn and Viseme::rr
- Full viseme set (14 visemes)
The most detailed mode providing full set of viseme weights. This is best used for characters with complex mouth pose abilities.
- Audio Analyzer
Lip synchronization uses an instance of AudioAnalyzer to analyze the audio. The subclasses configure the analyzer for their specific needs. The user is responsible to set the audio source if he does not want to use the default microphone.
Member Function Documentation
◆ forEachVisemeResult()
| void Dragengine.Audio.LipSync.LipSync.forEachVisemeResult | ( | Block | ablock | ) |
Visit viseme results with block.
Block receives one argument of type VisemeResult.
◆ forEachVisemeSubsetResult()
| void Dragengine.Audio.LipSync.LipSync.forEachVisemeSubsetResult | ( | Block | ablock | ) |
Visit viseme subset results with block.
Block receives one argument of type VisemeResult.
◆ getJawOpeness()
| float Dragengine.Audio.LipSync.LipSync.getJawOpeness | ( | ) |
Get jaw openess in range 0 to 1.
◆ getJawOpenessFilter()
| OneEuroFilter Dragengine.Audio.LipSync.LipSync.getJawOpenessFilter | ( | ) |
Jaw openess filter.
◆ getJawOpenessSmooth()
| float Dragengine.Audio.LipSync.LipSync.getJawOpenessSmooth | ( | ) |
Smoothed jaw openess in range 0 to 1.
Use this value for animating the character jaw bone.
◆ getUpdateInterval()
| float Dragengine.Audio.LipSync.LipSync.getUpdateInterval | ( | ) |
Update interval in seconds.
◆ getVisemeResult()
| VisemeResult Dragengine.Audio.LipSync.LipSync.getVisemeResult | ( | Viseme | viseme | ) |
Viseme result.
◆ new()
| LipSync Dragengine.Audio.LipSync.LipSync.new | ( | ) |
Create lip sync.
Reimplemented in Dragengine.Audio.LipSync.LipSyncJawFlaping, and Dragengine.Audio.LipSync.LipSyncSpectrogram.
◆ normalizeVisemes()
|
protected |
Normalize viseme weights.
◆ setJawOpeness()
| void Dragengine.Audio.LipSync.LipSync.setJawOpeness | ( | float | value | ) |
Set jaw openess in range 0 to 1.
◆ setJawOpenessFromVisemeSubset()
|
protected |
Set jaw openess from viseme subset results.
◆ setJawOpenessSmooth()
| void Dragengine.Audio.LipSync.LipSync.setJawOpenessSmooth | ( | float | value | ) |
Set smoothed jaw openess in range 0 to 1.
◆ setNoiseFloor()
| void Dragengine.Audio.LipSync.LipSync.setNoiseFloor | ( | float | noiseFloor | ) |
Set noise floor in RMS (root mean square) amplitude.
◆ setUpdateInterval()
| void Dragengine.Audio.LipSync.LipSync.setUpdateInterval | ( | float | updateInterval | ) |
Set update interval in seconds.
◆ setVisemeFullFromSubset()
|
protected |
Set full viseme results from viseme subset results.
◆ setVisemeSubsetFromFull()
|
protected |
Set viseme subset results from full viseme results.
◆ setVisemeSubsetFromJawOpeness()
|
protected |
Set viseme subset from jaw openess.
Member Data Documentation
◆ pAudioAnalyzer
| AudioAnalyzer Dragengine.Audio.LipSync.LipSync.pAudioAnalyzer |
◆ pJawOpeness
| float Dragengine.Audio.LipSync.LipSync.pJawOpeness |
◆ pJawOpenessFilter
| OneEuroFilter Dragengine.Audio.LipSync.LipSync.pJawOpenessFilter |
◆ pNoiseFloor
| float Dragengine.Audio.LipSync.LipSync.pNoiseFloor |
◆ pVisemeResults
| Dictionary Dragengine.Audio.LipSync.LipSync.pVisemeResults |
The documentation for this class was generated from the following file:
