Dragengine.Audio.LipSync.LipSyncSpectrogram Class Reference

Spectrogram based lip synchronization. More...

Inheritance diagram for Dragengine.Audio.LipSync.LipSyncSpectrogram:

Classes
class	VisemeTarget
	Viseme target. More...

Public Member Functions
LipSyncSpectrogram	new ()
	Create lip sync.
Public Member Functions inherited from Dragengine.Audio.LipSync.LipSync
void	forEachVisemeResult (Block ablock)
	Visit viseme results with block.
void	forEachVisemeSubsetResult (Block ablock)
	Visit viseme subset results with block.
float	getJawOpeness ()
	Get jaw openess in range 0 to 1.
OneEuroFilter	getJawOpenessFilter ()
	Jaw openess filter.
float	getJawOpenessSmooth ()
	Smoothed jaw openess in range 0 to 1.
float	getUpdateInterval ()
	Update interval in seconds.
VisemeResult	getVisemeResult (Viseme viseme)
	Viseme result.
void	setJawOpeness (float value)
	Set jaw openess in range 0 to 1.
void	setJawOpenessSmooth (float value)
	Set smoothed jaw openess in range 0 to 1.
void	setNoiseFloor (float noiseFloor)
	Set noise floor in RMS (root mean square) amplitude.
void	setUpdateInterval (float updateInterval)
	Set update interval in seconds.

Public Attributes
Array	pBandEnergies
OneEuroFilter	pLow
float	pTotalBandEnergy
Dictionary	pVisemeTargets
Public Attributes inherited from Dragengine.Audio.LipSync.LipSync
AudioAnalyzer	pAudioAnalyzer
float	pJawOpeness
OneEuroFilter	pJawOpenessFilter
float	pNoiseFloor
Dictionary	pVisemeResults

Protected Member Functions
void	calculateVisemes (float elapsed)
	Calculate visemes based on band energies.
void	collectBandEnergies ()
	Collect band energies.
Array	getBandEnergies ()
	Band energies in normalized log-mel.
float	getEnergyHigh ()
	Get high energy bucket.
float	getEnergyLowMid ()
	Get low-mid energy bucket.
float	getEnergyMid ()
	Get mid energy bucket.
float	getFlatness ()
	Get spectral flatness.
float	getRms ()
	Get RMS energy.
float	getTotalBandEnergy ()
	Total band energy in log-mel.
float	getZcr ()
	Get zero-crossing rate.
void	updateLipSync (float elapsed)
Protected Member Functions inherited from Dragengine.Audio.LipSync.LipSync
void	normalizeVisemes ()
	Normalize viseme weights.
void	setJawOpenessFromVisemeSubset ()
	Set jaw openess from viseme subset results.
void	setVisemeFullFromSubset ()
	Set full viseme results from viseme subset results.
void	setVisemeSubsetFromFull ()
	Set viseme subset results from full viseme results.
void	setVisemeSubsetFromJawOpeness ()
	Set viseme subset from jaw openess.

Detailed Description

Spectrogram based lip synchronization.

Version

1.32

Provides lip synchronization by analyzing the spectrogram of the audio signal. Provides full viseme set. For more details see section "Full viseme set" and "Audio Analyzer" in LipSync.

Member Function Documentation

calculateVisemes()

void Dragengine.Audio.LipSync.LipSyncSpectrogram.calculateVisemes ( float  elapsed )

protected

Calculate visemes based on band energies.

collectBandEnergies()

void Dragengine.Audio.LipSync.LipSyncSpectrogram.collectBandEnergies ( )

protected

Collect band energies.

After this call band energies can be retrieved from #getBandEnergies() and total band energy from #getTotalBandEnergy().

getBandEnergies()

Array Dragengine.Audio.LipSync.LipSyncSpectrogram.getBandEnergies ( )

protected

Band energies in normalized log-mel.

Array of float as log-mel divided by #getTotalBandEnergy().

getEnergyHigh()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getEnergyHigh ( )

protected

Get high energy bucket.

getEnergyLowMid()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getEnergyLowMid ( )

protected

Get low-mid energy bucket.

getEnergyMid()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getEnergyMid ( )

protected

Get mid energy bucket.

getFlatness()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getFlatness ( )

protected

Get spectral flatness.

getRms()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getRms ( )

protected

Get RMS energy.

getTotalBandEnergy()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getTotalBandEnergy ( )

protected

Total band energy in log-mel.

getZcr()

float Dragengine.Audio.LipSync.LipSyncSpectrogram.getZcr ( )

protected

Get zero-crossing rate.

new()

LipSyncSpectrogram Dragengine.Audio.LipSync.LipSyncSpectrogram.new

(

)

Create lip sync.

Reimplemented from Dragengine.Audio.LipSync.LipSync.

updateLipSync()

void Dragengine.Audio.LipSync.LipSyncSpectrogram.updateLipSync ( float  elapsed )

protected

Member Data Documentation

pBandEnergies

Array Dragengine.Audio.LipSync.LipSyncSpectrogram.pBandEnergies

pLow

OneEuroFilter Dragengine.Audio.LipSync.LipSyncSpectrogram.pLow

pTotalBandEnergy

float Dragengine.Audio.LipSync.LipSyncSpectrogram.pTotalBandEnergy

pVisemeTargets

Dictionary Dragengine.Audio.LipSync.LipSyncSpectrogram.pVisemeTargets

---

The documentation for this class was generated from the following file:

• LipSyncSpectrogram.ds