Skip to content

GaussianSpeech

Synthesise 3D head avatars from speech signals

Visit official site

What GaussianSpeech is

The method is described as synthesising high-fidelity animation sequences from speech signals, building realistic personalised 3D head avatars with 3D Gaussian rendering.

What you can do with it

  • Generate 3D head avatar animation from a speech signal
  • Render detail such as teeth, wrinkles and eye gloss
  • Present natural visual dynamics at real-time speed
  • Produce personalised colours tied to expression

Getting started

  • Visit the GitHub page and download code and datasets
  • Set up the environment and install dependencies
  • Process speech with a Wav2Vec 2.0 encoder
  • Feed audio features to the lip and wrinkle encoders

Pros & cons

✓ What we like

  • A large multi-view audio-visual dataset supports training
  • Wav2Vec 2.0 extracts audio features for lip and expression work

! What to watch out for

  • Animation is driven by the input speech signal
  • Training relies on the multi-view audio-visual dataset

FAQ

What does it generate?

The material describes high-fidelity animated 3D head avatars from speech.

How are audio features extracted?

The feature list names a Wav2Vec 2.0 encoder for general audio features.

Last reviewed: 2026-11-15

More AI video generation tools

View all →

How we review