Skip to content

NaturalSpeech 3

Zero-shot speech synthesis with factorized codec and diffusion

Visit official site

What NaturalSpeech 3 is

The system is described as a zero-shot speech synthesis system that decomposes content, prosody, timbre and acoustic details and generates each attribute separately. It uses a neural codec with factorized vector quantization and a factorized diffusion model for the attribute subspaces.

What you can do with it

  • Synthesize speech in a zero-shot setting
  • Generate natural and fluent speech in text-to-speech tasks
  • Adjust duration, prosody and timbre through attribute manipulation
  • Integrate the system into a speech recognition system

How it is built

  • A neural codec applies factorized vector quantization to speech waveforms
  • Content, prosody, timbre and acoustic details are generated separately
  • A factorized diffusion model generates attributes for each subspace
  • Each subspace attribute follows its corresponding prompt

Pros & cons

✓ What we like

  • Speech attributes are generated separately
  • Each subspace attribute follows a corresponding prompt

! What to watch out for

  • Generation depends on decomposed attribute subspaces
  • The neural codec relies on factorized vector quantization of waveforms

FAQ

What kind of synthesis is it?

The material describes a zero-shot speech synthesis system.

Which speech attributes are decomposed?

Content, prosody, timbre and acoustic details are named.

Last reviewed: 2026-11-15

More AI audio tools tools

View all →

How we review