What seed-tts-eval is
The tool is described as a test set for evaluating zero-shot speech generation across domains, with samples taken from English and Mandarin public corpora. It uses 1000 samples from Common Voice and 2000 samples from DiDiSpeech-2 to measure models on objective metrics.
What you can do with it
- Measure models with Word Error Rate and Speaker Similarity
- Evaluate zero-shot TTS and voice conversion tasks
- Download the test set samples through the provided link
Evaluation setup
- Use 1000 Common Voice samples and 2000 DiDiSpeech-2 samples
- Apply Whisper-large-v3 for English and Paraformer-zh for Mandarin ASR
- Use the WavLM-large model for speaker similarity scoring
Application scenarios
- Evaluate the performance of a new speech synthesis model
- Compare different speech synthesis techniques
- Use the test set as teaching material for speech synthesis
Getting started
- Visit the GitHub page of the test set
- Read the README for dependencies and usage
- Download the required test set samples
- Run the provided evaluation code on a model
Pros & cons
✓ What we like
- A download link for the test set is provided
- Both zero-shot TTS and VC tasks can be evaluated
! What to watch out for
- The samples come from English and Mandarin public corpora
- Speaker similarity is measured with the WavLM-large model
FAQ
What does it evaluate?
It evaluates zero-shot speech generation, covering TTS and voice conversion tasks.
Which metrics are used?
Word Error Rate (WER) and Speaker Similarity (SIM) are listed, with WavLM-large for speaker similarity.
Last reviewed: 2026-11-15
More AI audio tools tools
View all →-
Accent Guesser Accent recognition from a recording Free tools Creator tools Audio Speech to text Education tools -
AccurateScribe.ai Transcription for audio and video Free tools Creator tools Video to text Audio Speech to text -
Adobe Podcast Browser podcast recording and cleanup Free tools Creator tools Audio Speech to text Audio editing -
Agilotext French-first transcription and summaries Audio Speech to text Podcast Meeting transcription Meeting notes -
AI Dubbing Short video dubbing without signup Free tools Video translation Audio Voiceover Content localization -
AI Mastering Automatic online mastering Free tools Freemium tools Audio Podcast