Skip to content

seed-tts-eval

An objective test set for zero-shot speech generation

Visit official site

What seed-tts-eval is

The tool is described as a test set for evaluating zero-shot speech generation across domains, with samples taken from English and Mandarin public corpora. It uses 1000 samples from Common Voice and 2000 samples from DiDiSpeech-2 to measure models on objective metrics.

What you can do with it

  • Measure models with Word Error Rate and Speaker Similarity
  • Evaluate zero-shot TTS and voice conversion tasks
  • Download the test set samples through the provided link

Evaluation setup

  • Use 1000 Common Voice samples and 2000 DiDiSpeech-2 samples
  • Apply Whisper-large-v3 for English and Paraformer-zh for Mandarin ASR
  • Use the WavLM-large model for speaker similarity scoring

Application scenarios

  • Evaluate the performance of a new speech synthesis model
  • Compare different speech synthesis techniques
  • Use the test set as teaching material for speech synthesis

Getting started

  • Visit the GitHub page of the test set
  • Read the README for dependencies and usage
  • Download the required test set samples
  • Run the provided evaluation code on a model

Pros & cons

✓ What we like

  • A download link for the test set is provided
  • Both zero-shot TTS and VC tasks can be evaluated

! What to watch out for

  • The samples come from English and Mandarin public corpora
  • Speaker similarity is measured with the WavLM-large model

FAQ

What does it evaluate?

It evaluates zero-shot speech generation, covering TTS and voice conversion tasks.

Which metrics are used?

Word Error Rate (WER) and Speaker Similarity (SIM) are listed, with WavLM-large for speaker similarity.

Last reviewed: 2026-11-15

More AI audio tools tools

View all →

How we review