Category AI Voice Assistant
Tags AI audio
What SpeechGPT2 is
The model is described as an end-to-end speech dialogue language model developed at the School of Computer Science, Fudan University, able to sense and express emotion and give speech responses in multiple styles. The supplementary note calls it a fully end-to-end human-like speech dialogue model built on a 750bps ultra-low-bitrate speech codec and initialized from MIMO-LM.
What you can do with it
- Sense and express emotion in spoken dialogue
- Respond in styles such as rap, drama, robot, funny and whisper
- Model semantic and acoustic information with a 750bps speech codec
- Generate one second of speech in 25 autoregressive decoding steps
Getting started
- Visit the GitHub page for the technical report and code
- Read the technical report to understand the architecture and functions
- Download and install the required software dependencies
- Configure model parameters and training data from the documentation
What to watch out for
- The current system is turn-based while a full-duplex real-time version is in development
- The material notes limits in computing and data resources
- Noise robustness in speech understanding and stability of generated voice quality are stated as weak points
- The technical report, code and model weights are planned for future open source release
Pros & cons
✓ What we like
- Multiple speaking styles follow context and human instructions
- One second of speech takes 25 autoregressive decoding steps
! What to watch out for
- It is still a turn-based dialogue system
- Noise robustness and voice quality stability are noted as weak points
FAQ
Who developed it?
The material names the School of Computer Science, Fudan University.
Is everything released?
The material states a plan to open source the technical report, code and model weights in the future.
Last reviewed: 2026-11-15
More AI audio tools tools
View all →-
Accent Guesser Accent recognition from a recording Free tools Creator tools Audio Speech to text Education tools -
AccurateScribe.ai Transcription for audio and video Free tools Creator tools Video to text Audio Speech to text -
Adobe Podcast Browser podcast recording and cleanup Free tools Creator tools Audio Speech to text Audio editing -
Agilotext French-first transcription and summaries Audio Speech to text Podcast Meeting transcription Meeting notes -
AI Dubbing Short video dubbing without signup Free tools Video translation Audio Voiceover Content localization -
AI Mastering Automatic online mastering Free tools Freemium tools Audio Podcast