Category AI voice recognition
What whisper-diarization is
The project is described as an open-source combination of Whisper ASR, voice activity detection and speaker embeddings, extracting voiced parts of audio to improve speaker embedding accuracy. The tool produces transcripts with Whisper, corrects timestamps with WhisperX, and links word-level speaker detection to those timestamps.
What you can do with it
- Generate high-quality speech transcripts with Whisper ASR
- Exclude silence with voice activity detection
- Identify speakers with speaker embeddings
- Correct and align timestamps with WhisperX
How the pipeline works
- MarbleNet performs VAD and segmentation to drop silent parts
- TitaNet extracts speaker embeddings for each segment
- Results are linked to WhisperX timestamps to detect the speaker of each word
- A punctuation model realigns the transcript to compensate for small time shifts
Getting started
- Install FFMPEG and Cython as prerequisites
- Clone or download the code base
- Adjust WhisperX and NeMo parameters in diarize.py and helpers.py
- Run the model with a command line tool, passing parameters and an audio file name
Pros & cons
✓ What we like
- Batch inference is supported for more efficient processing
- Voiced parts are extracted before speaker embedding to raise accuracy
! What to watch out for
- FFMPEG and Cython are listed as prerequisites
- WhisperX and NeMo parameters need editing in the project files
FAQ
What components are involved?
The material names Whisper, WhisperX, MarbleNet, TitaNet, a punctuation model and NeMo parameters.
What must be installed first?
The material lists FFMPEG and Cython as prerequisites, then the code base.
Last reviewed: 2026-11-15
More AI audio tools tools
View all →-
Accent Guesser Accent recognition from a recording Free tools Creator tools Audio Speech to text Education tools -
AccurateScribe.ai Transcription for audio and video Free tools Creator tools Video to text Audio Speech to text -
Adobe Podcast Browser podcast recording and cleanup Free tools Creator tools Audio Speech to text Audio editing -
Agilotext French-first transcription and summaries Audio Speech to text Podcast Meeting transcription Meeting notes -
AI Dubbing Short video dubbing without signup Free tools Video translation Audio Voiceover Content localization -
AI Mastering Automatic online mastering Free tools Freemium tools Audio Podcast