Category AI voice recognition
What LookOnceToHear is
The system is described as a smart headset interaction system that lets users pick a target speaker by looking at the person for a few seconds. It performs real-time speech extraction with synthesized audio mixtures, head-related transfer functions and binaural room impulse responses, and received an honorable mention at CHI 2024.
What you can do with it
- Select a target speaker by looking at them for a few seconds
- Generate audio mixtures with the Scaper toolkit
- Use the self-contained dataset and .jams specification files for training
- Evaluate real-time speech extraction with the provided checkpoints
Application scenarios
- Choose a specific speaker to hear during a meeting
- Help hearing-impaired people focus on conversation in noisy public places
- Distinguish and extract multiple sound sources in audio analysis research
Getting started
- Download and extract the provided .zip file into the data/ directory
- Run the command to start the training process
- Use Scaper generate_from_jams on the .jams specification files to build audio mixtures
- Download and load the target speech hearing model checkpoints for evaluation
Pros & cons
✓ What we like
- Model checkpoints are provided for training and evaluation
- Real-time speech extraction works in noisy environments
! What to watch out for
- Training starts from the provided .zip data and .jams specification files
- The workflow relies on Scaper to generate audio mixtures
FAQ
How is a speaker selected?
The material states a user looks at the target speaker for a few seconds to choose the voice to hear.
What is supplied for training?
A self-contained dataset, .jams specification files, Scaper-based mixture generation and model checkpoints are listed.
Last reviewed: 2026-11-15
More AI audio tools tools
View all →-
Accent Guesser Accent recognition from a recording Free tools Creator tools Audio Speech to text Education tools -
AccurateScribe.ai Transcription for audio and video Free tools Creator tools Video to text Audio Speech to text -
Adobe Podcast Browser podcast recording and cleanup Free tools Creator tools Audio Speech to text Audio editing -
Agilotext French-first transcription and summaries Audio Speech to text Podcast Meeting transcription Meeting notes -
AI Dubbing Short video dubbing without signup Free tools Video translation Audio Voiceover Content localization -
AI Mastering Automatic online mastering Free tools Freemium tools Audio Podcast