Skip to content

AV-HuBERT

Learn audio visual speech representations by self supervision

Visit official site

What AV-HuBERT is

The framework is described as a self supervised approach to audio visual speech processing that learns speech representations.

What you can do with it

  • Run audio visual speech recognition experiments
  • Build recognition apps for varied language settings
  • Support language learning tool development

Research directions

  • Audio visual speech representation learning is listed
  • Automatic speech recognition system development is listed
  • Multimodal data cluster analysis is listed

Pros & cons

✓ What we like

  • Lip reading, ASR and audio visual recognition are covered
  • Robust self supervised recognition is described

! What to watch out for

  • Learning depends on masked multimodal clusters
  • The scope is audio visual speech processing

FAQ

What tasks are named?

Lip reading, ASR and audio visual speech recognition are listed.

How does it learn?

The material names masked multimodal cluster prediction.

Last reviewed: 2026-11-15

More AI audio tools tools

View all →

How we review