Skip to content

DenseAV

Learn audio-visual alignment from video

Visit official site

What DenseAV is

The model is described as a dual-encoder localisation architecture that learns dense audio-visual alignment features from video through self-supervision.

What you can do with it

  • Word meaning and sound location found without supervision
  • Multi-head feature aggregation for contrastive learning
  • Two new datasets contributed for evaluation

Application scenarios

  • Understand dialogue and scenes in video
  • Locate key sounds and objects in video
  • Improve sound and language based retrieval

Pros & cons

✓ What we like

  • Self-supervised learning works without labels
  • The material describes gains on semantic segmentation

! What to watch out for

  • The described input is video to be learned from
  • Localisation emerges from contrastive learning

FAQ

Does it need labelled data?

The material states patterns are learned without labels through self-supervision.

What does the operator do?

The material says a multi-head feature aggregation operator compares dense image and audio representations.

Last reviewed: 2026-11-15

More AI video generation tools

View all →

How we review