Skip to content

SALMONN

A large language model that hears speech, audio events and music

Visit official site

What SALMONN is

The model is described as a large language model developed by the Department of Electronic Engineering at Tsinghua University and ByteDance that accepts speech, audio events and music as input. It can perceive and understand varied audio input, which the material links to multilingual speech recognition, translation and audio-speech co-reasoning.

What you can do with it

  • Perform multilingual speech recognition
  • Perform multilingual speech translation
  • Apply audio-speech co-reasoning

Model notes

  • The project is hosted on GitHub at https://github.com/bytedance/SALMONN
  • Speech, audio events and music are all listed as input
  • The material contrasts it with models that accept only speech or audio events
  • The material describes hearing and cognitive hearing as abilities given to the LLM

Example inputs

  • gunshots.wav is listed as an input file
  • duck.wav is listed as an input file
  • music.wav is listed as an input file

Pros & cons

✓ What we like

  • Perceives varied audio input beyond speech alone
  • Emerging abilities include translation and co-reasoning

! What to watch out for

  • The described input scope is speech, audio events and music
  • The material frames the model as a step toward artificial general intelligence with hearing

FAQ

Who developed the model?

The material names the Department of Electronic Engineering at Tsinghua University and ByteDance.

What input does it accept?

Speech, audio events and music are listed as supported input.

Last reviewed: 2026-11-15

More AI audio tools tools

View all →

How we review