Skip to content

Large Concept Models

Language modelling in a sentence representation space

Visit official site

What Large Concept Models is

The model is described as a Facebook Research language model that works in a sentence representation space through SONAR embeddings.

What you can do with it

  • Compare semantics across languages in research
  • Build multilingual chatbots and voice assistants

Getting started

  • Install required packages including SONAR
  • Prepare training data and embed it with SONAR
  • Choose a training configuration and model parameters
  • Run the training script to start training

Pros & cons

✓ What we like

  • Official implementation and experiments are provided
  • Training and fine-tuning can be reproduced

! What to watch out for

  • The model works in a sentence representation space
  • Experiments use the stated 1.6B parameter setup

FAQ

What embedding space is used?

The material names the SONAR embedding space.

Is the implementation public?

The material states an official open-source implementation and experiments are available.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review