Skip to content

Moonlight

A 16B mixture-of-experts model trained with Muon

Visit official site

What Moonlight is

The model is described as a 16B parameter mixture-of-experts model trained with the Muon optimiser for efficient and stable large-scale training.

What you can do with it

  • Use the pre-trained or instruction-tuned version
  • Generate text and code
  • Deploy through Hugging Face
  • Run large-scale distributed training

Getting started

  • Install Python 3.10, PyTorch and transformers
  • Download the pretrained weights from Hugging Face
  • Load the model and tokenizer with transformers
  • Prepare input text such as a maths prompt

Pros & cons

✓ What we like

  • Compatible with Hugging Face for deployment
  • Supports both text generation and code generation

! What to watch out for

  • The listed setup requires Python 3.10, PyTorch and transformers
  • The pre-trained weights must be downloaded before loading

FAQ

How large is the model?

The material states 16B parameters in a mixture-of-experts design.

Which benchmarks are named?

The feature list names MMLU and BBH.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review