Skip to content

DCLM-7B

A language model built on curated data

Visit official site
Category AI models

What DCLM-7B is

The model is a Decoder-only Transformer language model from the DCLM team, trained mainly on English with systematic data curation. Training used PyTorch with the OpenLM framework and AdamW.

What you can do with it

  • Handle decoding tasks with a Transformer stack
  • Work with English language processing
  • Draw on StarCoder and ProofPile2 data

Example uses

  • Evaluate zero-shot and few-shot behavior
  • Feed question answering applications
  • Show how a language model works to students

Pros & cons

✓ What we like

  • Training details and evaluation results are provided
  • MMLU, HellaSwag and Jeopardy are listed as tasks

! What to watch out for

  • English is the primary language of the model
  • The published setup uses PyTorch with OpenLM

FAQ

Who developed it?

The material names the DataComp for Language Models team.

What optimizer is used?

The material states AdamW with a 2e-3 peak learning rate.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review