Skip to content

Nemotron-CC

A 6.3 trillion token pretraining dataset

Visit official site

What Nemotron-CC is

The dataset is a 6.3 trillion token collection that turns Common Crawl into a long-term pretraining set.

What you can do with it

  • Use raw and synthetic tokens
  • Choose a quality level and partition
  • Download jsonl or parquet data

Application scenarios

  • An 8B model is trained on the set
  • A model is trained for 15T tokens
  • Researchers study quality levels

Pros & cons

✓ What we like

  • Quality levels and partitions are available
  • Classifier ensembles and synthetic rewriting are used

! What to watch out for

  • The dataset is tied to English Common Crawl
  • Heuristic filter reliance is reduced, not removed

FAQ

How large is it?

The material states 6.3 trillion tokens.

Which formats?

jsonl and parquet are listed.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review