Skip to content

TinyLlama

A compact 1.1B Llama model trained on 3 trillion tokens

Visit official site
Category AI models

What TinyLlama is

The project is described as an effort to pretrain a 1.1B Llama model on 3 trillion tokens with the Llama 2 architecture.

What you can do with it

  • Generate text with the released model
  • Use it in projects built on Llama
  • Serve applications with limited memory

Training notes

  • Training began on 2023-09-01
  • 16 A100-40G GPUs were used over 90 days
  • The feature list names Transformers and safetensors

Pros & cons

✓ What we like

  • Training finished in 90 days with 16 A100-40G GPUs
  • The compact size suits constrained applications

! What to watch out for

  • The model size is fixed at 1.1B parameters
  • Training started on 2023-09-01

FAQ

What is the training target?

A 1.1B Llama model on 3 trillion tokens.

Why does it fit small setups?

The material cites 1.1B parameters and limited compute and memory needs.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review