What torchao is
torchao is described as a PyTorch library for custom data types that quantises and sparsifies weights, gradients, optimisers and activations.
What you can do with it
- Apply post training quantisation or QAT
- Quantise a KV cache for long context inference
- Train with scaled float8 or 2:4 sparsity
Example uses
- Quantise an image segmentation model for faster inference
- Apply QAT to a language model
- Cut memory use in diffusion inference with sparsity
Pros & cons
✓ What we like
- Compatible with torch.compile() and FSDP2
- A developer API allows custom quantisation algorithms
! What to watch out for
- Quantisation strategy is chosen per model
- Local work starts with installing the library
FAQ
What does it quantise?
Weights, gradients, optimisers and activations are named.
Is long context covered?
KV cache quantisation is listed for long context inference.
Last reviewed: 2026-11-15
More AI coding tools tools
View all →-
100 Vibe Coding Learn vibe coding by building Free tools Enterprise tools Developer tools API integration Coding -
10Web Prompt to WordPress site with hosting Free tools Developer tools No code development Coding Website builder -
16x Prompt Context management for AI coding Free tools Workflow automation Developer tools API integration Coding -
a0.dev Chat-built mobile apps to store Free tools Developer tools No code development Coding App builder -
Adalo No-code apps with a visual canvas Free tools Developer tools No code development Coding App builder -
AgentQL Natural language web data extraction Freemium tools Developer tools Browser extension Data extraction Coding