Skip to content

SliceGPT

Compress a language model by slicing matrices

Visit official site
Category AI models

What SliceGPT is

SliceGPT is described as a post-training sparsification scheme that cuts embedding dimensions by replacing weight matrices with smaller dense ones.

What you can do with it

  • Remove parameters from a trained model
  • Reduce memory use of a large model
  • Speed up inference on fewer GPUs

Results reported

  • Up to 25 percent of parameters are removed
  • Zero-shot performance stays at 99, 99 and 90 percent
  • Compute falls to 64 percent on a 24GB GPU
  • Compute falls to 66 percent on a 40GB A100 GPU

Pros & cons

✓ What we like

  • Runs on fewer GPUs without extra code changes
  • Inference compute drops to 64 percent on a 24GB GPU

! What to watch out for

  • The scheme targets post-training sparsification
  • Reported numbers come from named models

FAQ

How much can be removed?

The paper reports up to 25 percent of parameters including embeddings.

Where were the compute results measured?

On a 24GB consumer GPU and a 40GB A100 GPU.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review