Category AI models
What SliceGPT is
SliceGPT is described as a post-training sparsification scheme that cuts embedding dimensions by replacing weight matrices with smaller dense ones.
What you can do with it
- Remove parameters from a trained model
- Reduce memory use of a large model
- Speed up inference on fewer GPUs
Results reported
- Up to 25 percent of parameters are removed
- Zero-shot performance stays at 99, 99 and 90 percent
- Compute falls to 64 percent on a 24GB GPU
- Compute falls to 66 percent on a 40GB A100 GPU
Pros & cons
✓ What we like
- Runs on fewer GPUs without extra code changes
- Inference compute drops to 64 percent on a 24GB GPU
! What to watch out for
- The scheme targets post-training sparsification
- Reported numbers come from named models
FAQ
How much can be removed?
The paper reports up to 25 percent of parameters including embeddings.
Where were the compute results measured?
On a 24GB consumer GPU and a 40GB A100 GPU.
Last reviewed: 2026-11-15
More LLM API platform tools
View all →-
302.AI Aggregated cloud platform that hosts hundreds of large models behind one API Developer tools API integration Multi model platform Model deployment LLM API platform -
360智脑开放平台 360 model API platform API integration Multi model platform Knowledge base Q&A Enterprise automation LLM API platform -
4o Image API Image generation API gateway Free tools Freemium tools Developer tools API integration LLM API platform -
AI-Flow No-code multi-model workflows Workflow automation Developer tools API integration Knowledge base Q&A LLM API platform -
AI/ML API Single gateway to many models Developer tools API integration Multimodal Multi model platform LLM API platform -
Amazon Bedrock AWS code suggestions with security scans Free tools Code assistant IDE App builder LLM API platform