What deepgemm is
The library is a CUDA project for efficient FP8 matrix multiplication, using fine grained scaling and a full JIT design.
What you can do with it
- Accelerate FP8 matrix multiplication in training
- Optimize mixture of experts inference with grouped GEMM
- Integrate the kernels into an existing deep learning framework
Getting started
- Clone the repository with its submodules
- Install Python, CUDA and PyTorch dependencies
- Use develop mode install and run the JIT tests
Pros & cons
✓ What we like
- Persistent thread specialization and FFMA tuning are included
- The small core is easy to learn and adapt
! What to watch out for
- NVIDIA Hopper Tensor Cores are required
- Python 3.8+, CUDA 12.3+ and PyTorch 2.1+ are needed
FAQ
How large is the core?
The material states about 300 lines.
Does it need install time compilation?
No, the material states a full JIT design.
Last reviewed: 2026-11-15
More AI coding tools tools
View all →-
100 Vibe Coding Learn vibe coding by building Free tools Enterprise tools Developer tools API integration Coding -
10Web Prompt to WordPress site with hosting Free tools Developer tools No code development Coding Website builder -
16x Prompt Context management for AI coding Free tools Workflow automation Developer tools API integration Coding -
a0.dev Chat-built mobile apps to store Free tools Developer tools No code development Coding App builder -
Adalo No-code apps with a visual canvas Free tools Developer tools No code development Coding App builder -
AgentQL Natural language web data extraction Freemium tools Developer tools Browser extension Data extraction Coding