Skip to content

DeepGEMM

A CUDA library for efficient FP8 matrix multiplication

Visit official site

What deepgemm is

The library is a CUDA project for efficient FP8 matrix multiplication, using fine grained scaling and a full JIT design.

What you can do with it

  • Accelerate FP8 matrix multiplication in training
  • Optimize mixture of experts inference with grouped GEMM
  • Integrate the kernels into an existing deep learning framework

Getting started

  • Clone the repository with its submodules
  • Install Python, CUDA and PyTorch dependencies
  • Use develop mode install and run the JIT tests

Pros & cons

✓ What we like

  • Persistent thread specialization and FFMA tuning are included
  • The small core is easy to learn and adapt

! What to watch out for

  • NVIDIA Hopper Tensor Cores are required
  • Python 3.8+, CUDA 12.3+ and PyTorch 2.1+ are needed

FAQ

How large is the core?

The material states about 300 lines.

Does it need install time compilation?

No, the material states a full JIT design.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review