Skip to content

FlashMLA

An MLA decoding kernel tuned for Hopper GPUs

Visit official site

What flashmla is

The kernel is described as an efficient MLA decoding kernel for Hopper GPUs, aimed at variable-length sequence serving.

What you can do with it

  • Install the kernel with the setup script
  • Run the provided benchmark test
  • Import the module and fetch scheduling metadata

What to watch out for

  • CUDA 12.3 or newer is the stated baseline
  • PyTorch 2.0 or newer is the stated baseline
  • A paged KV cache uses a block size of 64

Pros & cons

✓ What we like

  • Hopper GPU acceleration is used directly
  • PyTorch 2.0 integration fits existing projects

! What to watch out for

  • CUDA 12.3 or newer is required
  • Users install the kernel and run the benchmark themselves

FAQ

Which CUDA version is needed?

The material states CUDA 12.3 or newer is supported.

What performance figures are given?

The material gives 3000 GB/s memory bandwidth and 580 TFLOPS on an H800 SXM5.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review