Skip to content

Star-Attention

Block-sparse attention for faster long-sequence inference

Visit official site

What Star-Attention is

The technique is described as a block-sparse attention method from NVIDIA for faster Transformer inference on long sequences.

What you can do with it

  • Apply two-stage local and global attention
  • Run it on an existing Transformer LLM
  • Combine it with Flash Attention

Performance notes

  • Up to 11x faster inference is cited
  • 95-100% accuracy is cited in the material
  • KV cache compression can be combined

Pros & cons

✓ What we like

  • No additional training or fine-tuning is required
  • Model and sequence length settings are flexible

! What to watch out for

  • The technique targets long-sequence workloads
  • Speed figures are cited from the feature list

FAQ

Does it need retraining?

The material states it works without extra training or fine-tuning.

What speed is cited?

The feature list cites up to 11x faster inference with 95-100% accuracy.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review