What ReDrafter is
The method is described as speculative decoding that combines an RNN draft model with dynamic tree attention to speed up token generation on NVIDIA GPUs.
What you can do with it
- Run speculative decoding on supported models
- Cut the delay a user experiences
- Reduce GPU usage and energy consumption
- Work with several open-source LLMs
Getting started
- Install and configure NVIDIA TensorRT-LLM
- Get the open-source code from GitHub
- Follow the docs to integrate the method
- Pick an open-source LLM for testing
Pros & cons
✓ What we like
- The method is described as reducing user latency
- Several open-source LLMs are supported
! What to watch out for
- A TensorRT-LLM environment must be set up first
- Integration follows the project documentation on GitHub
FAQ
Who developed it?
The material states Apple machine learning research developed it with NVIDIA.
What speed figures are given?
The material states up to 3.5 tokens per generation step and a 2.7x lift in tokens per second.
Last reviewed: 2026-11-15
More AI coding tools tools
View all →-
100 Vibe Coding Learn vibe coding by building Free tools Enterprise tools Developer tools API integration Coding -
10Web Prompt to WordPress site with hosting Free tools Developer tools No code development Coding Website builder -
16x Prompt Context management for AI coding Free tools Workflow automation Developer tools API integration Coding -
a0.dev Chat-built mobile apps to store Free tools Developer tools No code development Coding App builder -
Adalo No-code apps with a visual canvas Free tools Developer tools No code development Coding App builder -
AgentQL Natural language web data extraction Freemium tools Developer tools Browser extension Data extraction Coding