Category AI models
What T-MAC is
The library is described as a kernel set that uses lookup tables to speed up low-bit LLM inference on CPUs without dequantisation.
What you can do with it
- Run mixed-precision matrix multiplication directly
- Work with 1 to 4 bit quantised Llama models
- Deploy natively on Windows ARM
- Lower CPU core use and power draw
Getting started
- Install Python 3.8 for TVM
- Add cmake 3.22 or newer and dependencies
- Install T-MAC with pip in a virtual environment
- Run the provided scripts or integrate with llama.cpp
Model notes
- Stated throughput is 20 tokens per second on one core
- Four cores are stated at 48 tokens per second
- The source compares it with llama.cpp at 4 to 5 times
Pros & cons
✓ What we like
- The material states 4 to 5 times the speed of llama.cpp
- CPU core use and power draw are reduced
! What to watch out for
- Python 3.8 and cmake 3.22 or newer are needed
- Setup uses a virtual environment and pip
FAQ
What speeds are stated?
The material states 20 tokens per second on a single core and 48 on four cores.
Which platforms are named?
The material names OSX, Linux, Windows ARM and Intel CPUs.
Last reviewed: 2026-11-15
More LLM API platform tools
View all →-
302.AI Aggregated cloud platform that hosts hundreds of large models behind one API Developer tools API integration Multi model platform Model deployment LLM API platform -
360智脑开放平台 360 model API platform API integration Multi model platform Knowledge base Q&A Enterprise automation LLM API platform -
4o Image API Image generation API gateway Free tools Freemium tools Developer tools API integration LLM API platform -
AI-Flow No-code multi-model workflows Workflow automation Developer tools API integration Knowledge base Q&A LLM API platform -
AI/ML API Single gateway to many models Developer tools API integration Multimodal Multi model platform LLM API platform -
Amazon Bedrock AWS code suggestions with security scans Free tools Code assistant IDE App builder LLM API platform