Skip to content

PowerInfer-2

Run large MoE models on a smartphone with less memory

Visit official site

What PowerInfer-2 is

The framework is an inference framework optimised for smartphones, using heterogeneous computing and an I/O-Compute pipeline to cut memory use.

What you can do with it

  • Run MoE models with up to 47B parameters
  • Adjust compute unit size dynamically

What to watch out for

  • The described target device is a smartphone
  • Model sparsity is listed as a configuration step

Pros & cons

✓ What we like

  • Memory use is stated to drop significantly
  • On-device deployment is described as improving data privacy

! What to watch out for

  • The described target device is a smartphone
  • Model sparsity is listed as a configuration step

FAQ

What speed is stated?

11.68 tokens per second, 22 times faster than other frameworks.

What model size is supported?

MoE models with up to 47B parameters.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review