Skip to content

WebLLM

Run language models inside the browser

Visit official site

What WebLLM is

The project is described as an in-browser inference engine that uses WebGPU to run language models on the client, aiming at lower cost, more personalisation and better privacy.

What you can do with it

  • Run language model operations inside a browser
  • Use OpenAI-compatible JSON mode, function calls and streaming
  • Add custom models in the MLC format

Getting started

  • Visit the site and read the integration guide
  • Choose a language model to integrate
  • Add the project through NPM, Yarn or CDN

Pros & cons

✓ What we like

  • Llama, Phi, Gemma, RedPajama, Mistral and Qwen are supported natively
  • NPM, Yarn or CDN integration is provided

! What to watch out for

  • Model work is described as running with WebGPU acceleration
  • Custom models use the MLC format

FAQ

Does it need a server?

The material states inference runs in the browser without server-side processing.

How is it added to a project?

The material names NPM, Yarn and CDN.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review