What LMSYS is
The organisation is described as working to democratise large model technology and the systems infrastructure around it. Its work spans chatbots, evaluation arenas, datasets, runtimes and open platforms.
What you can do with it
- Develop large models and their systems infrastructure
- Train, serve and evaluate LLM-based chatbots with FastChat
- Run complex LLM programs with the SGLang interface and runtime
- Evaluate models at scale through crowdsourced battles
Projects and resources
- Vicuna is described at 7B, 13B and 33B scales, with the material citing 90% of ChatGPT quality
- Chatbot Arena combines crowdsourcing with an Elo rating system
- LMSYS-Chat-1M is a large real-world LLM conversation dataset
- MT-Bench is a set of challenging multi-turn open-ended questions
Example uses
- Build a high quality chatbot application with Vicuna
- Hold an LLM evaluation competition on Chatbot Arena
- Train and serve a chatbot with FastChat
Pros & cons
✓ What we like
- Models, datasets and evaluation tools in one organisation
- FastChat covers training, serving and evaluation
! What to watch out for
- Arena evaluation is based on crowdsourcing and Elo ratings
- MT-Bench relies on multi-turn open-ended questions
FAQ
What is Chatbot Arena?
The material describes large-scale, gamified LLM evaluation through crowdsourcing and an Elo rating system.
Which projects are listed?
Vicuna, Chatbot Arena, SGLang, LMSYS-Chat-1M, FastChat and MT-Bench are named.
Last reviewed: 2026-11-15
More AI coding tools tools
View all →-
100 Vibe Coding Learn vibe coding by building Free tools Enterprise tools Developer tools API integration Coding -
10Web Prompt to WordPress site with hosting Free tools Developer tools No code development Coding Website builder -
16x Prompt Context management for AI coding Free tools Workflow automation Developer tools API integration Coding -
a0.dev Chat-built mobile apps to store Free tools Developer tools No code development Coding App builder -
Adalo No-code apps with a visual canvas Free tools Developer tools No code development Coding App builder -
AgentQL Natural language web data extraction Freemium tools Developer tools Browser extension Data extraction Coding