Category AI Model Evaluation
What MMStar is
The benchmark is described as a set for evaluating the multimodal ability of large vision language models.
What you can do with it
- Evaluate models on many tasks
- Detect leakage problems
- Guide model improvements
Benchmark design
- Review samples by hand
- Cover six core abilities
- Add leakage and gain metrics
Pros & cons
✓ What we like
- Two new metrics are proposed
- Sixteen models are stated as benchmarked
! What to watch out for
- Samples are stated to require visual dependence
- Leakage metrics rest on the sample design
FAQ
How many samples?
The material states 1500 curated samples.
What is added beyond accuracy?
The material states metrics for data leakage and multimodal gain.
Last reviewed: 2026-11-15
More LLM API platform tools
View all →-
302.AI Aggregated cloud platform that hosts hundreds of large models behind one API Developer tools API integration Multi model platform Model deployment LLM API platform -
360智脑开放平台 360 model API platform API integration Multi model platform Knowledge base Q&A Enterprise automation LLM API platform -
4o Image API Image generation API gateway Free tools Freemium tools Developer tools API integration LLM API platform -
AI-Flow No-code multi-model workflows Workflow automation Developer tools API integration Knowledge base Q&A LLM API platform -
AI/ML API Single gateway to many models Developer tools API integration Multimodal Multi model platform LLM API platform -
Amazon Bedrock AWS code suggestions with security scans Free tools Code assistant IDE App builder LLM API platform