Skip to content

FlagPerf

An evaluation engine for AI hardware and software stacks

Visit official site

What FlagPerf is

The platform is described as an integrated AI hardware evaluation engine built jointly by a research institute and AI hardware vendors. The material says it sets up industry-oriented indicators and measures hardware under combinations of models, frameworks and compilers.

What you can do with it

  • Build indicators for performance, resource use and ecosystem fit
  • Run more than 30 classic models across vision and language tasks
  • Work with several training frameworks and inference engines
  • Check single-card, single-machine and multi-machine performance

Example uses

  • Measure how an A100 chip performs at NVIDIA
  • Bring in a Llama model for evaluation in a PaddlePaddle team
  • Check framework performance through a MindSpore team

Getting started

  • Install Docker and a Python environment
  • Check drivers, network and hardware virtualisation on the server
  • Download the project code and deploy it to the server
  • Edit the machine config with hardware and test environment settings

Pros & cons

✓ What we like

  • Several training frameworks and inference engines are supported
  • Submitted code is reviewed for fair and open testing

! What to watch out for

  • Measurements are tied to model, framework and compiler combinations
  • Results depend on the single-card, single-machine or multi-machine setup

FAQ

What does it evaluate?

The material describes AI hardware ability under software stack combinations of models, frameworks and compilers.

Which frameworks are named?

PyTorch and TensorFlow are listed, along with PaddlePaddle and MindSpore among domestic frameworks.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review