Skip to content

Promptfoo

Test and compare LLM prompts with a test-case library

Visit official site

What Promptfoo is

The library is described as a tool for evaluating LLM prompt quality and running tests, letting users create test cases, set evaluation metrics and integrate with existing test and CI workflows. A Web Viewer is also provided to compare different prompts and model outputs.

What you can do with it

  • Create lists of test cases
  • Use built-in evaluation metrics or custom metrics
  • Compare different prompts and model outputs
  • Integrate with existing test and CI workflows

Example uses

  • Evaluate whether LLM-generated text is accurate
  • Test the effect of different prompts on model output
  • Compare different models under the same prompt

Pros & cons

✓ What we like

  • Works with existing test and CI workflows
  • Metrics can be built-in or custom

! What to watch out for

  • Evaluation starts from test cases and metrics you define
  • Comparisons are made between prompts and model outputs

FAQ

What does it evaluate?

The material states it evaluates LLM prompt quality and tests prompt and model outputs.

Does it fit an existing pipeline?

The material says it integrates with existing test and CI workflows.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review