Skip to content

DIG-In

Audit geographic differences in text-to-image models

Visit official site

What DIG-In is

DIG-In is described as a library for evaluating differences in quality, diversity and consistency of text-to-image generation models across geographic regions. The library supports researchers and developers auditing the geographic diversity of image generation models.

What you can do with it

  • Compare generated image quality with GeoDE and DollarStreet
  • Compute precision, recall, coverage and density metrics
  • Measure image consistency with the CLIPScore metric
  • Run scripts that extract features from generated images

Application scenarios

  • Researchers compare model output quality in African regions
  • Developers check consistent user experience across the globe
  • Educational institutions teach evaluation of model fairness

Getting started

  • Generate images that correspond to prompts in a csv file
  • Point to the prompt csv and the generated image folder to extract features
  • Compute metrics such as precision, recall, coverage and density from the features
  • Update feature file paths as needed

Pros & cons

✓ What we like

  • Scripts are provided to extract features from generated images
  • Pointers to custom image or feature paths are supported

! What to watch out for

  • The audit is framed around text-to-image generation models
  • Metrics depend on features extracted from generated images

FAQ

Which datasets are used?

The material names GeoDE and DollarStreet as reference datasets.

What metrics are computed?

The feature list names precision, recall, coverage and density, plus CLIPScore for consistency.

Last reviewed: 2026-11-15

More AI image generation tools

View all →

How we review