Skip to content

TAG-Bench

A benchmark for database question answering

Visit official site

What TAG-Bench is

The benchmark is described as a way to evaluate and study NLP models answering database queries. It builds on BIRD Text2SQL and adds requirements for world knowledge or semantic reasoning.

What you can do with it

  • Measure a model on complex database queries
  • Optimise a query handling system
  • Demonstrate database question answering in class

Getting started

  • Create a conda environment and install dependencies
  • Download the BIRD database and convert it to Pandas DataFrames
  • Create table indexes and edit the query file

Pros & cons

✓ What we like

  • Pandas DataFrames can simulate the database setting
  • A detailed setup guide is provided

! What to watch out for

  • A GPU is recommended for creating table indexes
  • Queries may need knowledge beyond the stored data

FAQ

How many queries are included?

The material states 80 complex queries built on BIRD Text2SQL.

What methods are supported?

Hand-written TAG, Text2SQL, Text2SQL+LM, RAG and retrieval plus LM ranking are listed.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review