Skip to content

OneGen

Unify generation and retrieval in one forward pass

Visit official site
Category AI models

What OneGen is

OneGen is described as an efficient single-pass generation and retrieval framework for large language models. It merges generation and retrieval into the same context so an LLM performs both tasks in one forward pass.

What you can do with it

  • Fine-tune a model for generation, retrieval or mixed tasks
  • Run retrieval through autoregressively generated retrieval tokens
  • Download pre-trained models or train a model from scratch
  • Evaluate model performance with the provided scripts

Application scenarios

  • Identify entities in text through a pre-trained model
  • Generate answers for single-hop question answering
  • Reach answers by reasoning in multi-hop question answering

Getting started

  • Clone the OneGen repository to a local environment
  • Create and activate a Python virtual environment
  • Install the required dependency packages
  • Download and unpack the datasets for training or inference

Pros & cons

✓ What we like

  • Deployment cost is lowered by unified processing
  • No second forward pass is needed for the query

! What to watch out for

  • Setup requires cloning the repo and a Python virtual environment
  • Datasets must be downloaded and unpacked before training or inference

FAQ

Which tasks are named?

The material lists entity linking, single-hop question answering and multi-hop question answering.

How does it reduce inference cost?

Retrieval runs in the same forward pass as generation, so the query is not computed twice.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review