Skip to content

Unified-IO 2

One model for image, text, audio and action tasks

Visit official site

What Unified-IO 2 is

The model is described as a unified multimodal generation model that handles images, text, audio and actions inside one encoder-decoder Transformer.

What you can do with it

  • Caption an image
  • Follow a free-form instruction
  • Edit an image or detect objects

Model notes

  • Inputs and outputs share a semantic space
  • Pre-training used a large multimodal corpus
  • Fine-tuning covered 120 existing datasets

Pros & cons

✓ What we like

  • One model spans four modalities
  • Training used a large multimodal corpus from scratch

! What to watch out for

  • Fine-tuning drew on 120 existing datasets
  • The described design is a single encoder-decoder Transformer

FAQ

Which modalities are covered?

The material names images, text, audio and actions.

How many datasets were used for fine-tuning?

The material states 120 existing datasets.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review