Skip to content

UNIMO-G

A conditional diffusion framework for interleaved text and visual prompts

Visit official site

What UNIMO-G is

The framework is described as a simple multimodal conditional diffusion framework for interleaved text and visual input, built from a multimodal large language model and a conditional denoising diffusion network.

What you can do with it

  • Process interleaved text and visual input
  • Generate images from encoded multimodal input
  • Develop conditional image generation through pre-training
  • Reach unified image generation through instruction tuning

How it is built

  • A multimodal large language model encodes multimodal prompts
  • A conditional denoising diffusion network generates the images
  • A two-stage strategy combines pre-training with instruction tuning
  • Language grounding and image segmentation build the multimodal prompts

Reference cases

  • Generate high-fidelity images for complex prompts with several image entities
  • Perform text-to-image generation
  • Try zero-shot subject-driven synthesis

Pros & cons

✓ What we like

  • Interleaved text and visual input can be processed together
  • Unified image generation is the stated goal of the instruction tuning stage

! What to watch out for

  • Training is split into pre-training and instruction tuning
  • Multimodal prompts rely on language grounding and image segmentation in the data pipeline

FAQ

What are the two core components?

The material names a multimodal large language model for encoding prompts and a conditional denoising diffusion network for generating images.

How is it trained?

Pre-training on large-scale text-image pairs comes first, followed by instruction tuning with multimodal prompts.

Last reviewed: 2026-11-15

More AI image generation tools

View all →

How we review