UNIMO-G
A conditional diffusion framework for interleaved text and visual prompts
What UNIMO-G is
The framework is described as a simple multimodal conditional diffusion framework for interleaved text and visual input, built from a multimodal large language model and a conditional denoising diffusion network.
What you can do with it
- Process interleaved text and visual input
- Generate images from encoded multimodal input
- Develop conditional image generation through pre-training
- Reach unified image generation through instruction tuning
How it is built
- A multimodal large language model encodes multimodal prompts
- A conditional denoising diffusion network generates the images
- A two-stage strategy combines pre-training with instruction tuning
- Language grounding and image segmentation build the multimodal prompts
Reference cases
- Generate high-fidelity images for complex prompts with several image entities
- Perform text-to-image generation
- Try zero-shot subject-driven synthesis
Pros & cons
✓ What we like
- Interleaved text and visual input can be processed together
- Unified image generation is the stated goal of the instruction tuning stage
! What to watch out for
- Training is split into pre-training and instruction tuning
- Multimodal prompts rely on language grounding and image segmentation in the data pipeline
FAQ
What are the two core components?
The material names a multimodal large language model for encoding prompts and a conditional denoising diffusion network for generating images.
How is it trained?
Pre-training on large-scale text-image pairs comes first, followed by instruction tuning with multimodal prompts.
Last reviewed: 2026-11-15
More AI image generation tools
View all →-
123RF Stock library with integrated AI image tools Free tools Text to image Image editing Background remover Image upscaler -
360智图 Image search, licensing and AI editing from 360 Free tools Text to image Background remover Product image Model generator -
360鸿图 Image generation aimed at Chinese-speaking users, with prompts written in Chinese Free tools Creator tools Text to image Art generator Image to image -
a1.art Filter-driven image generation with face swap Free tools Creator tools Text to image Art generator -
AI Anime Generator Photos and text into anime-style art Freemium tools Creator tools Text to image Image to image Image to video -
AI Art Generator Text prompts into original artwork drafts Freemium tools Creator tools Text to image Image editing Art generator