Skip to content

ELLA

Equip diffusion models with an LLM for longer prompts

Visit official site

What ELLA is

The adapter is described as a lightweight method (Efficient Large Language Model Adapter) that equips CLIP-based diffusion models with a large language model so text-to-image models can follow prompts and understand long text. The material also describes a temporal-aware semantic connector that takes timestep-related conditions from a pre-trained LLM across denoising stages.

What you can do with it

  • Improve prompt following of a CLIP-based diffusion model without training U-Net or the LLM
  • Equip existing diffusion models with an LLM for better text alignment
  • Understand long text and dense prompts with multiple objects, attributes and relations
  • Integrate with community models and downstream tools such as LoRA and ControlNet

Application scenarios

  • A social media platform tunes prompt alignment for automatically generated images
  • Researchers generate images from complex articles with stronger prompt understanding
  • Designers turn detailed descriptions into images with closer text-to-image alignment

Model notes

  • A temporal-aware semantic connector extracts timestep-related conditions from a pre-trained LLM
  • The connector adapts semantic features at different sampling timesteps for a frozen U-Net
  • Dense Prompt Graph Benchmark (DPG-Bench) evaluates dense prompt following of text-to-image models

Pros & cons

✓ What we like

  • Improves prompt following on long text and dense prompts
  • Requires no training of U-Net or the LLM

! What to watch out for

  • The adapter targets existing CLIP-based diffusion models
  • Timestep conditions come from a pre-trained LLM through the connector

FAQ

Does it need to train U-Net or the LLM?

The material states prompt following improves without training U-Net or the LLM.

What is DPG-Bench?

The material describes the Dense Prompt Graph Benchmark for evaluating dense prompt following of text-to-image models.

Last reviewed: 2026-11-15

More AI image generation tools

View all →

How we review