Skip to content

Hallo

Diffusion-based portrait image animation driven by audio

Visit official site

What Hallo is

The technique is described as a portrait image animation method developed at Fudan University that uses a diffusion model to create lifelike and dynamic portrait animation.

What you can do with it

  • Create lifelike portrait animation with a diffusion model
  • Align lips, expressions and pose motion with audio
  • Control expression and pose diversity adaptively
  • Personalise animation for different identities

How it works

  • An end-to-end diffusion paradigm replaces parametric face representations
  • A UNet-based denoiser improves image quality
  • Time alignment keeps animation in sync with audio
  • A reference network sharpens facial motion precision

Application scenarios

  • Film production generates realistic facial expressions and lip shapes
  • Game development gives virtual characters dynamic facial animation
  • Virtual reality adds realistic facial feedback to user interaction

Pros & cons

✓ What we like

  • A UNet-based denoiser is used for image quality
  • A reference network improves facial motion precision

! What to watch out for

  • The documented starting point is the product page and its technical details
  • Lip, expression and pose alignment is tied to the audio input

FAQ

Who developed it?

The material names Fudan University as the developer.

What does the synthesis module align?

The material lists lips, expressions and pose motion against audio input.

Last reviewed: 2026-11-15

More AI image generation tools

View all →

How we review