Skip to content

Motion-I2V

A two-stage framework for controllable image-to-video generation

Visit official site

What Motion-I2V is

The framework is described as a consistent and controllable image-to-video generation approach that splits I2V into two stages and uses explicit motion modelling instead of learning a direct image-to-video mapping.

What you can do with it

  • Generate consistent and controllable image-to-video output
  • Predict motion fields with a diffusion-based predictor
  • Propagate reference image features into synthesised frames
  • Control motion trajectories and motion regions with sparse trajectories

Two-stage design

  • The first stage uses a diffusion-based motion field predictor to infer pixel trajectories of the reference image
  • The second stage uses motion-augmented temporal attention to strengthen limited one-dimensional temporal attention in a video latent diffusion model
  • Predicted trajectories guide the propagation of reference image features into synthesised frames
  • Videos are stated to stay consistent even with large motion and view changes

Application scenarios

  • A photography enthusiast turns a set of landscape photos into coherent controllable video work
  • An advertising company produces a product video with specific visual effects
  • An artist turns static artwork into a video presentation with motion

Pros & cons

✓ What we like

  • Motion trajectories and motion regions can be controlled precisely
  • Videos are stated to stay consistent under large motion and view changes

! What to watch out for

  • The approach splits image-to-video generation into two stages
  • Motion control is described through sparse trajectory control network training

FAQ

What does the framework do?

The material describes a framework for consistent and controllable image-to-video generation.

How is motion handled?

A diffusion-based motion field predictor infers pixel trajectories in the first stage, and motion-augmented temporal attention is used in the second.

Last reviewed: 2026-11-15

More AI video generation tools

View all →

How we review