Skip to content

UniMuMo

One model bridging text, music and motion

Visit official site

What UniMuMo is

The model is described as a multimodal system that bridges text, music and motion through token-based representations.

What you can do with it

  • Use text, music or motion as an input condition
  • Generate outputs across all three modalities
  • Align unpaired data by rhythm patterns

Application scenarios

  • Generate music and dance from a text description
  • Create synced music and motion for game characters
  • Produce natural motion for virtual characters

Pros & cons

✓ What we like

  • Three modalities share one architecture
  • Competitive results are reported on one-way benchmarks

! What to watch out for

  • Outputs depend on the chosen input modality
  • Alignment relies on rhythm patterns in the data

FAQ

Which modalities are covered?

Text, music and motion are named as inputs and outputs.

How is compute lowered?

The material states fine-tuning of existing pretrained single-modality models.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review