Skip to content

OpenDiLoCo

Train AI models across machines with low communication

Visit official site

What OpenDiLoCo is

The framework is described as an open-source implementation of DeepMind distributed low-communication training, built to make AI model training work across scattered resources.

What you can do with it

  • Run distributed training across global nodes
  • Sync communication with the Hivemind library
  • Scale a DiLoCo worker through PyTorch FSDP

Getting started

  • Secure at least two GPUs in any location
  • Create the initial DHT node with the given command
  • Start a worker in another terminal with the environment variables

Pros & cons

✓ What we like

  • Training across two continents and three countries kept 90-95% compute utilisation
  • Fault-tolerant training works on varied hardware setups

! What to watch out for

  • Setup needs access to at least two GPUs
  • Workers are started with PEER, NUM_DILOCO_WORKERS and WORLD_RANK variables

FAQ

What does it implement?

The material states it implements and scales the DeepMind DiLoCo method.

What hardware is required?

The steps say at least two GPUs, which need not be in the same place.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review