Skip to content

contrastors

Train and evaluate contrastive models efficiently

Visit official site

What contrastors is

The toolkit is described as a contrastive learning package for researchers and engineers to train and evaluate contrastive models efficiently.

What you can do with it

  • Train models across multiple GPUs
  • Run large batches with GradCache support
  • Pretrain with masked language modelling
  • Load common models through Huggingface

Example uses

  • Start BERT training from zero on 8 GPUs
  • Refine the nomic-bert-embed-v1-unsupervised model
  • Create a dataset with the bundled scripts

Pros & cons

✓ What we like

  • Masked language model pretraining is supported
  • Matryoshka representation learning is supported

! What to watch out for

  • Training runs within the Pytorch framework
  • Large batch training relies on GradCache

FAQ

What is it built on?

The material states it builds on Flash Attention with multi-GPU support.

Which pretraining is supported?

The features list names masked language model pretraining and Matryoshka representation learning.

Last reviewed: 2026-11-15

More AI coding tools tools

View all →

How we review