Skip to content

Aria-Base-64K

A base model tuned for long-context research

Visit official site
Category AI models

What Aria-Base-64K is

The model is described as a multimodal native Mixture-of-Experts base model formed after long-text pretraining, intended for research and continued training.

What you can do with it

  • Continue pretraining or finetune on long video QA data
  • Work on long document QA after short instruction tuning

Setup and limits

  • Install transformers, accelerate and sentencepiece to load it
  • Use AutoProcessor for text and image inputs before generation

Pros & cons

✓ What we like

  • Quick install and inference code examples are provided
  • A codebase supports advanced inference and custom dataset finetuning

! What to watch out for

  • The small chat template share may limit direct chat use
  • Loading needs transformers, accelerate and sentencepiece installed

FAQ

How much data was used?

The card states 33B tokens, with 21B multimodal, 12B language and 69% long text.

Is it ready for chat?

The card says only about 3% of data used the chat template format.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review