Skip to content

LongRoPE

Extend an LLM context window to 2048k tokens

Visit official site
Category AI models

What longrope is

The technique is described as a Microsoft method that expands a pretrained model context window to 2048k tokens.

What you can do with it

  • Extend a short-context model to a long context
  • Support machine reading comprehension on long text

What to watch out for

  • The listed features are window extension, low-cost fine-tuning and short-context retention
  • Work starts from a pretrained large language model

Pros & cons

✓ What we like

  • Short-context performance is stated to be preserved
  • Training cost and time are described as reduced

! What to watch out for

  • The method starts from a pretrained language model
  • The named tasks centre on reading, summarisation and long-form text

FAQ

How large is the window?

The material states an extension to 2048k, or 2 million, tokens.

What is preserved?

The material states the original short-context window performance is kept.

Last reviewed: 2026-11-15

More LLM API platform tools

View all →

How we review