Jet-Long extends language model context windows to 128K without fine-tuning

Jet-Long is a method for extending the context window of language models without fine-tuning, published on arXiv on July 10, 2026. In tests on Qwen3 with context up to 128K tokens, the method outperformed all existing zero-shot baselines in accuracy, and prefill-throughput on GPU H100 accelerated to 1.39× relative to FlashAttention 2.

Why extending context is such a problem

Most open LLMs are trained with context windows of 4K–32K tokens, while real tasks — RAG, agent systems with accumulated tool traces, analyzing entire repositories — regularly require ten times more. Fine-tuning each checkpoint for long context is expensive: you need long training examples, additional GPU memory and processing time. This is why zero-shot RoPE scaling methods have become the standard deployment path.

RoPE (Rotary Position Embedding) is the standard way to encode token positions in transformers. During inference beyond the training window, position vectors fall into ranges the model has never seen. Rescaling "compresses" positions back into familiar values, but with unavoidable tradeoffs: an aggressive coefficient breaks quality on short inputs, a conservative one fails with truly long sequences. The optimal value depends on the length of the specific request at inference time, and no constant can capture it.

How Jet-Long works

The authors propose Dynamic Bifocal RoPE — two parallel processing modes for position embeddings within a single attention layer.

• Local window — preserves standard RoPE frequencies for short positions so the model behaves exactly as it did during training
• Far window — scales frequencies dynamically based on current sequence length, not a predetermined constant
• Inclusion-exclusion merge — combines outputs of both windows without double-counting tokens
• On-the-fly RoPE correction rotation — eliminates phase shift during merge

The entire construction is implemented in a single CuTe kernel for CUDA, making bifocal attention practically free: no additional GPU pass. Prefill accelerates to 1.39× from FlashAttention 2 on H100 — approaching FlashAttention 4, which requires Hopper chips and is unavailable on older GPUs. Overhead during generation does not exceed 4% at any context length.

What results did Jet-Long show on 128K context?

The method was tested on the Qwen3 family — 1.7B, 4B and 8B parameters — with context up to 128K tokens.

• RULER: +4.79 pp for Qwen3-1.7B, +2.18 pp for 4B, +2.03 pp for 8B relative to the best competing method
• HELMET-RAG: best overall result among all compared approaches (HELMET authors highlighted this benchmark as the most accurate predictor of real downstream performance)
• PG-19 perplexity: minimum among all tested methods

Jet-Long was also generalized to hybrid Jet-Nemotron architecture with alternating dense and sparse attention layers — and achieved additional gains without retraining. The authors emphasize that the method requires no manual hyperparameter tuning and works "out of the box."

What this means

Jet-Long offers a practical way to extend the context of open-weight models without fine-tuning, special GPUs and manual hyperparameters. If the method reproduces on other popular architectures — Llama, Mistral, Gemma — it could become a standard tool in the inference stack for long-context tasks: agent systems, document analysis and code review at the repository level.

Frequently asked questions

Do I need to fine-tune the model for Jet-Long?

No. Jet-Long is a tuning-free zero-shot method: it is enough to connect it to an existing checkpoint without any additional training runs.

Which models was Jet-Long tested on?

In the preprint from July 10, 2026, Qwen3-1.7B, Qwen3-4B, Qwen3-8B and hybrid Jet-Nemotron architecture were tested. Testing on Llama, Mistral or other families is not provided in the paper.