You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
site: feature Figures 1, 10, 11 in a Highlights showcase at the top
Add a Highlights section right after the hero showing the best results — Fig 1
(workflow + agent-generated Pareto), Fig 10 (agent-generated algorithms across
RULER/AMC23/AIME24), and Fig 11 (long-horizon autonomous optimization). Move
those figures out of Vision/Design/Experiments to avoid duplication (those
sections keep their prose + trios), add a 'Results' nav link, and copy the new
figure assets in the docs workflow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
<h2class="sec-title reveal">Agents discover sparse attention — and it actually runs faster</h2>
258
+
<pclass="lead reveal">AI agents generate and refine diverse sparse-attention algorithms with Vortex, and every one is benchmarked <strong>end-to-end in a real serving stack</strong>. The headline results:</p>
259
+
260
+
<figureclass="fig show reveal">
261
+
<divclass="figrow">
262
+
<imgsrc="assets/fig1_workflow.png" alt="The Vortex workflow: papers and agents to vFlow, vTensor, and a serving system" style="width:40%" />
263
+
<imgsrc="assets/fig1_results.png" alt="Agent-generated sparse attention on the accuracy–throughput plane" style="width:54%" />
264
+
</div>
265
+
<figcaption><b>(a)</b> A workflow to study sparse attention with Vortex. <b>(b)</b> Agent-generated sparse attention (Qwen3-1.7B, AIME, NVIDIA H200) — the best reaches up to <strong>3.46×</strong> the throughput of full attention while preserving accuracy.</figcaption>
266
+
</figure>
267
+
268
+
<figureclass="fig show reveal">
269
+
<divclass="shots">
270
+
<imgsrc="assets/fig10_inno_a.png" alt="Agent-generated algorithms on RULER" />
271
+
<imgsrc="assets/fig10_inno_b.png" alt="Agent-generated algorithms on AMC23" />
272
+
<imgsrc="assets/fig10_inno_c.png" alt="Agent-generated algorithms on AIME24" />
273
+
</div>
274
+
<figcaption>Performance of AI-agent-generated algorithms across <b>(a)</b> RULER, <b>(b)</b> AMC23, and <b>(c)</b> AIME24 — Claude Opus 4.7, Claude Sonnet 4.6, and GPT-5 all produce Pareto-efficient designs with full-attention accuracy and substantially higher throughput.</figcaption>
275
+
</figure>
276
+
277
+
<figureclass="fig show reveal">
278
+
<divclass="shots">
279
+
<imgsrc="assets/fig11_iter_a.png" alt="mean@16 per iteration" />
280
+
<imgsrc="assets/fig11_iter_b.png" alt="throughput per iteration" />
281
+
<imgsrc="assets/fig11_iter_c.png" alt="accuracy–throughput frontier colored by iteration" />
282
+
</div>
283
+
<figcaption>Long-horizon autonomous optimization on AIME24 (23 iterations, 92 submissions): <b>(a)</b> mean@16 per iteration, <b>(b)</b> throughput per iteration, <b>(c)</b> the accuracy–throughput frontier of all submissions, colored by iteration order.</figcaption>
<h2class="sec-title reveal">Agents as algorithm designers</h2>
248
291
<pclass="lead reveal">Sparse attention has become a fundamental technique for serving large language models. As generation lengths explode across reasoning, agentic systems, and reinforcement learning, <strong>moving the KV cache during decoding</strong> — not compute — is the dominant bottleneck. Attending to only the tokens that matter is the way out, and it now appears both as a core architectural choice in frontier models (DeepSeek, GLM) and as a drop-in optimization for pretrained ones.</p>
249
-
<divclass="split reveal">
250
-
<divclass="prose">
251
-
<p>Yet deploying and evaluating <strong>new</strong> sparse-attention algorithms at scale, with real end-to-end speedups, has stayed painfully engineering-intensive — slowing both human researchers and the emerging class of AI agents that could explore this design space. Modern serving systems store the KV cache in a non-contiguous, paged, block-sparse layout reached through indirect addressing, which breaks the contiguous-tensor assumptions of frameworks like PyTorch. As a result, a new idea that is a few lines of math on paper can take thousands of lines of kernel and plumbing code to try.</p>
252
-
<p>Vortex removes that wall. It is built for <strong>autonomous algorithm discovery</strong>: an AI agent proposes a sparse-attention idea, expresses it in a few lines of high-level Python, and Vortex compiles it into fused kernels that run inside a production serving stack — then measures real throughput and accuracy. The agent reads the result and refines, closing the research loop without a human in the inner iteration.</p>
253
-
<p>Across hundreds of generated variants, agents consistently discover Pareto-efficient algorithms: full-attention–level accuracy at a fraction of the cost. The best reaches up to <strong>3.46× higher throughput</strong> than full attention while preserving accuracy — and because every variant is benchmarked in a real serving stack, those gains are measured, not theoretical.</p>
254
-
</div>
255
-
<figureclass="fig">
256
-
<imgsrc="assets/fig1_results.png" alt="Agent-generated sparse attention on the accuracy–throughput plane" />
257
-
<figcaption>Each point is one sparse-attention algorithm generated or optimized by AI agents using Vortex (Qwen3-1.7B, AIME, NVIDIA H200).</figcaption>
258
-
</figure>
292
+
<divclass="prose reveal" style="margin-top:18px">
293
+
<p>Yet deploying and evaluating <strong>new</strong> sparse-attention algorithms at scale, with real end-to-end speedups, has stayed painfully engineering-intensive — slowing both human researchers and the emerging class of AI agents that could explore this design space. Modern serving systems store the KV cache in a non-contiguous, paged, block-sparse layout reached through indirect addressing, which breaks the contiguous-tensor assumptions of frameworks like PyTorch. As a result, a new idea that is a few lines of math on paper can take thousands of lines of kernel and plumbing code to try.</p>
294
+
<p>Vortex removes that wall. It is built for <strong>autonomous algorithm discovery</strong>: an AI agent proposes a sparse-attention idea, expresses it in a few lines of high-level Python, and Vortex compiles it into fused kernels that run inside a production serving stack — then measures real throughput and accuracy. The agent reads the result and refines, closing the research loop without a human in the inner iteration.</p>
295
+
<p>Across hundreds of generated variants, agents consistently discover Pareto-efficient algorithms: full-attention–level accuracy at a fraction of the cost. The best reaches up to <strong>3.46× higher throughput</strong> than full attention while preserving accuracy — and because every variant is benchmarked in a real serving stack, those gains are measured, not theoretical.</p>
259
296
</div>
260
297
<pclass="lead reveal" style="margin-top:34px">It runs as a closed <strong>discovery loop</strong>:</p>
261
298
<divclass="pipeline reveal">
@@ -274,16 +311,10 @@ <h2 class="sec-title reveal">Agents as algorithm designers</h2>
<h2class="sec-title reveal">A frontend for ideas, a backend for serving</h2>
276
313
<pclass="lead reveal">Vortex pairs a <strong>Python-embedded frontend</strong> over a <strong>page-centric tensor abstraction</strong> (<code>vTensor</code>) — concise enough to express a broad range of sparse-attention algorithms — with an efficient backend tightly integrated into modern LLM serving stacks (SGLang). The guiding principle is a clean split: you say <em>what</em> sparsity to apply and <em>how</em> attention is computed, and the framework owns the low-level tensor layout and memory management. Theoretical efficiency becomes real-world throughput, without touching core model code.</p>
277
-
<divclass="split rev reveal">
278
-
<figureclass="fig">
279
-
<imgsrc="assets/fig1_workflow.png" alt="The Vortex workflow: papers and agents to vFlow, vTensor, and a serving system" />
280
-
<figcaption>From idea to serving: a <code>vFlow</code> program over <code>vTensor</code> pages compiles straight into the serving system.</figcaption>
281
-
</figure>
282
-
<divclass="prose">
283
-
<p>You describe <em>what</em> to attend to — score pages, reduce per-block summaries, select a top-<em>k</em> — and Vortex handles batching, paged KV caching, gather minimization, and kernel fusion. A flow is written as <strong>modular, composable operators</strong> (<code>GeMM</code>, <code>Reduce</code>, <code>Top-K</code>, …) over paged tensors, rather than a monolithic custom kernel per algorithm — so new patterns combine instead of being reimplemented from scratch.</p>
284
-
<p>Crucially, Vortex treats the <strong>dynamic part</strong> — deciding the sparsity pattern on the fly — as a first-class, optimized stage, not an afterthought. The same abstraction covers MHA and <strong>MLA</strong> models, exact and approximate top-<em>k</em>, and even a programmable per-sequence token budget written as a small CUDA snippet.</p>
285
-
<p>The result drops into FlashInfer / CUDA Graph / radix-cache decoding — so a new algorithm is deployable and benchmarkable the moment it compiles, and its speedups survive contact with a real serving system.</p>
286
-
</div>
314
+
<divclass="prose reveal" style="margin-top:18px">
315
+
<p>You describe <em>what</em> to attend to — score pages, reduce per-block summaries, select a top-<em>k</em> — and Vortex handles batching, paged KV caching, gather minimization, and kernel fusion. A flow is written as <strong>modular, composable operators</strong> (<code>GeMM</code>, <code>Reduce</code>, <code>Top-K</code>, …) over paged tensors, rather than a monolithic custom kernel per algorithm — so new patterns combine instead of being reimplemented from scratch.</p>
316
+
<p>Crucially, Vortex treats the <strong>dynamic part</strong> — deciding the sparsity pattern on the fly — as a first-class, optimized stage, not an afterthought. The same abstraction covers MHA and <strong>MLA</strong> models, exact and approximate top-<em>k</em>, and even a programmable per-sequence token budget written as a small CUDA snippet.</p>
317
+
<p>The result drops into FlashInfer / CUDA Graph / radix-cache decoding — so a new algorithm is deployable and benchmarkable the moment it compiles, and its speedups survive contact with a real serving system.</p>
287
318
</div>
288
319
<pclass="lead reveal" style="margin-top:34px">Under the hood, a flow flows through <strong>three composable layers</strong>:</p>
<divclass="blk reveal"><spanclass="idx">②</span><h3>Scale & architectures</h3><p>Sparse attention extended to MLA models and very large models that are otherwise hard to experiment with.</p><spanclass="metric">4.7× GLM-4.7 · 1.37× 229B</span></div>
310
341
<divclass="blk reveal"><spanclass="idx">③</span><h3>Research instrument</h3><p>A lens on sparse attention itself — pinpointing where the routing signal lives.</p><spanclass="metric">interpretability</span></div>
311
342
</div>
343
+
<pclass="lead reveal" style="margin-top:8px">Autonomous discovery is shown in the <ahref="#highlights">Highlights</a> above; here we focus on reach — new architectures and the largest models:</p>
312
344
<divclass="gallery">
313
-
<articleclass="exp reveal">
314
-
<divclass="imgwrap"><imgsrc="assets/fig11_iter_c.png" alt="Accuracy–throughput frontier across 23 autonomous iterations" /></div>
315
-
<divclass="body">
316
-
<divclass="tag">Autonomous optimization</div>
317
-
<h3>23 iterations, 92 submissions</h3>
318
-
<p>Given only the framework and a goal, a Claude agent runs a long-horizon loop on AIME24 (Qwen3-1.7B), generating four variants per round and pushing the accuracy–throughput frontier steadily outward — each point colored by iteration order.</p>
319
-
</div>
320
-
</article>
321
345
<articleclass="exp reveal">
322
346
<divclass="imgwrap"><imgsrc="assets/fig12_glm_a.png" alt="Sparse attention on the MLA-based GLM-4.7-Flash" /></div>
0 commit comments