feat(inference): add DeepSeek V4 Pro model architecture (#432)

* feat(inference): add DeepSeek V4 Pro model architecture

Add a MODEL_ARCHITECTURES entry for DeepSeek V4 Pro (1.6T/49B MoE, 61
layers, 1M context) following the existing per-model pattern. Attention
is modeled as a Hybrid stack of two interleaved compressed variants —
Heavily Compressed Attention (31 layers) and Compressed Sparse Attention
(30 layers) — each carrying a 128-token sliding-window branch and a
learnable attention sink.

Surface sliding-window attention on both alternating blocks: the diagram's
window note now derives from a per-spec slidingWindow field instead of the
block index, so hybrid models show window=128 on every attention variant
(gpt-oss behavior preserved). The specs-bar attention cell now derives from
attentionType so it reads "Hybrid" instead of the hardcoded "Sink/Full GQA".

Sourced from deepseek-ai/DeepSeek-V4-Pro (config.json, inference/model.py,
DeepSeek_V4.pdf).

* feat(inference): surface SWA as an explicit block; fix residual + glyph centering

DeepSeek V4 hybrid attention now drills down: expanding the CSA/HCA
attention block reveals the sliding-window branch as its own block
alongside the compressed branch (lightning indexer for CSA, heavy
compression for HCA), converging at shared-KV MQA + sink + output
projection. Gated to attentionType === 'Hybrid' so gpt-oss is unchanged.

Also fixes two diagram issues affecting all models:
- Residual bypass tapped at the RMSNorm's top edge, so its horizontal
  connector ran across the norm block. Tap from the arrow gap above the
  norm instead.
- Circle glyphs (+, ×, −) rendered off-center because
  dominant-baseline: central is unreliable (Safari falls back to the
  alphabetic baseline). Use dy=0.35em, which centers consistently across
  browsers and matches central where it already worked.

* fix(inference): balance hybrid attention drill-down, center circle glyphs

Addresses three rendering issues in the model architecture diagram:

- Hybrid (CSA/HCA) attention now drills down into symmetric 2x2 columns:
  Local (Sliding Window + Attention Sink) beside the two-stage Compressed
  branch (compression + selector). This removes the lonely long connector
  that made the expanded box look unbalanced and promotes the attention
  sink to an explicit block; the merge block is now plain "Shared-KV MQA".

- The +, -, and x symbols inside merge/expand circles are drawn as
  geometric strokes instead of <text>. Font baseline drift (even with dy
  tuning) left the glyph sitting slightly low, which also made the residual
  bypass line read as misaligned with the "+". The strokes are centered on
  the circle's center, so the residual line is now co-linear with the arm.

* fix(inference): represent the hybrid attention sink accurately

The sliding-window and compressed branches are two KV *sources* whose
selected indices are unioned into a single shared-KV MQA softmax — not two
attentions merged after the fact. The attention sink is a per-head learnable
softmax-denominator bias on that MQA (model.py attn_sink / kernel.py
sum_exp += exp(attn_sink - max)), not literal "first tokens" in the local
branch.

- Local branch is just the sliding-window source (one block); the sink moves
  back onto the merge block as "Shared-KV MQA + Sink".
- CSA compressed branch = Token Compression -> Lightning Indexer (2 stages);
  HCA = a single Heavy Compression source. This makes CSA a 1-vs-2 split
  again.
- Center each column within the shared column area in drawParallelFlow so an
  unequal split reads as an intentional branch merge instead of leaving the
  shorter column's connector dangling as a long unattached line. Also
  improves the 2-vs-1 SwiGLU expert merge.

* feat(inference): caption hybrid attention drill-down as one fused softmax

When the DeepSeek V4 hybrid attention drill-down is expanded, show a short
note clarifying that the Local (sliding-window) and Compressed (CSA/HCA)
columns are two KV *sources* unioned into a single shared-KV MQA softmax —
not two separate attentions that get summed — with the attention sink being
a learnable per-head softmax-denominator bias. Prevents the parallel-column
schematic from being read as two independent attention paths.

Shown only while a hybrid attention block is expanded; covered by an e2e
assertion.

* fix(inference): rename SWA feature label so it isn't read as a separate attention type

The 128-token sliding window is the shared local base of every hybrid layer
(both HCA and CSA extend it; the final layer runs it alone) — not a third
attention type alongside CSA/HCA. Rename the features badge from
"Sliding Window Attention (128 tokens)" to "Sliding window (128 tokens)" so it
reads as a windowing mechanism rather than a standalone attention. The
drill-down's "Sliding Window" KV-source block is unchanged.

* feat(inference): draw hash-routed prefix block and mHC hyper-connections for V4

Two DeepSeek V4 architectural facts were only feature badges; surface them in
the diagram structure.

Hash-routed layers (num_hash_layers=3): the first 3 MoE layers route by token
id, not a learned gate. They now render as a separate stacked prefix block
(between embedding and the alternating blocks) with a "Hash Router" instead of
"MoE Router". Alternating HCA/CSA counts drop 31/30 → 29/29 so they describe the
learned-router layers (3 + 29 + 29 = 61); drawExpertGrid gains optional
routerLabel / routerSub params.

mHC (hc_mult=4): residuals are replaced by 4 parallel hyper-connection streams
with learned, Sinkhorn-normalized A/B/C mixing. Residual merges now render as an
"mHC ×N" mixer node instead of a plain "+" when arch.hyperConnections > 1, plus
a caption shown while a block exposing the nodes is expanded. Models without
hyper-connections keep the "+" residual.

Adds arch fields hashRoutedLayers and hyperConnections; unit + e2e coverage.

* fix(inference): connect arrow into RMSNorm and gap attention drill-down

Two diagram glitches in expanded transformer blocks:

- The incoming arrow stopped at the dashed container border, leaving no line
  above the first RMSNorm. Route each block's incoming arrow to its first
  RMSNorm when the block is expanded (through the border), so there is a
  continuous connector; collapsed blocks still target the block top.

- The attention drill-down rect sat flush against the attention block's bottom
  border, reading as an overlap. Add a small gap (drillGap) between an
  attention block and its expansion flow.

* fix(inference): count the shared expert in the specs-bar active count

The specs bar showed "6/385", but the always-on shared expert is active too,
so 7 experts run per token (6 routed + 1 shared). Show "6+1/385" for
shared-expert MoE models (e.g. R1 → "8+1/257") so the active count isn't
undersold; the title's "N active" params and the router subtitle already
account for the shared expert.

* final

Author: Oseltamivir

packages/app/cypress/e2e/model-architecture.cy.ts

@@ -316,4 +316,94 @@ describe('Model Architecture Diagram', () => {
       cy.contains('Released by OpenAI').should('be.visible');
     });
   });
+
+  describe('Hybrid Attention Blocks (MoE model - DeepSeek V4 Pro)', () => {
+    before(() => {
+      // Clear any stale Radix scroll lock from prior Select interactions
+      cy.document().then((doc) => {
+        delete doc.body.dataset.scrollLocked;
+        doc.body.style.removeProperty('pointer-events');
+      });
+      cy.get('[role="combobox"]').filter(':visible').first().click();
+      cy.get('[role="option"]').contains('DeepSeek V4 Pro').click();
+
+      cy.get('[data-testid="model-architecture-toggle"]').should('be.visible');
+      cy.get('body').then(($body) => {
+        if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) {
+          cy.get('[data-testid="model-architecture-toggle"]').click();
+        }
+      });
+      cy.get('[data-testid="model-architecture-svg"]').should('be.visible');
+    });
+
+    it('shows MoE and Hybrid badges for DeepSeek V4 Pro', () => {
+      cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE');
+      cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'Hybrid');
+      cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '1.6T');
+    });
+
+    it('shows two separate hybrid (CSA/HCA) blocks with an alternating indicator', () => {
+      cy.get('[data-testid="expand-altBlock0"]').should('exist');
+      cy.get('[data-testid="expand-altBlock1"]').should('exist');
+      cy.get('[data-testid="expand-transformer"]').should('not.exist');
+      cy.get('[data-testid="expand-denseTransformer"]').should('not.exist');
+      cy.get('[data-testid="alternating-indicator"]').should('exist');
+    });
+
+    it('shows a hash-routed MoE prefix block; mHC caption appears once a block is open', () => {
+      // First 3 layers render as a separate hash-routed prefix block
+      cy.get('[data-testid="expand-hashBlock"]').should('exist');
+      // mHC caption only appears once a block exposing the mixer nodes is expanded
+      cy.get('[data-testid="mhc-note"]').should('not.exist');
+      cy.get('[data-testid="expand-hashBlock"]').click({ force: true });
+      cy.get('[data-testid="collapse-hashBlock"]').should('exist');
+      cy.get('[data-testid="model-architecture-svg"]').contains('Hash Router').should('exist');
+      cy.get('[data-testid="mhc-note"]').should('be.visible').and('contain', 'Hyper-Connections');
+      // Restore collapsed state for subsequent tests (shared state: testIsolation off)
+      cy.get('[data-testid="collapse-hashBlock"]').click({ force: true });
+      cy.get('[data-testid="mhc-note"]').should('not.exist');
+    });
+
+    it('Hybrid attention is expandable and drills down to a Sliding Window block', () => {
+      cy.get('[data-testid="expand-altBlock0"]').click({ force: true });
+      cy.get('[data-testid="collapse-altBlock0"]').should('exist');
+      // The union-softmax caption only appears once the attention drill-down is open
+      cy.get('[data-testid="hybrid-attention-note"]').should('not.exist');
+      // Hybrid attention drills down (unlike gpt-oss sink/full GQA, which does not)
+      cy.get('[data-testid="expand-altAttention0"]').should('exist');
+      cy.get('[data-testid="expand-altAttention0"]').click({ force: true });
+      cy.get('[data-testid="model-architecture-svg"]').should('be.visible');
+      // Caption clarifies the two branches feed one softmax (not two attentions)
+      cy.get('[data-testid="hybrid-attention-note"]')
+        .should('be.visible')
+        .and('contain', 'single softmax');
+      // Expert grid still expandable within the block
+      cy.get('[data-testid="expand-altExperts0"]').should('exist');
+    });
+
+    it('expert grid can be expanded to show SwiGLU details', () => {
+      cy.get('[data-testid="expand-altExperts0"]').click({ force: true });
+      cy.get('[data-testid="model-architecture-svg"]').should('be.visible');
+    });
+
+    it('collapsing the parent block hides the union-softmax caption (no orphan caption)', () => {
+      // altBlock0 + altAttention0 are expanded from the previous tests; collapsing
+      // the parent removes the drill-down from the SVG, so the caption must go too
+      // even though altAttention0 stays in the expansion state.
+      cy.get('[data-testid="hybrid-attention-note"]').should('be.visible');
+      cy.get('[data-testid="collapse-altBlock0"]').click({ force: true });
+      cy.get('[data-testid="hybrid-attention-note"]').should('not.exist');
+      // Re-expanding the parent restores the remembered drill-down and its caption.
+      cy.get('[data-testid="expand-altBlock0"]').click({ force: true });
+      cy.get('[data-testid="hybrid-attention-note"]')
+        .should('be.visible')
+        .and('contain', 'single softmax');
+    });
+
+    it('shows DeepSeek V4 Pro features (incl. sliding window) and developer info', () => {
+      cy.contains('Hybrid CSA + HCA Attention').should('be.visible');
+      cy.contains('Sliding window (128 tokens)').should('be.visible');
+      cy.contains('Released by DeepSeek').should('be.visible');
+    });
+  });
 });