diff --git a/docs/superpowers/plans/2026-06-23-model-ranking-plan.md b/docs/superpowers/plans/2026-06-23-model-ranking-plan.md new file mode 100644 index 0000000..8fb166f --- /dev/null +++ b/docs/superpowers/plans/2026-06-23-model-ranking-plan.md @@ -0,0 +1,194 @@ +# Model Ranking Feature Implementation Plan + +**Date:** 2026-06-23 +**Feature:** Model KVCache Size Ranking +**Branch:** feat/model-ranking + +## Overview + +Implement a model ranking feature that allows users to compare different models by their KV cache size requirements. The feature will be available both in the web UI (calculator.html) and as a CLI command. + +## Context + +From brainstorming session: +- **Platform:** Both web UI and CLI +- **Metrics:** KV bytes per token AND total KV cache for configurable sequence lengths +- **Model scope:** Existing presets only (Qwen3-27B, Llama3.1-70B, Qwen2.5-72B, Mistral-7B, Llama3.1-8B, DeepSeek-V3-671B) +- **UI presentation:** New section in calculator +- **Deployment config:** Include tp_size/pp_size to show per-rank KV cache size + +## Implementation Tasks + +### Task 1: Extract Model Registry + +**Goal:** Create a single source of truth for model specifications + +**Steps:** +1. Extract model profiles from PRESETS into a separate MODELS object +2. Each model entry includes: name, family, model_profile, description +3. Update PRESETS to reference MODELS instead of duplicating model_profile +4. Verify calculator still works with existing presets + +**Files:** `website/calculator.html` + +**Verification:** Open calculator.html, select different presets, verify model fields populate correctly + +--- + +### Task 2: Add Ranking Calculation Functions (JavaScript) + +**Goal:** Implement ranking logic in the web calculator + +**Steps:** +1. Add `calculateModelRanking()` function that: + - Takes sequence_length as input + - Iterates through all models in MODELS + - Calculates for each model: + - `kv_bytes_per_token` (using kvBytesPerToken) + - `kv_bytes_per_token_per_rank` (using kvBytesPerTokenPerRank) + - `total_kv_bytes` = kv_bytes_per_token * sequence_length + - `total_kv_bytes_per_rank` = kv_bytes_per_token_per_rank * sequence_length + - `total_kv_gb` = total_kv_bytes / BYTES_PER_GB + - Returns sorted array (default: by total_kv_gb descending) +2. Add `formatBytes()` helper for human-readable byte formatting +3. Add `renderRankingTable(rankingData, sortBy)` function that: + - Renders HTML table with columns: Rank, Model, KV/Token, KV/Token/Rank, Total KV (X tokens), Total GB + - Supports sorting by clicking column headers + - Highlights currently selected model + +**Files:** `website/calculator.html` + +**Verification:** Open browser console, call `calculateModelRanking(8192)`, verify output structure + +--- + +### Task 3: Add Model Ranking UI Section + +**Goal:** Create user-facing ranking interface in calculator + +**Steps:** +1. Add new section after "Model Preset" card: + ```html +
+
Model KVCache Ranking
+
+ + + +
+
+
+ ``` +2. Style the section with CSS (add to existing + + +
+

KVCache Model Ranking

+

Compare KV cache size requirements across popular LLM models

+
+ +
+ +
+
+ +
+ +
+
+ + +
+
+ + +
+
+
+ +
+ + +
+ +
+ + + + + + + + + + + + + + + + + + +
Model KV Layers KV Heads Head Dim TP PP Parameters Weight (GB) KV/Token KV/Token/Rank Total KV Total KV/Rank
+
+ + + + + +