|
| 1 | +# AI Chat Context and Payload Strategy |
| 2 | + |
| 3 | +This document describes the current AI chat request construction approach in @knighted/develop, including context shaping, tool usage, payload-size controls, and known improvement opportunities. |
| 4 | + |
| 5 | +## Current Approach |
| 6 | + |
| 7 | +### 1. System prompt and mode-aware policy |
| 8 | + |
| 9 | +Each request includes a system prompt with policy guidance, then augments that prompt with mode-aware constraints: |
| 10 | + |
| 11 | +- Render mode guidance (DOM vs React) |
| 12 | +- Style mode guidance (css, module, less, sass) |
| 13 | +- DOM-mode JSX guidance for @knighted/jsx runtime |
| 14 | +- Explicit React-avoidance in DOM mode unless migration is requested |
| 15 | +- Dialect preservation guidance (avoid cross-dialect rewrites unless requested) |
| 16 | + |
| 17 | +Primary implementation: |
| 18 | + |
| 19 | +- src/modules/github/chat/payload.js |
| 20 | + |
| 21 | +### 2. Repository context |
| 22 | + |
| 23 | +Each request includes repository targeting context as a dedicated system message: |
| 24 | + |
| 25 | +- Selected repository full name |
| 26 | +- Repository URL |
| 27 | +- Default branch |
| 28 | +- Policy to treat selected repository as default unless overridden |
| 29 | + |
| 30 | +Primary implementation: |
| 31 | + |
| 32 | +- src/modules/github/chat/drawer.js |
| 33 | + |
| 34 | +### 3. Editor context (Send tab content) |
| 35 | + |
| 36 | +When enabled, the drawer includes active tab context as a system message: |
| 37 | + |
| 38 | +- Render mode and style mode |
| 39 | +- Active tab label/path |
| 40 | +- Available tab targets list (id/path/name/language), currently capped to 20 |
| 41 | +- Active tab source code block |
| 42 | + |
| 43 | +This context is designed to support dynamic proposal targeting by tab id/path and reduce ambiguity. |
| 44 | + |
| 45 | +Primary implementation: |
| 46 | + |
| 47 | +- src/modules/github/chat/active-tab-context.js |
| 48 | +- src/modules/github/chat/drawer.js |
| 49 | + |
| 50 | +### 4. Tooling model |
| 51 | + |
| 52 | +AI proposal actions currently use a function tool: |
| 53 | + |
| 54 | +- propose_editor_update |
| 55 | + |
| 56 | +Contract: |
| 57 | + |
| 58 | +- target: tab id or path |
| 59 | +- content: full replacement tab content |
| 60 | +- language: optional disambiguation hint |
| 61 | +- rationale: optional explanation |
| 62 | + |
| 63 | +Primary implementation: |
| 64 | + |
| 65 | +- src/modules/github/chat/proposals.js |
| 66 | +- src/modules/github/chat/tab-target-resolver.js |
| 67 | +- src/modules/github/chat/drawer.js |
| 68 | + |
| 69 | +### 5. Apply and undo behavior |
| 70 | + |
| 71 | +- Apply is proposal-driven and tab-target-aware (id/path resolution) |
| 72 | +- Undo is scoped per tab (latest snapshot per tab) |
| 73 | +- Undo UI is visible for active tab snapshot only |
| 74 | + |
| 75 | +Primary implementation: |
| 76 | + |
| 77 | +- src/modules/github/chat/drawer.js |
| 78 | +- src/modules/github/chat/tab-scoped-undo-state.js |
| 79 | + |
| 80 | +### 6. Payload size controls and summary strategy |
| 81 | + |
| 82 | +The payload builder includes bounded-conversation controls: |
| 83 | + |
| 84 | +- Hard byte budget: 120_000 bytes |
| 85 | +- Direct conversation retention cap: latest 14 chat messages |
| 86 | +- Summary cap: 3_600 characters |
| 87 | +- Older dropped conversation turns are compacted into a rolling system summary |
| 88 | + |
| 89 | +Primary implementation: |
| 90 | + |
| 91 | +- src/modules/github/chat/payload.js |
| 92 | + |
| 93 | +### 7. Fallback and transport behavior |
| 94 | + |
| 95 | +- Streaming request path is attempted first |
| 96 | +- Non-stream fallback is attempted on streaming failure |
| 97 | +- Model access errors are surfaced with tailored status text |
| 98 | + |
| 99 | +Primary implementation: |
| 100 | + |
| 101 | +- src/modules/github/chat/drawer.js |
| 102 | +- src/modules/github/api/chat.js |
| 103 | + |
| 104 | +## Why this approach |
| 105 | + |
| 106 | +- Keeps active-tab workflows lightweight and responsive |
| 107 | +- Supports explicit user review before applying generated edits |
| 108 | +- Preserves model guidance quality with mode/dialect policy constraints |
| 109 | +- Reduces request-size growth with bounded message history and rolling summaries |
| 110 | + |
| 111 | +## Possible Areas for Improvement |
| 112 | + |
| 113 | +### 1. Hard-fit protection when system context alone is large |
| 114 | + |
| 115 | +Current shrinking behavior primarily trims conversation turns. Add a final hard-fit step that can selectively trim editor context sections when total payload still exceeds budget. |
| 116 | + |
| 117 | +Potential ideas: |
| 118 | + |
| 119 | +- Trim available tab target list length adaptively |
| 120 | +- Clip active tab source with clear truncation markers |
| 121 | +- Retry once on 413 with reduced context envelope |
| 122 | + |
| 123 | +### 2. Create-tab capability |
| 124 | + |
| 125 | +Add a dedicated tool for creating workspace tabs so requests like "create a new styles tab" can be completed in one interaction. |
| 126 | + |
| 127 | +Potential tool: |
| 128 | + |
| 129 | +- create_workspace_tab(path, language, initialContent?, activate?) |
| 130 | + |
| 131 | +### 3. Cross-tab source access |
| 132 | + |
| 133 | +Support workflows where the user references a non-active tab. |
| 134 | + |
| 135 | +Potential options: |
| 136 | + |
| 137 | +- Add Send all tabs mode with explicit byte budgeting |
| 138 | +- Add read_workspace_tab tool for targeted lookup |
| 139 | + |
| 140 | +### 4. Better summary fidelity |
| 141 | + |
| 142 | +Current summary is compact and bounded, but can lose nuanced intent over long sessions. |
| 143 | + |
| 144 | +Potential ideas: |
| 145 | + |
| 146 | +- Structured summary sections (goals, constraints, pending asks) |
| 147 | +- Weighted retention for user constraints and accepted decisions |
| 148 | + |
| 149 | +### 5. Context observability in UI |
| 150 | + |
| 151 | +Provide optional diagnostics showing what context is being sent in the next request. |
| 152 | + |
| 153 | +Potential ideas: |
| 154 | + |
| 155 | +- "Preview outgoing context" drawer section |
| 156 | +- Approximate byte-count indicator before send |
| 157 | + |
| 158 | +### 6. Tool-call UX clarity |
| 159 | + |
| 160 | +Continue improving copy and actions so users understand what is proposed versus what is already applied. |
| 161 | + |
| 162 | +Potential ideas: |
| 163 | + |
| 164 | +- Show target tab path in each action |
| 165 | +- Add optional diff preview before apply |
| 166 | + |
| 167 | +### 7. Optional stricter policy profiles |
| 168 | + |
| 169 | +Allow policy strictness presets depending on user goals. |
| 170 | + |
| 171 | +Potential ideas: |
| 172 | + |
| 173 | +- Conservative mode: fewer tool proposals, stronger minimal-change bias |
| 174 | +- Refactor mode: broader architectural proposal tolerance |
| 175 | + |
| 176 | +## Validation status |
| 177 | + |
| 178 | +Current strategy has focused Playwright coverage for the chat drawer behavior and context policy assertions in: |
| 179 | + |
| 180 | +- playwright/github-byot-ai.spec.ts |
| 181 | + |
| 182 | +## Scope note |
| 183 | + |
| 184 | +This document is intentionally implementation-oriented. It describes current behavior and practical next improvements without locking future UX or API contracts. |
0 commit comments