Skip to content

Commit cd14290

Browse files
mayurpisemayurpise
authored
Merge pull request #43 from drafthq/claude/pageindex-draft-artifacts-emfxyy
Tree-search retrieval over the OKF bundle (PageIndex-style)
2 parents 890eb20 + e3bef1b commit cd14290

6 files changed

Lines changed: 338 additions & 0 deletions

File tree

‎core/shared/draft-context-loading.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,15 @@ Apply relevance scoring when ALL of these conditions are true:
117117

118118
Do NOT apply relevance scoring for commands that need full context (`/draft:init`, `/draft:deep-review`, `/draft:decompose`).
119119

120+
### Retrieval Path Selection (okf vs monolith)
121+
122+
Relevance scoring has two implementations; pick by output mode:
123+
124+
- **okf mode** — if `draft/wiki/` exists, use **tree-search retrieval** (`core/shared/okf-retrieval.md`): navigate the OKF Concept Map by reasoning over each concept's routing `description`, descending only the matching subtrees. This is the vectorless, reasoning-based path (PageIndex-style) and supersedes the static section table below for okf bundles.
125+
- **monolith mode** — if `draft/wiki/` does not exist, use the static **Scoring Procedure** below against `.ai-context.md` sections.
126+
127+
The minimum context floor (`META`, `INVARIANTS`, `TEST`, `FILES`) applies to both paths.
128+
120129
### Scoring Procedure
121130

122131
1. **Extract key concepts** from the active task:

‎core/shared/okf-retrieval.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# OKF Tree-Search Retrieval
2+
3+
Reasoning-based retrieval over the OKF knowledge bundle. When a project was emitted in `okf` mode (`draft/wiki/` exists), agents locate relevant context by **navigating the concept tree** — reading routing descriptions and descending only the matching subtrees — instead of loading sections by a static heuristic. No embeddings, no chunking, no similarity search: relevance is decided by reasoning over the tree, the same way a human expert scans a table of contents.
4+
5+
Referenced by: `core/shared/draft-context-loading.md` (Relevance-Scored Context Loading). Applies to every command that loads focused project context (`/draft:implement`, `/draft:bughunt`, `/draft:review`, `/draft:debug`, `/draft:change`).
6+
7+
> **Prior art.** This adapts the navigation model of [PageIndex](https://github.com/VectifyAI/PageIndex) (vectorless, reasoning-based RAG) to Draft's self-authored artifacts. Draft already builds the tree — the OKF bundle (`index.md` → section indexes → concept pages) is a table-of-contents whose `description` frontmatter is the per-node routing key. What this procedure adds is the **retrieval loop**: a reasoning descent over that tree. Draft does not need PageIndex's tree-*generation* engine (it authors the tree itself) and stays vectorless by design.
8+
9+
## When this applies
10+
11+
Apply tree-search retrieval when ALL of these hold:
12+
13+
1. `draft/wiki/` exists (project emitted in `okf` mode) **and** `draft/wiki/index.md` carries a populated `<!-- CONCEPT-MAP:START -->…:END -->` block.
14+
2. A specific track or task is active (focused retrieval — broad tasks terminate at the Synopsis, see below).
15+
3. The command benefits from focused context (the relevance-scoring conditions in `draft-context-loading.md`).
16+
17+
If `draft/wiki/` does **not** exist (monolith mode), skip this procedure entirely and use the static section-scoring table in `draft-context-loading.md`. The two are mutually exclusive: tree-search is the okf-mode retrieval path; section-scoring is the monolith path.
18+
19+
## The retrieval loop
20+
21+
The bundle is a tree: `.ai-context.md` (index root: Synopsis + Concept Map) → section indexes (`systems/`, `features/`, `reference/`, `entrypoints/`, `overview/`) → concept pages. Each node advertises a `description` routing key. Navigate it, do not flatten it.
22+
23+
```
24+
1. Frame the query
25+
Extract routing terms from the active task: domain nouns from spec.md
26+
acceptance criteria, file paths / module names / tech terms from plan.md,
27+
and the primary concern (data flow, API, security, perf, config, …).
28+
29+
2. Enter at the root
30+
Read draft/.ai-context.md. The Synopsis is the cheap broad-context path —
31+
for a BROAD task (onboarding, architecture overview, "how does X work
32+
end-to-end") it is sufficient: TERMINATE here, do not descend.
33+
For a FOCUSED task, read the Concept Map (the root routing table).
34+
35+
3. Select subtrees (reason, don't match strings)
36+
For each Concept Map row, judge the `description` as a ROUTING DECISION:
37+
"does opening this concept help THIS task?" Score each candidate:
38+
- strong — description names the task's responsibility or its own terms
39+
- possible — adjacent/depends-on the task area
40+
- skip — unrelated
41+
Descend `strong` first; hold `possible` as a frontier for step 5.
42+
43+
4. Descend to leaves
44+
For a selected section, open its index.md and repeat step 3 against the
45+
section's concept rows (one routing description per concept). Open the
46+
matching concept page(s). A concept page is a LEAF — its `x-grounded-paths`
47+
are the exact source files the task should open; `Used by` / `x-callers`
48+
give the next hop if the task spans callers.
49+
50+
5. Expand only if under-covered
51+
If the opened leaves do not cover the task's routing terms, expand the
52+
highest-scored `possible` frontier node (step 3) and recurse. Otherwise stop.
53+
```
54+
55+
## Routing decision criteria
56+
57+
The `description` frontmatter is load-bearing — it is written as a routing decision, not a summary (`core/templates/okf/concept.md`). Judge each node by:
58+
59+
| Signal | Descend when… |
60+
|--------|---------------|
61+
| Responsibility match | The description names the capability/module the task changes |
62+
| Term overlap | Task's domain nouns / file paths appear in the description or `x-grounded-paths` |
63+
| Caller/blast-radius reach | Task modifies a symbol whose `x-callers` / `Used by` point at this node |
64+
| Concern alignment | Task's primary concern (security, perf, data flow) is this node's stated focus |
65+
66+
Reason about relevance — do not keyword-match. A concept whose description does not justify opening it for the task at hand is skipped even if a term coincidentally overlaps (similarity ≠ relevance).
67+
68+
## Termination & budget
69+
70+
- **Broad task** → terminate at the Synopsis (step 2). Do not open concept pages.
71+
- **Focused task** → terminate when opened leaves cover the task's routing terms, or when **≤ 5 concept pages** have been opened (default budget; raise only if the task explicitly spans many subsystems, e.g. a cross-cutting refactor).
72+
- **Depth** → the tree is shallow by construction (root → section → concept ≈ 2 hops). If a descent has not reached a leaf in 3 hops, stop and open the best leaf seen so far.
73+
- **No match** → if no Concept Map row scores above `skip`, fall back to the Synopsis plus `## INVARIANTS` / `## FILES` / `## TEST` floor from `draft-context-loading.md`.
74+
75+
The minimum context floor from `draft-context-loading.md` (`META`, `INVARIANTS`, `TEST`, `FILES`) still applies and is always loaded regardless of the descent.
76+
77+
## Output contract (traceability)
78+
79+
Tree-search retrieval is explainable by construction — record the path taken, mirroring PageIndex's node-ID grounding:
80+
81+
- **Opened concepts** — the leaf pages selected, each with the one-line reason it was opened.
82+
- **Grounded paths** — the union of `x-grounded-paths` across opened leaves: the precise source files the task will read or modify.
83+
- **Skipped frontier** — `possible` nodes held but not expanded (so a follow-up task can resume from them).
84+
85+
Surface this trace when the command reports which context it loaded (e.g. `/draft:implement` plan preamble, `/draft:review` scope note). It replaces "loaded sections A, B, C" with "navigated to concepts X, Y because …".
86+
87+
## Degradation
88+
89+
| Scenario | Behavior |
90+
|----------|----------|
91+
| `draft/wiki/` missing | Skip; use monolith section-scoring in `draft-context-loading.md` |
92+
| Concept Map markers empty/absent | Fall back to reading `wiki/*/index.md` section tables directly; if those are missing, use the Synopsis + floor |
93+
| Routing descriptions thin/uninformative | Open the section `index.md` and skim concept titles; flag for `/draft:init refresh` to regenerate descriptions |
94+
| Task is broad | Terminate at Synopsis — descending is over-fetch (a Red Flag per `red-flags.md`) |

‎hld-draft-init-okf-taxonomy.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,16 @@ Agent reads this one file first. Broad tasks terminate here. Focused tasks route
138138

139139
---
140140

141+
## 6.5. Retrieval over the bundle (tree-search, vectorless)
142+
143+
The emitter builds the tree; retrieval *navigates* it. The procedure is specified in `core/shared/okf-retrieval.md` and wired into `core/shared/draft-context-loading.md` as the okf-mode relevance path (monolith keeps the static section-scoring table).
144+
145+
Loop: enter at `.ai-context.md` → Synopsis satisfies broad tasks (terminate); for focused tasks reason over the Concept Map's `description` routing keys, descend `strong` subtrees into section indexes and concept leaves, and stop when the opened leaves cover the task's routing terms (budget ≤5 concept pages, depth ≈2 hops). Leaves expose `x-grounded-paths` (exact source files) and `x-callers` (next hop). The navigation path is recorded for traceability.
146+
147+
**Prior art.** This adapts [PageIndex](https://github.com/VectifyAI/PageIndex) (vectorless, reasoning-based RAG) — same thesis Draft already holds: *similarity ≠ relevance; retrieval needs reasoning over a navigable tree, not vector lookup over chunks.* Draft converged on this from the codebase-context direction; the OKF Concept Map **is** a PageIndex tree, and the `description` frontmatter **is** a node summary. Draft does not adopt PageIndex's tree-*generation* engine (it authors the tree itself in §7) and adds no vector/embedding layer — only the retrieval loop was missing, and §6.5 supplies it.
148+
149+
---
150+
141151
## 7. Generation pipeline
142152

143153
Reuses existing analysis; adds decomposition + serialization. No new LLM analysis engine.

‎integrations/agents/AGENTS.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16274,6 +16274,15 @@ Apply relevance scoring when ALL of these conditions are true:
1627416274

1627516275
Do NOT apply relevance scoring for commands that need full context (`draft init`, `draft deep-review`, `draft decompose`).
1627616276

16277+
### Retrieval Path Selection (okf vs monolith)
16278+
16279+
Relevance scoring has two implementations; pick by output mode:
16280+
16281+
- **okf mode** — if `draft/wiki/` exists, use **tree-search retrieval** (`core/shared/okf-retrieval.md`): navigate the OKF Concept Map by reasoning over each concept's routing `description`, descending only the matching subtrees. This is the vectorless, reasoning-based path (PageIndex-style) and supersedes the static section table below for okf bundles.
16282+
- **monolith mode** — if `draft/wiki/` does not exist, use the static **Scoring Procedure** below against `.ai-context.md` sections.
16283+
16284+
The minimum context floor (`META`, `INVARIANTS`, `TEST`, `FILES`) applies to both paths.
16285+
1627716286
### Scoring Procedure
1627816287

1627916288
1. **Extract key concepts** from the active task:
@@ -17647,6 +17656,109 @@ The engine indexes C/C++, Go, Python, TypeScript/JS, and more (tree-sitter, 159
1764717656

1764817657
---
1764917658

17659+
## core/shared/okf-retrieval.md
17660+
17661+
<core-file path="core/shared/okf-retrieval.md">
17662+
17663+
# OKF Tree-Search Retrieval
17664+
17665+
Reasoning-based retrieval over the OKF knowledge bundle. When a project was emitted in `okf` mode (`draft/wiki/` exists), agents locate relevant context by **navigating the concept tree** — reading routing descriptions and descending only the matching subtrees — instead of loading sections by a static heuristic. No embeddings, no chunking, no similarity search: relevance is decided by reasoning over the tree, the same way a human expert scans a table of contents.
17666+
17667+
Referenced by: `core/shared/draft-context-loading.md` (Relevance-Scored Context Loading). Applies to every command that loads focused project context (`draft implement`, `draft bughunt`, `draft review`, `draft debug`, `draft change`).
17668+
17669+
> **Prior art.** This adapts the navigation model of [PageIndex](https://github.com/VectifyAI/PageIndex) (vectorless, reasoning-based RAG) to Draft's self-authored artifacts. Draft already builds the tree — the OKF bundle (`index.md` → section indexes → concept pages) is a table-of-contents whose `description` frontmatter is the per-node routing key. What this procedure adds is the **retrieval loop**: a reasoning descent over that tree. Draft does not need PageIndex's tree-*generation* engine (it authors the tree itself) and stays vectorless by design.
17670+
17671+
## When this applies
17672+
17673+
Apply tree-search retrieval when ALL of these hold:
17674+
17675+
1. `draft/wiki/` exists (project emitted in `okf` mode) **and** `draft/wiki/index.md` carries a populated `<!-- CONCEPT-MAP:START -->…:END -->` block.
17676+
2. A specific track or task is active (focused retrieval — broad tasks terminate at the Synopsis, see below).
17677+
3. The command benefits from focused context (the relevance-scoring conditions in `draft-context-loading.md`).
17678+
17679+
If `draft/wiki/` does **not** exist (monolith mode), skip this procedure entirely and use the static section-scoring table in `draft-context-loading.md`. The two are mutually exclusive: tree-search is the okf-mode retrieval path; section-scoring is the monolith path.
17680+
17681+
## The retrieval loop
17682+
17683+
The bundle is a tree: `.ai-context.md` (index root: Synopsis + Concept Map) → section indexes (`systems/`, `features/`, `reference/`, `entrypoints/`, `overview/`) → concept pages. Each node advertises a `description` routing key. Navigate it, do not flatten it.
17684+
17685+
```
17686+
1. Frame the query
17687+
Extract routing terms from the active task: domain nouns from spec.md
17688+
acceptance criteria, file paths / module names / tech terms from plan.md,
17689+
and the primary concern (data flow, API, security, perf, config, …).
17690+
17691+
2. Enter at the root
17692+
Read draft/.ai-context.md. The Synopsis is the cheap broad-context path —
17693+
for a BROAD task (onboarding, architecture overview, "how does X work
17694+
end-to-end") it is sufficient: TERMINATE here, do not descend.
17695+
For a FOCUSED task, read the Concept Map (the root routing table).
17696+
17697+
3. Select subtrees (reason, don't match strings)
17698+
For each Concept Map row, judge the `description` as a ROUTING DECISION:
17699+
"does opening this concept help THIS task?" Score each candidate:
17700+
- strong — description names the task's responsibility or its own terms
17701+
- possible — adjacent/depends-on the task area
17702+
- skip — unrelated
17703+
Descend `strong` first; hold `possible` as a frontier for step 5.
17704+
17705+
4. Descend to leaves
17706+
For a selected section, open its index.md and repeat step 3 against the
17707+
section's concept rows (one routing description per concept). Open the
17708+
matching concept page(s). A concept page is a LEAF — its `x-grounded-paths`
17709+
are the exact source files the task should open; `Used by` / `x-callers`
17710+
give the next hop if the task spans callers.
17711+
17712+
5. Expand only if under-covered
17713+
If the opened leaves do not cover the task's routing terms, expand the
17714+
highest-scored `possible` frontier node (step 3) and recurse. Otherwise stop.
17715+
```
17716+
17717+
## Routing decision criteria
17718+
17719+
The `description` frontmatter is load-bearing — it is written as a routing decision, not a summary (`core/templates/okf/concept.md`). Judge each node by:
17720+
17721+
| Signal | Descend when… |
17722+
|--------|---------------|
17723+
| Responsibility match | The description names the capability/module the task changes |
17724+
| Term overlap | Task's domain nouns / file paths appear in the description or `x-grounded-paths` |
17725+
| Caller/blast-radius reach | Task modifies a symbol whose `x-callers` / `Used by` point at this node |
17726+
| Concern alignment | Task's primary concern (security, perf, data flow) is this node's stated focus |
17727+
17728+
Reason about relevance — do not keyword-match. A concept whose description does not justify opening it for the task at hand is skipped even if a term coincidentally overlaps (similarity ≠ relevance).
17729+
17730+
## Termination & budget
17731+
17732+
- **Broad task** → terminate at the Synopsis (step 2). Do not open concept pages.
17733+
- **Focused task** → terminate when opened leaves cover the task's routing terms, or when **≤ 5 concept pages** have been opened (default budget; raise only if the task explicitly spans many subsystems, e.g. a cross-cutting refactor).
17734+
- **Depth** → the tree is shallow by construction (root → section → concept ≈ 2 hops). If a descent has not reached a leaf in 3 hops, stop and open the best leaf seen so far.
17735+
- **No match** → if no Concept Map row scores above `skip`, fall back to the Synopsis plus `## INVARIANTS` / `## FILES` / `## TEST` floor from `draft-context-loading.md`.
17736+
17737+
The minimum context floor from `draft-context-loading.md` (`META`, `INVARIANTS`, `TEST`, `FILES`) still applies and is always loaded regardless of the descent.
17738+
17739+
## Output contract (traceability)
17740+
17741+
Tree-search retrieval is explainable by construction — record the path taken, mirroring PageIndex's node-ID grounding:
17742+
17743+
- **Opened concepts** — the leaf pages selected, each with the one-line reason it was opened.
17744+
- **Grounded paths** — the union of `x-grounded-paths` across opened leaves: the precise source files the task will read or modify.
17745+
- **Skipped frontier** — `possible` nodes held but not expanded (so a follow-up task can resume from them).
17746+
17747+
Surface this trace when the command reports which context it loaded (e.g. `draft implement` plan preamble, `draft review` scope note). It replaces "loaded sections A, B, C" with "navigated to concepts X, Y because …".
17748+
17749+
## Degradation
17750+
17751+
| Scenario | Behavior |
17752+
|----------|----------|
17753+
| `draft/wiki/` missing | Skip; use monolith section-scoring in `draft-context-loading.md` |
17754+
| Concept Map markers empty/absent | Fall back to reading `wiki/*/index.md` section tables directly; if those are missing, use the Synopsis + floor |
17755+
| Routing descriptions thin/uninformative | Open the section `index.md` and skim concept titles; flag for `draft init refresh` to regenerate descriptions |
17756+
| Task is broad | Terminate at Synopsis — descending is over-fetch (a Red Flag per `red-flags.md`) |
17757+
17758+
</core-file>
17759+
17760+
---
17761+
1765017762
## core/shared/parallel-analysis.md
1765117763

1765217764
<core-file path="core/shared/parallel-analysis.md">

0 commit comments

Comments
 (0)