Skip to content

Commit 16e6dd8

Browse files
raifdmuellerclaude
andcommitted
fix(socratic-recovery): carry code evidence into Phase 2 citations
Phase 2 synthesized documentation cited only the Q-ID (e.g. [Q3.5]). The file:line code evidence sits in the [ANSWERED] leaf of the Question Tree, so a reader had to open the tree to find where a claim came from. Phase 2 now copies the Evidence line from the [ANSWERED] leaf into the citation alongside the Q-ID — [Q3.5; src/app/Ports.java:12] — so the source location is visible in the documentation itself. Team answers keep the (team answer, Q-ID) form (no code evidence exists); deferred questions stay explicit gaps. Updated the skill (phase-2-synthesize prompt, output-schema, SKILL.md) and the website brownfield-workflow / socratic-recovery-skill docs including the copy-paste Phase 2 cheat-sheet prompt, EN and DE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent e9b2066 commit 16e6dd8

10 files changed

Lines changed: 56 additions & 24 deletions

File tree

docs/brownfield-workflow.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,7 @@ The LLM synthesizes the answered questions plus the code evidence from Phase 1 i
163163
* *arc42* with all 12 chapters from Q-3 branch
164164
* *Nygard ADRs* with Pugh Matrix from Q-3.9 branch
165165

166-
Every claim references a Question ID and marks team-provided information with `(team answer)`. This dual traceability (code evidence + team input) is the key difference from a simple reverse-engineering prompt.
166+
Every claim references a Question ID. A code-derived claim also carries the `file:line` evidence from its `[ANSWERED]` leaf, copied into the citation so the source is visible without opening the Question Tree; team-provided information is marked `(team answer)`. This dual traceability (code evidence + team input) is the key difference from a simple reverse-engineering prompt.
167167

168168
=== Establish Baseline Tests
169169

@@ -261,7 +261,7 @@ Stable code that nobody touches does not need specs.
261261
|{empty}--
262262

263263
|Theory Recovery (Phase 2)
264-
|`Synthesize documentation from the Question Tree and team answers. Every claim references a Q-ID. Mark team input with (team answer).`
264+
|`Synthesize documentation from the Question Tree and team answers. Every claim references a Q-ID; for code-derived claims, also cite the file:line evidence from the ANSWERED leaf. Mark team input with (team answer).`
265265
|link:#/spec-driven-development[Spec-Driven Workflow]
266266

267267
|Baseline Tests

docs/brownfield-workflow.de.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ Das LLM synthetisiert die beantworteten Fragen plus die Code-Evidenz aus Phase 1
161161
* *arc42* mit allen 12 Kapiteln aus dem Q3-Ast
162162
* *Nygard-ADRs* mit Pugh-Matrix aus dem Q3.9-Ast
163163

164-
Jede Aussage referenziert eine Question-ID und markiert teamgegebene Information mit `(team answer)`. Diese doppelte Rückverfolgbarkeit (Code-Evidenz + Team-Input) ist der entscheidende Unterschied zu einem einfachen Reverse-Engineering-Prompt.
164+
Jede Aussage referenziert eine Question-ID. Eine code-basierte Aussage trägt zusätzlich die `file:line`-Evidenz aus ihrem `[ANSWERED]`-Leaf — in die Zitation kopiert, damit die Quelle sichtbar ist, ohne den Question Tree zu öffnen; teamgegebene Information wird mit `(team answer)` markiert. Diese doppelte Rückverfolgbarkeit (Code-Evidenz + Team-Input) ist der entscheidende Unterschied zu einem einfachen Reverse-Engineering-Prompt.
165165

166166
=== Basis-Tests aufbauen
167167

@@ -259,7 +259,7 @@ Stabiler Code, den niemand anfasst, braucht keine Specs.
259259
|{empty}--
260260

261261
|Theory Recovery (Phase 2)
262-
|`Synthesize documentation from the Question Tree and team answers. Every claim references a Q-ID. Mark team input with (team answer).`
262+
|`Synthesize documentation from the Question Tree and team answers. Every claim references a Q-ID; for code-derived claims, also cite the file:line evidence from the ANSWERED leaf. Mark team input with (team answer).`
263263
|link:#/spec-driven-development[Spec-Driven Workflow]
264264

265265
|Basis-Tests

docs/socratic-recovery-skill.adoc

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Outputs two AsciiDoc files: `QUESTION_TREE.adoc` (full reasoning trace) and `OPE
2222

2323
=== Phase 2 — Synthesize documentation
2424

25-
The skill takes the answered tree and produces a PRD, Cockburn use cases, an arc42 architecture document, and Nygard ADRs with Pugh matrices. Every claim cites a Q-ID; team-supplied facts are marked `(team answer)`.
25+
The skill takes the answered tree and produces a PRD, Cockburn use cases, an arc42 architecture document, and Nygard ADRs with Pugh matrices. Every claim cites a Q-ID; code-derived claims also carry the `file:line` evidence from their `[ANSWERED]` leaf, and team-supplied facts are marked `(team answer)`.
2626

2727
== When to use it
2828

@@ -105,7 +105,8 @@ https://github.com/LLM-Coding/Semantic-Anchors/tree/main/skill/socratic-code-the
105105
Build a Question Tree before writing any documentation. Mark each leaf
106106
[ANSWERED] (with file:line evidence) or [OPEN] (with Category and Ask role).
107107
Synthesize docs from the answered tree only after the team has filled in
108-
the OPEN leaves. Cite Q-IDs in every claim.
108+
the OPEN leaves. Cite a Q-ID in every claim; for code-derived claims,
109+
also cite the file:line evidence from the ANSWERED leaf.
109110
----
110111

111112
=== link:https://docs.cursor.com/[Cursor]

docs/socratic-recovery-skill.de.adoc

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Output sind zwei AsciiDoc-Dateien: `QUESTION_TREE.adoc` (vollständige Begründu
2222

2323
=== Phase 2 — Dokumentation synthetisieren
2424

25-
Der Skill nimmt den beantworteten Baum und erzeugt ein PRD, Cockburn Use Cases, eine arc42-Architekturbeschreibung und Nygard-ADRs mit Pugh-Matrix. Jede Aussage zitiert eine Q-ID; team-gegebene Fakten sind mit `(team answer)` markiert.
25+
Der Skill nimmt den beantworteten Baum und erzeugt ein PRD, Cockburn Use Cases, eine arc42-Architekturbeschreibung und Nygard-ADRs mit Pugh-Matrix. Jede Aussage zitiert eine Q-ID; code-basierte Aussagen tragen zusätzlich die `file:line`-Evidenz aus ihrem `[ANSWERED]`-Leaf, und team-gegebene Fakten sind mit `(team answer)` markiert.
2626

2727
== Wann zu verwenden
2828

@@ -105,7 +105,8 @@ https://github.com/LLM-Coding/Semantic-Anchors/tree/main/skill/socratic-code-the
105105
Build a Question Tree before writing any documentation. Mark each leaf
106106
[ANSWERED] (with file:line evidence) or [OPEN] (with Category and Ask role).
107107
Synthesize docs from the answered tree only after the team has filled in
108-
the OPEN leaves. Cite Q-IDs in every claim.
108+
the OPEN leaves. Cite a Q-ID in every claim; for code-derived claims,
109+
also cite the file:line evidence from the ANSWERED leaf.
109110
----
110111

111112
=== link:https://docs.cursor.com/[Cursor]

plugins/semantic-anchors/skills/socratic-code-theory-recovery/SKILL.md

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

plugins/semantic-anchors/skills/socratic-code-theory-recovery/prompts/phase-2-synthesize.md

Lines changed: 12 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

plugins/semantic-anchors/skills/socratic-code-theory-recovery/references/output-schema.md

Lines changed: 10 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

skill/socratic-code-theory-recovery/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ Use [prompts/phase-2-synthesize.md](prompts/phase-2-synthesize.md). The Phase 2
104104
- **arc42** with all 12 chapters from the Q3 branch
105105
- **Nygard ADRs** with Pugh Matrix from the Q3.9 branch
106106

107-
Every claim references a Q-ID. Team-supplied information is marked `(team answer)`. This dual traceability — code evidence plus team input — is the difference from a simple reverse-engineering prompt that fills in gaps silently.
107+
Every claim references a Q-ID. A code-derived claim also carries the `file:line` evidence from its `[ANSWERED]` leaf, copied into the citation so the source is visible without opening the Question Tree. Team-supplied information is marked `(team answer)`. This dual traceability — code evidence plus team input — is the difference from a simple reverse-engineering prompt that fills in gaps silently.
108108

109109
## What the LLM can and cannot recover
110110

skill/socratic-code-theory-recovery/prompts/phase-2-synthesize.md

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,10 +50,18 @@ Produce four artifacts:
5050
- Anchor: ADR according to Nygard
5151
5252
Rules for traceability:
53-
- Every paragraph references the Q-IDs that support it, in square brackets:
54-
"The system uses Hexagonal Architecture [Q3.5]."
55-
- Team-supplied facts get an inline marker: "Sessions expire after 24 hours
56-
(team answer, Q3.4.2)."
53+
- Every paragraph references the Q-IDs that support it, in square brackets.
54+
- For a claim backed by an [ANSWERED] leaf, carry the code evidence from
55+
that leaf into the citation alongside the Q-ID — so the reader sees the
56+
source location without opening the Question Tree:
57+
"The system uses Hexagonal Architecture [Q3.5; src/app/Ports.java,
58+
src/adapter/JpaOrderRepository.java:30]."
59+
Copy the Evidence line verbatim from the leaf; do not invent, shorten,
60+
or re-derive file paths. If a leaf has no Evidence line it is not
61+
[ANSWERED] and must not be cited as fact.
62+
- Team-supplied facts have no code evidence — mark them (team answer)
63+
with the Q-ID only: "Sessions expire after 24 hours (team answer,
64+
Q3.4.2)."
5765
- Deferred questions stay as explicit gaps: "Quality-goal priorities are
5866
deferred (Q4.1.deferred) and must be resolved before the next release."
5967
- Do not introduce facts that do not appear in QUESTION_TREE.adoc or

skill/socratic-code-theory-recovery/references/output-schema.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -141,13 +141,20 @@ _(write here)_
141141

142142
## Phase 2 traceability
143143

144-
After Phase 2, every paragraph in the synthesized documentation cites at least one Q-ID:
144+
After Phase 2, every paragraph in the synthesized documentation cites at least one Q-ID. For a claim backed by an `[ANSWERED]` leaf, the citation also carries the code evidence copied from that leaf, so the reader sees the source location without opening the Question Tree:
145145

146146
```
147-
The system uses Hexagonal Architecture [Q3.9.HexagonalArchitecture]. Sessions
147+
The system uses Hexagonal Architecture [Q3.9.HexagonalArchitecture;
148+
src/app/Ports.java, src/adapter/JpaOrderRepository.java:30]. Sessions
148149
expire after 24 hours (team answer, Q3.8.Security.SessionLifetime).
149150
Quality-goal priorities are deferred (Q4.0.deferred) and must be resolved
150151
before the next release.
151152
```
152153

153-
This is the auditable trace from documentation back to either code evidence or a team answer. Anything without a Q-ID is invention.
154+
The three citation forms are deliberate:
155+
156+
- `[Q-ID; file:line, ...]` — code-derived fact. The `file:line` part is the `Evidence` line of the `[ANSWERED]` leaf, copied verbatim.
157+
- `(team answer, Q-ID)` — team-supplied fact. No code evidence exists; the Q-ID points to the answered `[OPEN]` leaf.
158+
- `(Q-ID.deferred)` — an explicit gap, not a fact.
159+
160+
This is the auditable trace from documentation back to either code evidence or a team answer. Anything without a Q-ID is invention; a code-derived claim without its `file:line` evidence is incomplete.

0 commit comments

Comments
 (0)