Skip to content

Commit 4b6b04c

Browse files
committed
docs: update ask_self documentation and integration guide
1 parent c5cb8e0 commit 4b6b04c

2 files changed

Lines changed: 231 additions & 0 deletions

File tree

ask_self/README.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,26 @@
44

55
This README is written for the next LLM or engineer who inherits this folder and wants to maintain or spin it out into its own repository.
66

7+
**Requirements**
8+
9+
Python 3.10+ with the following packages:
10+
11+
| Package | Purpose | Install |
12+
|--------------|----------------------------------------|--------------------------|
13+
| `requests` | Gemini API and GitHub API calls | `pip install requests` |
14+
| `sqlite-vec` | Vector similarity search in SQLite | `pip install sqlite-vec` |
15+
16+
Standard library modules used: `sqlite3`, `json`, `re`, `pathlib`, `dataclasses`, `concurrent.futures`, `argparse`, `array`, `os`, `sys`.
17+
18+
Quick setup:
19+
```sh
20+
python3 -m venv .venv
21+
source .venv/bin/activate
22+
pip install requests sqlite-vec
23+
```
24+
25+
You also need a `GOOGLE_API_KEY` for Gemini embedding and synthesis. Optionally a `GITHUB_TOKEN` for PR ingestion. See **Credential Handling** below.
26+
727
**Purpose**
828
This package exists to answer questions about a repository using the repository itself as the source of truth.
929

@@ -12,6 +32,21 @@ This package exists to answer questions about a repository using the repository
1232
- It synthesizes a final answer from retrieved context instead of answering from model memory alone.
1333
- It can assess whether the original question is vague, broad, or underspecified, and suggest a better question.
1434

35+
**Use Cases**
36+
37+
*Standalone*
38+
39+
- **Codebase Q&A** — Ask natural-language questions about a repo and get answers grounded in the actual source, docs, and changelog rather than model memory. Useful for onboarding, auditing, or navigating a large codebase.
40+
- **PR-aware context** — With GitHub PR ingestion enabled, answers can draw on merged PR descriptions and discussion, surfacing decisions that live in review threads rather than code comments.
41+
- **Question quality feedback** — The system assesses whether a question is vague or underspecified, and suggests a sharper reformulation. Useful for building self-service Q&A interfaces where users may not know how to ask.
42+
43+
*Integrated with the WPCC scanner*
44+
45+
- **Finding triage and explanation** — After the scanner flags issues, pipe findings into ask_self to get repo-grounded context: "Is this `eval()` usage intentional?" or "How does this repo typically handle `$wpdb->prepare()`?" Cuts triage time by distinguishing known patterns from real concerns.
46+
- **False positive reduction** — The scanner's grep-based detection cannot see multi-line sanitization or architectural intent. ask_self can retrieve chunks showing a flagged pattern is wrapped in a mitigation elsewhere, acting as a semantic second opinion on structural matches.
47+
- **Pattern gap discovery** — "What risks exist in this repo that the scanner doesn't currently check for?" is a question ask_self can attempt since it has both the pattern library and the source indexed together.
48+
- **Repo-grounded remediation** — Instead of generic remediation advice, ask_self can show how the specific repo already handles a given pattern, producing fix suggestions that match the existing codebase conventions.
49+
1550
**Core Files**
1651
- `ask_self_ingest.py`: builds the local sqlite-vec index from the configured corpus.
1752
- `ask_self_query.py`: embeds a query, performs KNN retrieval, then calls Gemini Pro for structured synthesis.
Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
Integration ideas worth considering:
2+
3+
The scanner is grep-based and structural; ask_self is semantic. That's a complementary pairing, not a redundant one:
4+
5+
Finding explainer / triage assistant — Scanner flags 30 issues across a site. Feed those findings into ask_self as a query: "Why does this repo use $wpdb->prepare() this way?" or "Is this eval() usage intentional?" It can pull from PRs, changelogs, and docs to tell you whether a finding is a known pattern or a real concern. Cuts triage time.
6+
7+
False positive reducer — Scanner's grep can't see multi-line sanitization or architectural intent. ask_self can retrieve chunks showing that a flagged pattern is wrapped in a mitigation elsewhere. This is the "secondary evaluator" you're intuiting — a semantic second opinion on structural matches.
8+
9+
Pattern gap discovery — "What risks exist in this repo that the scanner doesn't check for?" is a question ask_self can actually attempt, since it has the pattern library and the source indexed together.
10+
11+
Remediation grounded in repo conventions — Instead of generic "use prepared statements" advice, ask_self can show how this specific repo already handles that pattern elsewhere.
12+
13+
AI-DDTK - POTENTIAL INTEGRATION
14+
https://github.com/Hypercart-Dev-Tools/AI-DDTK-Fix-Iterate-Loop/
15+
16+
## Review System Instruction Layer (Sketch)
17+
18+
This layer would live alongside the existing Q&A layers in `ask_self_system_instructions.json`.
19+
It is designed to be invoked by an AI agent inside a Fix-Iterate Loop (see AI-DDTK)
20+
rather than by a human asking freeform questions.
21+
22+
### System instruction additions
23+
24+
```json
25+
{
26+
"system_layers": {
27+
"...existing layers unchanged...": "...",
28+
29+
"review_system": "You are a code reviewer with access to the full indexed codebase. You are given a diff or file excerpt. Your job is to identify violations of DRY, SOLID, and repo-specific conventions by comparing the submitted code against existing patterns in the retrieved context. Do not invent violations — only flag issues where the retrieved context provides concrete evidence of an existing pattern, utility, or abstraction that the new code duplicates or contradicts. If you find no violations, say so explicitly.",
30+
31+
"review_severity_system": "Classify each violation as: MUST_FIX (blocks merge — duplication of existing utility, broken interface contract, security regression), SHOULD_FIX (strong suggestion — could use existing abstraction, naming inconsistency with established conventions), or CONSIDER (style preference — alternative exists but current approach is acceptable). Never inflate severity."
32+
}
33+
}
34+
```
35+
36+
### Review response contract
37+
38+
```json
39+
{
40+
"review_response_contract": {
41+
"format": "json",
42+
"fields": {
43+
"verdict": "one of: clean, has_violations",
44+
"violations": [
45+
{
46+
"principle": "DRY | SRP | OCP | LSP | ISP | DIP | CONVENTION",
47+
"severity": "MUST_FIX | SHOULD_FIX | CONSIDER",
48+
"location": "file path and line range in the submitted diff",
49+
"description": "what the violation is",
50+
"existing_pattern": "file path and excerpt from the indexed codebase that the new code duplicates or contradicts",
51+
"suggested_fix": "concrete refactoring suggestion grounded in the existing pattern"
52+
}
53+
],
54+
"summary": "one-sentence overall assessment",
55+
"sources_consulted": ["array of retrieved chunk sources"]
56+
}
57+
}
58+
}
59+
```
60+
61+
### How this fits the AI-DDTK Fix-Iterate Loop
62+
63+
The review layer slots into Step 3 (Verify) of the Fix-Iterate Loop.
64+
The AI agent is the loop controller; ask_self is the verification oracle.
65+
66+
```
67+
Fix-Iterate Loop (AI-DDTK) ask_self (WPCC)
68+
───────────────────────── ───────────────
69+
1. Agent writes/modifies code
70+
71+
2. Agent extracts diff:
72+
git diff --cached > /tmp/diff
73+
74+
3. Verify via ask_self: ◄── ask_self_review.py --diff /tmp/diff
75+
--checks dry,srp,convention
76+
--harness-config ask_self_harness.json
77+
--json
78+
79+
Returns structured verdict:
80+
{
81+
"verdict": "has_violations",
82+
"violations": [
83+
{
84+
"principle": "DRY",
85+
"severity": "MUST_FIX",
86+
"location": "src/utils/parse.py:42-58",
87+
"existing_pattern": "src/lib/parser.py:10-25 — parse_input() already does this",
88+
"suggested_fix": "Import and call parse_input() from src/lib/parser.py"
89+
}
90+
]
91+
}
92+
93+
4. Agent reads verdict:
94+
- "clean" → done, exit loop
95+
- "has_violations" → apply
96+
suggested_fix, loop to step 1
97+
98+
5. Guardrails (from AI-DDTK):
99+
- 5 failed iterations → stop
100+
- 10 total iterations → stop
101+
- Confidence trending down → stop
102+
```
103+
104+
### Iteration template (extends AI-DDTK format)
105+
106+
```
107+
ITERATION N:
108+
1. What I changed: [describe code change]
109+
2. Diff: [file paths and line counts]
110+
3. ask_self command: ask_self_review.py --diff /tmp/diff --checks dry,srp --json
111+
4. Expected result: {"verdict": "clean"}
112+
5. Actual result: [paste structured verdict]
113+
6. Status: CLEAN / N violations (X MUST_FIX, Y SHOULD_FIX, Z CONSIDER)
114+
7. Next action: [apply suggested fixes or stop]
115+
116+
META-REFLECTION (Iteration N):
117+
Confidence: [1-10]
118+
Violations trending: [up/down/flat]
119+
Risk: [LOW / MEDIUM / HIGH]
120+
Continue? [YES / NO / ASK_HUMAN]
121+
```
122+
123+
### MCP tool definition (for VS Code agent integration)
124+
125+
For Claude Code or other VS Code agents to call ask_self natively in the loop,
126+
expose it as an MCP tool. Minimal definition:
127+
128+
```json
129+
{
130+
"tools": [
131+
{
132+
"name": "ask_self_review",
133+
"description": "Review a code diff against the indexed codebase for DRY, SOLID, and convention violations. Returns structured violations with evidence from existing code.",
134+
"inputSchema": {
135+
"type": "object",
136+
"properties": {
137+
"diff": {
138+
"type": "string",
139+
"description": "Unified diff text to review"
140+
},
141+
"checks": {
142+
"type": "array",
143+
"items": {"type": "string", "enum": ["dry", "srp", "ocp", "lsp", "isp", "dip", "convention"]},
144+
"description": "Which principles to check. Defaults to all."
145+
},
146+
"harness_config": {
147+
"type": "string",
148+
"description": "Path to ask_self_harness.json"
149+
}
150+
},
151+
"required": ["diff"]
152+
}
153+
},
154+
{
155+
"name": "ask_self_query",
156+
"description": "Ask a natural-language question about the indexed codebase. Returns a grounded answer with sources.",
157+
"inputSchema": {
158+
"type": "object",
159+
"properties": {
160+
"question": {
161+
"type": "string",
162+
"description": "The question to ask"
163+
},
164+
"harness_config": {
165+
"type": "string",
166+
"description": "Path to ask_self_harness.json"
167+
}
168+
},
169+
"required": ["question"]
170+
}
171+
}
172+
]
173+
}
174+
```
175+
176+
The MCP server would be a thin wrapper: parse input → call ask_self CLI → return JSON.
177+
The `--json` output contract already exists — no new serialization needed.
178+
179+
### What lives where
180+
181+
| Concern | Home | Why |
182+
|---------|------|-----|
183+
| Loop control, guardrails, iteration template | AI-DDTK | Generic agent pattern, not ask_self-specific |
184+
| Review system instructions, severity rules | ask_self_system_instructions.json | Answer-behavior config, not code |
185+
| Diff parsing, review-mode query construction | ask_self_review.py (new) | Engine code, repo-agnostic |
186+
| MCP server adapter | AI-DDTK or standalone | Integration glue, should live near the agent |
187+
| Repo-specific convention rules | ask_self_harness.json per repo | Policy, not code |
188+
189+
### Implementation order (suggested)
190+
191+
1. Add `review_system` and `review_severity_system` layers to ask_self_system_instructions.json
192+
2. Add `review_response_contract` to the same file
193+
3. Write `ask_self_review.py` — accepts a diff, embeds it, queries with review instructions, returns structured verdict
194+
4. Test manually: `git diff HEAD~1 | python3 ask_self_review.py --json`
195+
5. Wrap as MCP tool in AI-DDTK
196+
6. Wire into Fix-Iterate Loop template as the verify step

0 commit comments

Comments
 (0)