chore: checkpoint forensic fixes and docs alignment

Author: deepakdgupta1

README.md

@@ -47,7 +47,8 @@ knowcode context "MyClass.important_method"
 # 4. Export documentation
 knowcode export -o docs/
 
-# 5. Build semantic search index
+# 5. (Optional) Build semantic search index explicitly
+#    `analyze` also attempts indexing; this command is useful for rebuilds.
 knowcode index src/
 
 # 6. Perform semantic search
@@ -101,12 +102,12 @@ knowcode query deps "PythonParser" --json
 Generate a context bundle for an entity (ready for AI consumption).
 
 ```bash
-knowcode context <entity> [--store <path>] [--max-chars <n>]
+knowcode context <entity> [--store <path>] [--max-tokens <n>]
 ```
 
 **Example:**
 ```bash
-knowcode context "GraphBuilder.build_from_directory" --max-chars 4000
+knowcode context "GraphBuilder.build_from_directory" --max-tokens 4000
 ```
 
 ### `export`
@@ -267,7 +268,7 @@ KnowCode enables token-efficient IDE agent workflows. When your IDE agent needs
 
 ## Supported Languages (MVP)
 
-- **Python** (.py) - Full AST parsing (Supports Python 3.9 - 3.12)
+- **Python** (.py) - Full AST parsing (Supports Python 3.10 - 3.12)
 - **JavaScript / TypeScript** (.js, .ts) - Classes, functions, imports (via tree-sitter)
 - **Java** (.java) - Classes, methods, imports, inheritance (via tree-sitter)
 - **Markdown** (.md) - Document structure with heading hierarchy

README_MCP.md

@@ -46,7 +46,7 @@ KnowCode MCP Server returns:
     ...
 }
     ↓
-Agent checks: sufficiency_score >= 0.88?
+Agent checks: sufficiency_score >= 0.8?
     ↓
 YES → Answer from context_text only (no external LLM)
 NO  → Use external LLM (Claude Sonnet 4.5)
@@ -82,7 +82,7 @@ source .venv/bin/activate
 knowcode analyze . -o .
 ```
 
-**Note:** This automatically rebuilds both the knowledge store AND the semantic index!
+**Note:** This rebuilds the knowledge store and attempts semantic indexing. If indexing is skipped, run `knowcode index .` after configuring embeddings.
 
 ## 🐛 Troubleshooting
 
@@ -107,9 +107,9 @@ knowcode analyze . -o .
    ls -la knowcode_index/
    ```
 
-2. If missing, re-run analyze (it will rebuild the index):
+2. If missing, run a dedicated index build:
    ```bash
-   knowcode analyze . -o .
+   knowcode index . --output knowcode_index
    ```
 
 3. Increase token budget in `.agent/context.md`:
@@ -120,7 +120,7 @@ knowcode analyze . -o .
 ## 📊 Success Metrics
 
 After setup, you should see:
-- ✅ 70%+ queries with `sufficiency_score >= 0.88`
+- ✅ 70%+ queries with `sufficiency_score >= 0.8`
 - ✅ Faster responses for codebase questions
 - ✅ 50%+ reduction in external LLM token usage
 - ✅ Accurate answers from local context
@@ -134,8 +134,8 @@ After setup, you should see:
 ## 🎓 Key Concepts
 
 **Sufficiency Score**: Confidence that retrieved context is enough to answer the query
-- `>= 0.88` → Answer locally
-- `< 0.88` → Use external LLM
+- `>= 0.8` → Answer locally
+- `< 0.8` → Use external LLM
 
 **Retrieval Modes**:
 - **Semantic**: Uses embeddings + vector search (better)

docs/MCP_SETUP.md

@@ -24,8 +24,8 @@ Knowledge Store + Semantic Index
 Context Bundle (with sufficiency_score)
     ↓
 Agent Decision:
-    • If sufficiency_score >= 0.88 → Answer from context only
-    • If sufficiency_score < 0.88 → Use external LLM
+    • If sufficiency_score >= 0.8 → Answer from context only
+    • If sufficiency_score < 0.8 → Use external LLM
 ```
 
 ## Prerequisites
@@ -66,9 +66,9 @@ Agent Decision:
 ```markdown
 Always call the tool retrieve_context_for_query before answering.
 Use task_type=auto, max_tokens=3000, limit_entities=3, expand_deps=true.
-If sufficiency_score >= 0.88 and context_text is non-empty, answer ONLY from context_text.
+If sufficiency_score >= 0.8 and context_text is non-empty, answer ONLY from context_text.
 Do not call other tools or request more context.
-If sufficiency_score < 0.88, then proceed with a full LLM answer.
+If sufficiency_score < 0.8, then proceed with a full LLM answer.
 ```
 
 This ensures the agent follows the local-first workflow.
@@ -83,13 +83,13 @@ source .venv/bin/activate
 knowcode analyze . -o .
 ```
 
-This **automatically**:
+This command:
 - ✅ Parses the codebase and builds the knowledge graph
 - ✅ Creates `knowcode_knowledge.json` with all entities and relationships
-- ✅ **Builds the semantic index** at `knowcode_index/` with vector embeddings
-- ✅ Reports statistics including indexed chunk count
+- ✅ Attempts to build the semantic index at `knowcode_index/`
+- ✅ Reports indexed chunk count (or indexing warning if embedding setup is unavailable)
 
-**Note:** You do NOT need to run a separate `knowcode index` command - it's built-in!
+**Note:** If semantic indexing is skipped (e.g., missing embedding API credentials), run `knowcode index .` after configuring embeddings.
 
 ### Step 3: Configure MCP Server
 
@@ -131,8 +131,8 @@ How does search work in KnowCode?
 **Expected behavior:**
 1. Agent calls `retrieve_context_for_query`
 2. Returns context with `sufficiency_score`
-3. If score >= 0.88, answers from context only
-4. If score < 0.88, uses external LLM
+3. If score >= 0.8, answers from context only
+4. If score < 0.8, uses external LLM
 
 ### Test 3: Verify All Tools
 
@@ -179,7 +179,7 @@ Answering from local context only.
 ### Issue 3: Low Sufficiency Scores
 
 **Symptoms:**
-- `sufficiency_score` always < 0.88
+- `sufficiency_score` always < 0.8
 - Agent always uses external LLM
 
 **Solutions:**
@@ -196,7 +196,7 @@ Answering from local context only.
 
 **Solutions:**
 1. Verify `knowcode_index/` directory exists
-2. Check `knowcode_index/manifest.json` exists
+2. Check `knowcode_index/index_manifest.json` exists
 3. Verify embedding model is configured in `aimodels.yaml`
 4. Check API keys for embedding provider (VoyageAI, OpenAI, etc.)
 
@@ -276,13 +276,13 @@ Use task_type=auto, max_tokens=3000, limit_entities=3, expand_deps=true.
 ### Check Token Savings
 
 The MCP server logs show:
-- Queries answered locally (sufficiency >= 0.88)
-- Queries sent to external LLM (sufficiency < 0.88)
+- Queries answered locally (sufficiency >= 0.8)
+- Queries sent to external LLM (sufficiency < 0.8)
 - Token counts for each response
 
 ### Metrics to Track
 
-1. **Sufficiency Score Distribution**: Aim for >70% of queries with score >= 0.88
+1. **Sufficiency Score Distribution**: Aim for >70% of queries with score >= 0.8
 2. **Token Consumption**: Compare before/after MCP integration
 3. **Response Quality**: Verify local answers are accurate
 

docs/index.md

@@ -47,7 +47,8 @@ knowcode context "MyClass.important_method"
 # 4. Export documentation
 knowcode export -o docs/
 
-# 5. Build semantic search index
+# 5. (Optional) Build semantic search index explicitly
+#    `analyze` also attempts indexing; this command is useful for rebuilds.
 knowcode index src/
 
 # 6. Perform semantic search
@@ -98,12 +99,12 @@ knowcode query deps "PythonParser" --json
 Generate a context bundle for an entity (ready for AI consumption).
 
 ```bash
-knowcode context <entity> [--store <path>] [--max-chars <n>]
+knowcode context <entity> [--store <path>] [--max-tokens <n>]
 ```
 
 **Example:**
 ```bash
-knowcode context "GraphBuilder.build_from_directory" --max-chars 4000
+knowcode context "GraphBuilder.build_from_directory" --max-tokens 4000
 ```
 
 ### `export`
@@ -210,17 +211,18 @@ knowcode ask "How does the graph builder work?"
 Start an MCP (Model Context Protocol) server for IDE agent integration.
 
 ```bash
-knowcode mcp-server [--store <path>]
+knowcode mcp-server [--store <path>] [--config <path>]
 ```
 
 **Tools Exposed:**
 - `search_codebase` - Search for code entities by name
 - `get_entity_context` - Get detailed context for an entity
 - `trace_calls` - Trace call graph (callers/callees) with depth
+- `retrieve_context_for_query` - Unified query→retrieval→context bundle
 
 ## Supported Languages (MVP)
 
-- **Python** (.py) - Full AST parsing (Supports Python 3.9 - 3.12)
+- **Python** (.py) - Full AST parsing (Supports Python 3.10 - 3.12)
 - **JavaScript / TypeScript** (.js, .ts) - Classes, functions, imports (via tree-sitter)
 - **Java** (.java) - Classes, methods, imports, inheritance (via tree-sitter)
 - **Markdown** (.md) - Document structure with heading hierarchy

scripts/mypy_autofix/fix_colons.py

@@ -1,4 +1,3 @@
-import os
 from pathlib import Path
 
 def main():

scripts/mypy_autofix/fix_last_mypy.py

@@ -1,4 +1,3 @@
-import re
 from collections import defaultdict
 from pathlib import Path
 
@@ -16,7 +15,8 @@ def main():
                 errors_by_file[filepath].append((lineno, msg))
 
     for filepath, file_errors in errors_by_file.items():
-        if not Path(filepath).exists(): continue
+        if not Path(filepath).exists():
+            continue
 
         file_lines = Path(filepath).read_text().splitlines()
 
@@ -27,7 +27,8 @@ def main():
 
         for lineno in sorted(line_actions.keys(), reverse=True):
             idx = lineno - 1
-            if idx < 0 or idx >= len(file_lines): continue
+            if idx < 0 or idx >= len(file_lines):
+                continue
 
             line = file_lines[idx]
             msgs = line_actions[lineno]

scripts/mypy_autofix/fix_mypy.py

@@ -1,5 +1,3 @@
-import re
-import sys
 from collections import defaultdict
 from pathlib import Path
 
@@ -34,15 +32,17 @@ def main():
             Path(fn).write_text(content)
 
     for filepath, file_errors in errors_by_file.items():
-        if not Path(filepath).exists(): continue
+        if not Path(filepath).exists():
+            continue
 
         file_lines = Path(filepath).read_text().splitlines()
 
         # Sort in reverse to apply changes from bottom up (if inserting lines, though we are mostly appending or modifying lines)
         # For appending # type: ignore, order doesn't impact line numbers
         for lineno, msg in sorted(file_errors, key=lambda x: x[0], reverse=True):
             idx = lineno - 1
-            if idx < 0 or idx >= len(file_lines): continue
+            if idx < 0 or idx >= len(file_lines):
+                continue
 
             line = file_lines[idx]
 

scripts/mypy_autofix/fix_mypy2.py

@@ -1,4 +1,3 @@
-import re
 from collections import defaultdict
 from pathlib import Path
 
@@ -16,14 +15,16 @@ def main():
                 errors_by_file[filepath].append((lineno, msg))
 
     for filepath, file_errors in errors_by_file.items():
-        if not Path(filepath).exists(): continue
+        if not Path(filepath).exists():
+            continue
 
         file_lines = Path(filepath).read_text().splitlines()
 
         # apply changes bottom-up
         for lineno, msg in sorted(file_errors, key=lambda x: x[0], reverse=True):
             idx = lineno - 1
-            if idx < 0 or idx >= len(file_lines): continue
+            if idx < 0 or idx >= len(file_lines):
+                continue
 
             line = file_lines[idx]
 

scripts/mypy_autofix/fix_syntax.py

@@ -1,4 +1,3 @@
-import sys
 from pathlib import Path
 
 def main():