Skip to content

Commit e97df1b

Browse files
docs: add dogfooding report for v2.2.0
Tested all 22 CLI commands against the codegraph repo itself. Found 4 bugs: structure "." filter, stale embeddings after rebuild, default embed model requires HF auth, cross-language false positive.
1 parent 68e274e commit e97df1b

1 file changed

Lines changed: 82 additions & 0 deletions

File tree

generated/DOGFOOD_REPORT_v2.2.0.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Dogfooding Report: @optave/codegraph@2.2.0
2+
3+
**Date:** 2026-02-23
4+
**Tested against:** codegraph repo itself (92 files, 527 nodes)
5+
**Engine:** Native v0.1.0 (auto)
6+
7+
## Working Commands (20/22)
8+
9+
| Command | Status | Notes |
10+
|---------|--------|-------|
11+
| `build` | PASS | Native engine, 92 files, 527 nodes, 526 edges |
12+
| `query` | PASS | Correct callers/callees for `buildGraph` |
13+
| `impact` | PASS | 13 transitive deps for `src/db.js` |
14+
| `map` | PASS | Clean module overview |
15+
| `stats` | PASS | Full graph health overview |
16+
| `deps` | PASS | Correct imports/imported-by |
17+
| `fn` | PASS | Function-level call chain |
18+
| `fn-impact` | PASS | 3 transitive dependents |
19+
| `context` | PASS | Full source, deps, callers, tests |
20+
| `explain` (file) | PASS | Clean structural summary |
21+
| `explain` (function) | PASS | Calls, callers, tests |
22+
| `where` | PASS | Fast symbol lookup |
23+
| `diff-impact` | PASS | 11 changed functions, 44 callers affected |
24+
| `cycles` | PASS | 1 cycle: queries.js <-> cycles.js |
25+
| `hotspots` | PASS | Correct fan-in rankings |
26+
| `export` (DOT/Mermaid/JSON) | PASS | All 3 formats work |
27+
| `info` | PASS | Correct version + engine info |
28+
| `models` | PASS | Lists all 7 models |
29+
| `registry` | PASS | list/add/remove/prune subcommands |
30+
| `watch` | PASS | Starts, watches for changes |
31+
| `mcp` | PASS | Server initializes correctly via JSON-RPC |
32+
33+
## Bugs Found
34+
35+
### 1. `structure .` returns empty results (Medium severity)
36+
37+
- `codegraph structure .` → "No directory structure found"
38+
- `codegraph structure` (no arg) → works perfectly (18 directories)
39+
- `codegraph structure src` → works correctly
40+
41+
**Root cause:** In `structureData()` (`src/structure.js`), passing `.` as the `directory` filter normalizes to `"."` and then filters `d.name === '.' || d.name.startsWith('./')` — which matches nothing since directory names stored in the DB are relative paths like `src`, `tests`, etc.
42+
43+
**Fix:** Treat `.` (or current dir equivalent) as `null`/no filter in `structureData()`.
44+
45+
### 2. Stale embeddings after rebuild (Medium severity)
46+
47+
- After an incremental `build`, embedding `node_id`s become orphaned (e.g. old IDs in 3077-range, new IDs in 4335-range)
48+
- `search` returns 0 results even at `--min-score 0.05` because no embeddings join to current nodes
49+
- Verified: 310 embeddings existed but 0 matched any node in the `nodes` table
50+
51+
**Root cause:** `build` deletes and re-inserts nodes (getting new auto-increment IDs) but does not invalidate or rebuild embeddings.
52+
53+
**Fix:** Either preserve node IDs across rebuilds, invalidate embeddings when node IDs change, or warn the user to re-run `embed`.
54+
55+
### 3. `embed` default model requires HuggingFace auth (Medium severity)
56+
57+
- `codegraph embed .` crashes with `Error: Unauthorized access to file` for the default `jina-code` model
58+
- The Jina model is gated on HuggingFace and requires an `HF_TOKEN` environment variable
59+
- `codegraph embed . --model minilm` works fine (public model)
60+
- The error is an unhandled exception with a full stack trace — not user-friendly
61+
62+
**Fix:** Either default to a public model (e.g. `minilm`), auto-fallback to `minilm` on auth failure, or catch the error and provide a clear message with instructions.
63+
64+
### 4. Cross-language false positive in export (Low severity)
65+
66+
- One low-confidence (0.3) call edge: `main` (build.rs) → `setup` (tests/unit/structure.test.js)
67+
- Shows up in Mermaid/DOT exports as a spurious connection
68+
- Only 1 instance found across the entire graph
69+
70+
**Fix:** Export commands could support a `--min-confidence` filter, or the default export could exclude edges below a threshold (e.g. 0.5).
71+
72+
## `--no-tests` Flag
73+
74+
Tested on `stats` and `map` — both correctly filter out test files:
75+
- `stats --no-tests`: 427 nodes (vs 527 total), 59 files (vs 92)
76+
- `map --no-tests`: excludes test files from ranking
77+
78+
## Embedding & Search
79+
80+
- `embed --model minilm` successfully generated 392 embeddings (384d)
81+
- `search "build graph"` returned 15 results after fresh embeddings (top hit: 37.9% `test_triangle_cycle`)
82+
- Search quality is reasonable but not ideal — `buildGraph` itself didn't appear in results for "build graph"

0 commit comments

Comments
 (0)