|
| 1 | +# Phase 1: Extract @agents/kg (Knowledge Graph) |
| 2 | + |
| 3 | +## Goal |
| 4 | + |
| 5 | +Create a new `packages/kg/` workspace package containing all knowledge-graph modules: graph construction, schema validation, file locking, embedding, search indexing, and markdown chunking. |
| 6 | + |
| 7 | +## Non-Goals |
| 8 | + |
| 9 | +- Moving Python files (`chunker.py`, `embedder.py`) -- they stay in `cli/src/lib/`. |
| 10 | +- Changing chunking algorithms or embedding behavior. |
| 11 | +- Adding new KG features. |
| 12 | +- Wiring KG into the SDK catalog system (that is a future concern beyond this migration). |
| 13 | + |
| 14 | +## Prerequisites |
| 15 | + |
| 16 | +- None. This phase has no blockers -- KG modules are self-contained and only consumed by CLI commands and the graph-viewer server. |
| 17 | + |
| 18 | +## Files |
| 19 | + |
| 20 | +### Create (new package scaffold) |
| 21 | + |
| 22 | +| File | Description | |
| 23 | +|------|-------------| |
| 24 | +| `packages/kg/package.json` | Package manifest with KG-specific deps | |
| 25 | +| `packages/kg/tsconfig.json` | TypeScript config extending root | |
| 26 | +| `packages/kg/src/index.ts` | Barrel re-exports | |
| 27 | +| `packages/kg/src/graph.ts` | Graph construction (from `cli/src/lib/graph.ts`, 202 lines) | |
| 28 | +| `packages/kg/src/graph-schema.ts` | JSON Schema validation (from `cli/src/lib/graph-schema.ts`, 121 lines) | |
| 29 | +| `packages/kg/src/graph-lock.ts` | File-based graph locking (from `cli/src/lib/graph-lock.ts`, 158 lines) | |
| 30 | +| `packages/kg/src/embedder.ts` | Ollama embedding client (from `cli/src/lib/embedder.ts`, 59 lines) | |
| 31 | +| `packages/kg/src/meilisearch.ts` | Meilisearch client wrapper (from `cli/src/lib/meilisearch.ts`, 478 lines) | |
| 32 | +| `packages/kg/src/chunker.ts` | Markdown chunking + frontmatter (from `cli/src/lib/chunker.ts`, 364 lines) | |
| 33 | +| `packages/kg/test/chunker.test.ts` | Unit tests for chunker | |
| 34 | +| `packages/kg/test/graph.test.ts` | Unit tests for graph construction | |
| 35 | +| `packages/kg/test/embedder.test.ts` | Unit tests for embedder | |
| 36 | + |
| 37 | +### Modify (update imports) |
| 38 | + |
| 39 | +| File | Change | |
| 40 | +|------|--------| |
| 41 | +| `packages/cli/src/commands/kg.ts` | Change `from '../lib/chunker'` to `from '@agents/kg/chunker'`, etc. | |
| 42 | +| `packages/cli/src/server/graph-viewer/routes/*.ts` | Change `from '../../lib/graph'` to `from '@agents/kg/graph'`, etc. | |
| 43 | +| `packages/cli/package.json` | Add `"@agents/kg": "workspace:*"` dependency | |
| 44 | +| `root package.json` | Add `packages/kg` to workspaces array | |
| 45 | + |
| 46 | +### Delete (after consumers updated) |
| 47 | + |
| 48 | +| File | Lines | |
| 49 | +|------|-------| |
| 50 | +| `packages/cli/src/lib/graph.ts` | 202 | |
| 51 | +| `packages/cli/src/lib/graph-schema.ts` | 121 | |
| 52 | +| `packages/cli/src/lib/graph-lock.ts` | 158 | |
| 53 | +| `packages/cli/src/lib/embedder.ts` | 59 | |
| 54 | +| `packages/cli/src/lib/meilisearch.ts` | 478 | |
| 55 | +| `packages/cli/src/lib/chunker.ts` | 364 | |
| 56 | + |
| 57 | +## Steps |
| 58 | + |
| 59 | +- [ ] **1.1** Create `packages/kg/` directory structure. |
| 60 | +- [ ] **1.2** Write `packages/kg/package.json`: |
| 61 | + |
| 62 | + ```json |
| 63 | + { |
| 64 | + "name": "@agents/kg", |
| 65 | + "version": "0.1.0", |
| 66 | + "private": true, |
| 67 | + "type": "module", |
| 68 | + "exports": { |
| 69 | + ".": "./src/index.ts", |
| 70 | + "./*": "./src/*.ts" |
| 71 | + }, |
| 72 | + "dependencies": { |
| 73 | + "@agents/core": "workspace:*", |
| 74 | + "ajv": "^8.18.0", |
| 75 | + "ajv-formats": "^3.0.1", |
| 76 | + "graphology": "^0.26.0", |
| 77 | + "graphology-layout": "^0.6.1", |
| 78 | + "graphology-layout-forceatlas2": "^0.10.1", |
| 79 | + "js-yaml": "^4.1.0", |
| 80 | + "meilisearch": "^0.56.0", |
| 81 | + "ollama": "^0.5.0" |
| 82 | + } |
| 83 | + } |
| 84 | + ``` |
| 85 | + |
| 86 | +- [ ] **1.3** Write `packages/kg/tsconfig.json` extending root tsconfig. |
| 87 | +- [ ] **1.4** Copy the 6 source files from `cli/src/lib/` to `packages/kg/src/`. |
| 88 | +- [ ] **1.5** Update imports in each copied file: |
| 89 | + - `chunker.ts`: `@agents/core/runtime` stays (cross-package import is fine). |
| 90 | + - `meilisearch.ts`: `@agents/core/types` stays. |
| 91 | + - `graph.ts`, `graph-schema.ts`, `graph-lock.ts`: No external package imports to change (only node built-ins and direct deps). |
| 92 | + - `embedder.ts`: Only imports `ollama` -- no changes. |
| 93 | +- [ ] **1.6** Replace `CliError` with a KG-specific error type in meilisearch.ts: |
| 94 | + |
| 95 | + ```typescript |
| 96 | + // Before |
| 97 | + import { CliError } from '@agents/core/types' |
| 98 | + // After |
| 99 | + import { CliError } from '@agents/core/types' // Keep CliError for now -- it's from core, not CLI |
| 100 | + ``` |
| 101 | +
|
| 102 | + Note: `CliError` is actually defined in `@agents/core/types`, not in the CLI package. It can be used anywhere. No change needed in Phase 1; error code rename happens in Phase 6 cleanup. |
| 103 | +- [ ] **1.7** Write `packages/kg/src/index.ts` barrel: |
| 104 | +
|
| 105 | + ```typescript |
| 106 | + export * from './chunker' |
| 107 | + export * from './embedder' |
| 108 | + export * from './graph' |
| 109 | + export * from './graph-lock' |
| 110 | + export * from './graph-schema' |
| 111 | + export * from './meilisearch' |
| 112 | + ``` |
| 113 | + |
| 114 | +- [ ] **1.8** Add `"@agents/kg": "workspace:*"` to `packages/cli/package.json` dependencies. |
| 115 | +- [ ] **1.9** Add `"packages/kg"` to root `package.json` workspaces. |
| 116 | +- [ ] **1.10** Run `bun install` from root to link the new workspace package. |
| 117 | +- [ ] **1.11** Update `packages/cli/src/commands/kg.ts` imports: |
| 118 | + |
| 119 | + ```typescript |
| 120 | + // Before |
| 121 | + import { chunkMarkdown, parseFrontmatter } from '../lib/chunker' |
| 122 | + import { embed } from '../lib/embedder' |
| 123 | + import { createClient, searchKeyword } from '../lib/meilisearch' |
| 124 | + // After |
| 125 | + import { chunkMarkdown, parseFrontmatter } from '@agents/kg/chunker' |
| 126 | + import { embed } from '@agents/kg/embedder' |
| 127 | + import { createClient, searchKeyword } from '@agents/kg/meilisearch' |
| 128 | + ``` |
| 129 | + |
| 130 | +- [ ] **1.12** Update `packages/cli/src/server/graph-viewer/routes/*.ts` imports similarly. |
| 131 | +- [ ] **1.13** Update any other CLI files that import from the 6 moved modules (search with `grep`). |
| 132 | +- [ ] **1.14** Write tests for `packages/kg/test/`: |
| 133 | + - `chunker.test.ts`: Test `parseFrontmatter`, `chunkMarkdown` with sample markdown. |
| 134 | + - `graph.test.ts`: Test graph construction from mock data. |
| 135 | + - `embedder.test.ts`: Test embed function signature (mock Ollama). |
| 136 | +- [ ] **1.15** Run full test suite: |
| 137 | + - `bun test --cwd packages/kg` -- all new tests pass. |
| 138 | + - `bun test --cwd packages/cli` -- 1684 pass / 10 fail (unchanged). |
| 139 | +- [ ] **1.16** Delete the 6 source files from `cli/src/lib/`. |
| 140 | +- [ ] **1.17** Run CLI test suite again to confirm no regressions. |
| 141 | + |
| 142 | +## Acceptance Criteria |
| 143 | + |
| 144 | +1. `packages/kg/` exists with `package.json`, `tsconfig.json`, `src/`, and `test/`. |
| 145 | +2. `bun test --cwd packages/kg` passes with at least 6 test cases. |
| 146 | +3. `bun test --cwd packages/cli` maintains 1684 pass / 10 fail. |
| 147 | +4. No file in `packages/cli/src/lib/` imports from `packages/kg/src/` via relative path. |
| 148 | +5. `packages/cli/src/lib/` no longer contains: `graph.ts`, `graph-schema.ts`, `graph-lock.ts`, `embedder.ts`, `meilisearch.ts`, `chunker.ts`. |
| 149 | +6. `bun run packages/cli/src/bin/agents.ts kg --help` still works. |
| 150 | + |
| 151 | +## Failure Criteria |
| 152 | + |
| 153 | +- **Stop if:** Moving `chunker.ts` breaks `manifest.ts` (Phase 2 dependency). In that case, leave a re-export shim in `cli/src/lib/chunker.ts`: |
| 154 | + |
| 155 | + ```typescript |
| 156 | + export { parseFrontmatter, chunkMarkdown, type Chunk, type ParsedMarkdown } from '@agents/kg/chunker' |
| 157 | + ``` |
| 158 | + |
| 159 | + This shim allows Phase 1 to complete independently. Phase 2 will update `manifest.ts` to import from `@agents/kg/chunker` or `@agents/sdk/context/frontmatter` directly. |
| 160 | + |
| 161 | +- **Stop if:** Graph-viewer server fails to start after import changes. Debug the import path resolution in Bun's module system before proceeding. |
| 162 | + |
| 163 | +## Fallback Logic |
| 164 | + |
| 165 | +1. If any moved module fails to resolve via `@agents/kg/*`, check that `packages/kg/package.json` exports map is correct and that `bun install` was re-run. |
| 166 | +2. If Ollama types cause issues in `embedder.ts`, add `ollama` to `@agents/kg` devDependencies and use dynamic import. |
| 167 | +3. If test count drops, diff the test output against the baseline and identify which tests regressed -- do not proceed to Phase 2 until resolved. |
| 168 | + |
| 169 | +## Test Files |
| 170 | + |
| 171 | +Move corresponding test files from `packages/cli/test/` to `packages/kg/test/`: |
| 172 | +- `chunker.test.ts` --> `packages/kg/test/chunker.test.ts` |
| 173 | +- `graph.test.ts` --> `packages/kg/test/graph.test.ts` |
| 174 | + |
| 175 | +Update imports in moved tests to use `@agents/kg/*` paths. If tests have CLI-specific fixtures, keep them in CLI and update imports to point to `@agents/kg`. |
| 176 | + |
| 177 | +## Dependency Notes |
| 178 | + |
| 179 | +- `chunker.ts` is imported by `manifest.ts` (for `parseFrontmatter`). After Phase 1 moves chunker to `@agents/kg`, `manifest.ts` temporarily imports from `@agents/kg/chunker`. Phase 2 resolves this by switching manifest to use `@agents/sdk/context/frontmatter.parseFrontmatter` instead, removing the cross-package KG dependency from the content-parsing layer. |
| 180 | +- `meilisearch.ts` is imported by `skill-search-api.ts` (Phase 5). After Phase 1, skill-search-api imports from `@agents/kg/meilisearch`. Phase 6 may move search to SDK, at which point the import path updates again. |
| 181 | +- Graph modules are only consumed by `commands/kg.ts` and `server/graph-viewer/` -- no transitive dependency concerns. |
| 182 | + |
| 183 | +## Examples |
| 184 | + |
| 185 | +### Before (cli/src/commands/kg.ts) |
| 186 | + |
| 187 | +```typescript |
| 188 | +import { chunkMarkdown, parseFrontmatter } from '../lib/chunker' |
| 189 | +import { embed } from '../lib/embedder' |
| 190 | +import { buildGraph } from '../lib/graph' |
| 191 | +``` |
| 192 | + |
| 193 | +### After (cli/src/commands/kg.ts) |
| 194 | + |
| 195 | +```typescript |
| 196 | +import { chunkMarkdown, parseFrontmatter } from '@agents/kg/chunker' |
| 197 | +import { embed } from '@agents/kg/embedder' |
| 198 | +import { buildGraph } from '@agents/kg/graph' |
| 199 | +``` |
| 200 | + |
| 201 | +### Before (cli/src/lib/manifest.ts) |
| 202 | + |
| 203 | +```typescript |
| 204 | +import { parseFrontmatter } from './chunker' |
| 205 | +``` |
| 206 | + |
| 207 | +### After Phase 1 (temporary -- resolved in Phase 2) |
| 208 | + |
| 209 | +```typescript |
| 210 | +import { parseFrontmatter } from '@agents/kg/chunker' |
| 211 | +``` |
| 212 | + |
| 213 | +### packages/kg/package.json (new) |
| 214 | + |
| 215 | +```json |
| 216 | +{ |
| 217 | + "name": "@agents/kg", |
| 218 | + "version": "0.1.0", |
| 219 | + "private": true, |
| 220 | + "type": "module", |
| 221 | + "exports": { |
| 222 | + ".": "./src/index.ts", |
| 223 | + "./*": "./src/*.ts" |
| 224 | + }, |
| 225 | + "dependencies": { |
| 226 | + "@agents/core": "workspace:*", |
| 227 | + "ajv": "^8.18.0", |
| 228 | + "ajv-formats": "^3.0.1", |
| 229 | + "graphology": "^0.26.0", |
| 230 | + "graphology-layout": "^0.6.1", |
| 231 | + "graphology-layout-forceatlas2": "^0.10.1", |
| 232 | + "js-yaml": "^4.1.0", |
| 233 | + "meilisearch": "^0.56.0", |
| 234 | + "ollama": "^0.5.0" |
| 235 | + } |
| 236 | +} |
| 237 | +``` |
0 commit comments