|
1 | 1 | = Anamnesis |
2 | | -Jonathan D.A. Jewell <jonathan.jewell@gmail.com> |
3 | | -:toc: macro |
4 | | -:icons: font |
5 | | -:source-highlighter: rouge |
6 | | -:experimental: |
7 | | -:url-github: https://github.com/hyperpolymath/Anamnesis |
8 | | -:url-gitlab: https://gitlab.com/hyperpolymath/Anamnesis |
9 | | -:url-bitbucket: https://bitbucket.org/hyperpolymath/Anamnesis |
10 | | -:url-codeberg: https://codeberg.org/hyperpolymath/Anamnesis |
11 | 2 |
|
12 | | -Memory and knowledge graph persistence layer |
| 3 | +*A conversation knowledge extraction and reconciliation system for multi-LLM development workflows.* |
13 | 4 |
|
14 | | -image:https://img.shields.io/badge/RSR-Certified-gold[RSR Certified] |
15 | | -image:https://img.shields.io/badge/License-AGPL%20v3-blue[License] |
| 5 | +== Problem |
16 | 6 |
|
17 | | -toc::[] |
| 7 | +Modern AI-assisted development creates chaos: |
18 | 8 |
|
19 | | -== Overview |
| 9 | +- *Mixed-project conversations* - Multiple projects discussed in single threads |
| 10 | +- *Dead-end tangents* - Valuable ideas lost in conversation noise |
| 11 | +- *Lost context* - Decisions and rationale buried across chat histories |
| 12 | +- *Artifact fragmentation* - Code/docs created, modified, abandoned across sessions |
| 13 | +- *Multi-LLM mayhem* - Switching between Claude, ChatGPT, Mistral with no unified view |
20 | 14 |
|
21 | | -Anamnesis is part of the link:https://rhodium.sh[Rhodium Standard] (RSR) ecosystem. |
| 15 | +== Solution |
22 | 16 |
|
23 | | -Domain: *software-development* |
| 17 | +Anamnesis parses tangled multi-LLM discussion threads, tracks artifact lifecycles (created→modified→removed→evaluated), cross-links related fragments across conversations, and generates clean summaries with content properly bucketed by project/topic. |
24 | 18 |
|
25 | | -== Installation |
| 19 | +== Architecture |
26 | 20 |
|
27 | | -[source,bash] |
28 | | ----- |
29 | | -# Clone from GitHub (primary) |
30 | | -git clone {url-github} |
| 21 | +Multi-language functional system leveraging each language's strengths: |
31 | 22 |
|
32 | | -# Or from mirrors |
33 | | -git clone {url-gitlab} |
34 | | -git clone {url-codeberg} |
35 | | ----- |
| 23 | +``` |
| 24 | +┌─────────────────────────────────────────────────────────────────┐ |
| 25 | +│ Data Sources: Claude, ChatGPT, Mistral, Git, Browser History │ |
| 26 | +└────────────────────────┬────────────────────────────────────────┘ |
| 27 | + │ |
| 28 | + ▼ |
| 29 | +┌─────────────────────────────────────────────────────────────────┐ |
| 30 | +│ Parser (OCaml): Angstrom combinators, type-safe format parsing│ |
| 31 | +└────────────────────────┬────────────────────────────────────────┘ |
| 32 | + │ |
| 33 | + ▼ |
| 34 | +┌─────────────────────────────────────────────────────────────────┐ |
| 35 | +│ Orchestrator (Elixir): OTP supervision, port pools, HTTP API │ |
| 36 | +└────────────────────────┬────────────────────────────────────────┘ |
| 37 | + │ |
| 38 | + ┌────────────┴────────────┐ |
| 39 | + │ │ |
| 40 | + ▼ ▼ |
| 41 | +┌───────────────────────┐ ┌──────────────────────────────────────┐ |
| 42 | +│ Reasoning (λProlog) │ │ Analytics (Julia) │ |
| 43 | +│ - Artifact lifecycle │ │ - RDF generation (Serd.jl) │ |
| 44 | +│ - Contamination │ │ - SPARQL queries (HTTP.jl) │ |
| 45 | +│ - Fuzzy categorization│ │ - Reservoir computing (ESN) │ |
| 46 | +│ - Meta-reasoning │ │ - KBANN (knowledge-augmented nets) │ |
| 47 | +└───────────┬───────────┘ └──────────┬───────────────────────────┘ |
| 48 | + │ │ |
| 49 | + └────────────┬────────────┘ |
| 50 | + ▼ |
| 51 | +┌─────────────────────────────────────────────────────────────────┐ |
| 52 | +│ Storage (Virtuoso): RDF triplestore, SPARQL 1.1, named graphs │ |
| 53 | +└────────────────────────┬────────────────────────────────────────┘ |
| 54 | + │ |
| 55 | + ▼ |
| 56 | +┌─────────────────────────────────────────────────────────────────┐ |
| 57 | +│ Visualization (ReScript): Reagraph graphs, timeline charts, │ |
| 58 | +│ fuzzy category viz, type-safe functional React components │ |
| 59 | +└─────────────────────────────────────────────────────────────────┘ |
| 60 | +``` |
36 | 61 |
|
37 | | -== RSR Stack |
| 62 | +== Technology Stack |
38 | 63 |
|
39 | | -This project follows RSR conventions: |
| 64 | +| Layer | Technology | Why | |
| 65 | +|-------|-----------|-----| |
| 66 | +| *Parsing* | OCaml (Angstrom, Atdgen) | Type safety, parser combinators, ELPI integration | |
| 67 | +| *Orchestration* | Elixir/OTP | Fault tolerance, concurrency, supervision trees | |
| 68 | +| *Reasoning* | λProlog (ELPI) | Higher-order abstract syntax, meta-reasoning | |
| 69 | +| *Analytics* | Julia | Scientific computing without Python, RDF, reservoir computing | |
| 70 | +| *Storage* | Virtuoso + SPARQL | RDF native, named graphs, inference | |
| 71 | +| *Visualization* | ReScript + React | Type safety, functional paradigm, eventual JS replacement | |
40 | 72 |
|
41 | | -* ✅ ReScript for frontend/logic |
42 | | -* ✅ Deno for JS runtime |
43 | | -* ✅ WASM for performance-critical code |
44 | | -* ✅ Rust/OCaml/Haskell for systems/proofs |
45 | | -* ✅ Guile/Scheme for configuration |
46 | | -* ❌ No TypeScript |
47 | | -* ❌ No Go |
48 | | -* ❌ No npm |
| 73 | +=== Critical Constraints |
49 | 74 |
|
50 | | -== Mirrors |
| 75 | +- ❌ *NO PYTHON* - Use Julia for all scientific computing |
| 76 | +- ✅ *Type safety* throughout (OCaml, ReScript, Elixir typespecs) |
| 77 | +- ✅ *Functional paradigm* - Immutability, pure functions, declarative |
| 78 | +- ✅ *Multi-LLM* from start (stub complexity initially) |
| 79 | +- ✅ *Academic rigor* - Research-grade documentation |
51 | 80 |
|
52 | | -[cols="1,2"] |
53 | | -|=== |
54 | | -| Platform | URL |
| 81 | +== Project Structure |
55 | 82 |
|
56 | | -| GitHub (primary) | {url-github} |
57 | | -| GitLab | {url-gitlab} |
58 | | -| Bitbucket | {url-bitbucket} |
59 | | -| Codeberg | {url-codeberg} |
60 | | -|=== |
| 83 | +``` |
| 84 | +. |
| 85 | +├── CLAUDE.md # Claude Code guidance |
| 86 | +├── README.md # This file |
| 87 | +├── parser/ # OCaml parsing components |
| 88 | +│ ├── lib/parsers/ # Format-specific parsers |
| 89 | +│ ├── bin/ # Port executable |
| 90 | +│ └── README.md |
| 91 | +├── orchestrator/ # Elixir orchestration layer |
| 92 | +│ ├── lib/anamnesis/ # Core logic |
| 93 | +│ ├── lib/anamnesis_web/ # Phoenix HTTP API |
| 94 | +│ └── README.md |
| 95 | +├── reasoning/ # λProlog meta-reasoning |
| 96 | +│ ├── lifecycle/ # Artifact state machines |
| 97 | +│ ├── categorization/ # Fuzzy membership, contamination |
| 98 | +│ ├── linking/ # Cross-conversation references |
| 99 | +│ ├── episodic/ # Memory queries |
| 100 | +│ └── README.md |
| 101 | +├── learning/ # Julia analytics |
| 102 | +│ ├── src/rdf/ # RDF manipulation |
| 103 | +│ ├── src/reservoir/ # Reservoir computing, KBANN |
| 104 | +│ ├── src/graphs/ # Graph processing |
| 105 | +│ └── README.md |
| 106 | +├── visualization/ # ReScript UI |
| 107 | +│ ├── src/components/ # React components |
| 108 | +│ ├── src/bindings/ # JS library bindings |
| 109 | +│ ├── src/types/ # Domain types |
| 110 | +│ └── README.md |
| 111 | +├── proving-ground/ # Test workspace |
| 112 | +│ └── (zotero-voyant-export copy) |
| 113 | +└── docs/ |
| 114 | + ├── architecture/ # System architecture |
| 115 | + ├── research/ # Tech stack research |
| 116 | + ├── guides/ # Integration guides |
| 117 | + └── specifications/ # RDF schemas, protocols |
| 118 | +``` |
61 | 119 |
|
62 | | -== License |
| 120 | +== Getting Started |
| 121 | + |
| 122 | +=== Prerequisites |
| 123 | + |
| 124 | +- *OCaml* 5.0+ (via OPAM) |
| 125 | +- *Elixir* 1.15+ (via asdf or package manager) |
| 126 | +- *Julia* 1.10+ (via juliaup) |
| 127 | +- *Node.js* 18+ (for ReScript) |
| 128 | +- *Virtuoso* 7+ (via Docker or native) |
| 129 | + |
| 130 | +=== Quick Start |
| 131 | + |
| 132 | +```bash |
| 133 | += 1. Install language runtimes (see component READMEs) |
| 134 | + |
| 135 | += 2. Build parser (OCaml) |
| 136 | +cd parser |
| 137 | +opam install --deps-only . |
| 138 | +dune build |
| 139 | + |
| 140 | += 3. Build orchestrator (Elixir) |
| 141 | +cd ../orchestrator |
| 142 | +mix deps.get |
| 143 | +mix compile |
| 144 | + |
| 145 | += 4. Build reasoning (λProlog via OCaml/ELPI) |
| 146 | +cd ../reasoning |
| 147 | +dune build |
| 148 | + |
| 149 | += 5. Setup analytics (Julia) |
| 150 | +cd ../learning |
| 151 | +julia --project=. -e 'using Pkg; Pkg.instantiate()' |
| 152 | + |
| 153 | += 6. Build visualization (ReScript) |
| 154 | +cd ../visualization |
| 155 | +npm install |
| 156 | +npm run res:build |
| 157 | + |
| 158 | += 7. Start Virtuoso (Docker) |
| 159 | +docker run -d -p 8890:8890 -p 1111:1111 \ |
| 160 | + -e DBA_PASSWORD=anamnesis \ |
| 161 | + --name virtuoso \ |
| 162 | + openlink/virtuoso-opensource-7 |
| 163 | + |
| 164 | += 8. Run Anamnesis |
| 165 | +cd ../orchestrator |
| 166 | +mix phx.server |
| 167 | += Visit http://localhost:4000 |
| 168 | +``` |
| 169 | + |
| 170 | +== Current Status |
| 171 | + |
| 172 | +*Phase: Planning → Initial Implementation* |
| 173 | + |
| 174 | +=== Completed ✅ |
| 175 | + |
| 176 | +- [x] Project architecture designed |
| 177 | +- [x] Comprehensive tech stack research (6 detailed reports) |
| 178 | +- [x] Directory structure created |
| 179 | +- [x] Component READMEs written |
| 180 | +- [x] CLAUDE.md guidance created |
| 181 | + |
| 182 | +=== In Progress 🚧 |
| 183 | + |
| 184 | +- [ ] OCaml parser starter code |
| 185 | +- [ ] Elixir supervision tree |
| 186 | +- [ ] λProlog reasoning modules |
| 187 | +- [ ] Julia RDF manipulation |
| 188 | +- [ ] ReScript component scaffolding |
| 189 | + |
| 190 | +=== Phase 1 Roadmap 📋 |
| 191 | + |
| 192 | +1. ✅ Create Anamnesis repository |
| 193 | +2. ✅ Research tech options (Julia RDF, reservoir computing, ReScript viz, OCaml parsers, λProlog) |
| 194 | +3. ⬜ Copy proving ground (zotero-voyant-export) |
| 195 | +4. ⬜ Setup basic infrastructure (build systems, test frameworks) |
| 196 | +5. ⬜ *Milestone 1*: Parse single Claude conversation JSON → Virtuoso RDF triples |
| 197 | + |
| 198 | +=== Phase 2: Milestone 1 🎯 |
| 199 | + |
| 200 | +*Goal*: End-to-end pipeline for one conversation format |
| 201 | + |
| 202 | +- [ ] Claude JSON format parser (OCaml) |
| 203 | +- [ ] Port communication (OCaml ↔ Elixir) |
| 204 | +- [ ] Basic RDF generation (Julia) |
| 205 | +- [ ] Virtuoso storage |
| 206 | +- [ ] Simple graph visualization (ReScript) |
| 207 | + |
| 208 | +== Key Design Concepts |
| 209 | + |
| 210 | +=== Multi-Category Membership |
| 211 | + |
| 212 | +Conversations don't belong to just one project. Anamnesis supports fuzzy boundaries: |
| 213 | + |
| 214 | +```turtle |
| 215 | +anamnesis:conv:123 anamnesis:belongsTo anamnesis:proj:anamnesis ; |
| 216 | + anamnesis:membershipStrength 0.8 . |
| 217 | + |
| 218 | +anamnesis:conv:123 anamnesis:belongsTo anamnesis:proj:rescript-evangeliser ; |
| 219 | + anamnesis:membershipStrength 0.3 . |
| 220 | +``` |
| 221 | + |
| 222 | +Visualized via color mixing: 80% green (Anamnesis) + 30% orange (ReScript) = blended color. |
63 | 223 |
|
64 | | -Licensed under AGPL-3.0-or-later OR LicenseRef-Palimpsest-0.5. |
| 224 | +=== Artifact Lifecycle Tracking |
65 | 225 |
|
66 | | -See link:LICENSE[LICENSE] for details. |
| 226 | +Track state transitions over time: |
| 227 | + |
| 228 | +```prolog |
| 229 | +% λProlog rules |
| 230 | +valid_transition created modified. |
| 231 | +valid_transition modified evaluated. |
| 232 | +valid_transition modified removed. |
| 233 | + |
| 234 | +current_state artifact_123 timestamp_456 State. |
| 235 | +% State = modified |
| 236 | +``` |
| 237 | + |
| 238 | +=== Cross-Conversation Linking |
| 239 | + |
| 240 | +Fragment references span conversations: |
| 241 | + |
| 242 | +```sparql |
| 243 | += SPARQL query |
| 244 | +SELECT ?conv1 ?conv2 ?artifact |
| 245 | +WHERE { |
| 246 | + GRAPH ?g1 { ?conv1 anamnesis:discusses ?artifact . } |
| 247 | + GRAPH ?g2 { ?conv2 anamnesis:discusses ?artifact . } |
| 248 | + FILTER(?conv1 != ?conv2) |
| 249 | +} |
| 250 | +``` |
| 251 | + |
| 252 | +=== Episodic Memory |
| 253 | + |
| 254 | +Named graphs partition conversations by episode: |
| 255 | + |
| 256 | +``` |
| 257 | +graph:conversation:uuid-1 # Conversation metadata |
| 258 | +graph:artifacts:uuid-1 # Artifact lifecycles |
| 259 | +graph:links:uuid-1 # Cross-references |
| 260 | +graph:categories:uuid-1 # Project membership |
| 261 | +``` |
| 262 | + |
| 263 | +== Documentation |
| 264 | + |
| 265 | +- *Architecture*: `/docs/architecture/system-architecture.adoc` - Complete system design |
| 266 | +- *Research*: `/docs/research/` - Tech stack evaluations: |
| 267 | + - `julia-rdf-ecosystem.adoc` - RDF libraries (Serd.jl recommended) |
| 268 | + - `julia-reservoir-computing.adoc` - ESN, KBANN (ReservoirComputing.jl) |
| 269 | + - `puppygraph-evaluation.adoc` - Property graphs vs RDF (stick with Virtuoso) |
| 270 | + - `rescript-visualization.adoc` - Reagraph, rescript-recharts |
| 271 | + - `ocaml-parsing-elixir-bridge.adoc` - Angstrom, ports vs NIFs |
| 272 | + - `lambda-prolog-meta-reasoning.adoc` - ELPI, HOAS, integration |
| 273 | +- *Component READMEs*: Each directory has detailed setup/usage docs |
| 274 | +- *CLAUDE.md*: Guidance for Claude Code when working on this project |
| 275 | + |
| 276 | +== Testing |
| 277 | + |
| 278 | +=== Proving Ground |
| 279 | + |
| 280 | +The `zotero-voyant-export` repository is our test case - it's contaminated with Anamnesis planning discussions mixed into Zotero export tool development. This IS the problem we're solving. |
| 281 | + |
| 282 | +*Strategy*: |
| 283 | +1. Copy entire repo to `proving-ground/` |
| 284 | +2. Preserve original |
| 285 | +3. Run Anamnesis pipeline |
| 286 | +4. Verify correct untangling |
| 287 | +5. Store iterations for comparison |
| 288 | + |
| 289 | +=== Test Coverage Targets |
| 290 | + |
| 291 | +- Parser: 90%+ (property-based testing with qcheck) |
| 292 | +- Orchestrator: 85%+ (ExUnit + StreamData) |
| 293 | +- Analytics: 80%+ (Test.jl) |
| 294 | +- Visualization: 75%+ (Jest + React Testing Library) |
| 295 | +- Reasoning: Example-driven (λProlog test harness) |
| 296 | + |
| 297 | +== Performance Targets |
| 298 | + |
| 299 | +| Component | Target | Strategy | |
| 300 | +|-----------|--------|----------| |
| 301 | +| Parser throughput | 100 conversations/sec | Parallel ports, incremental parsing | |
| 302 | +| λProlog queries | <100ms | ETS caching, tabled predicates | |
| 303 | +| SPARQL queries | <50ms simple, <500ms complex | Virtuoso indexing | |
| 304 | +| Graph viz | 1000-5000 nodes interactive | WebGL (Reagraph), LOD | |
| 305 | +| RDF storage | 10M+ triples | Named graph partitioning | |
| 306 | + |
| 307 | +== Related Projects |
| 308 | + |
| 309 | +- *ReScript Evangeliser* - Learning curve synergies |
| 310 | +- *Zotero:NSAI* - Neurosymbolic AI integration |
| 311 | +- *Fogbinder* - Uncertainty-as-feature framework |
67 | 312 |
|
68 | 313 | == Contributing |
69 | 314 |
|
70 | | -See link:CONTRIBUTING.adoc[CONTRIBUTING.adoc]. |
| 315 | +This is a personal research project, but architecture decisions are documented for transparency. See component READMEs for development guidelines. |
| 316 | + |
| 317 | +== Academic Context |
| 318 | + |
| 319 | +Relevant to user's research/journalism work: |
| 320 | + |
| 321 | +- *Agnotology* - Study of culturally-induced ignorance |
| 322 | +- *Axiology* - Theory of value |
| 323 | +- *Ethics* - Moral philosophy applications |
| 324 | +- *Knowledge representation* - Formal semantics, RDF, logic programming |
| 325 | + |
| 326 | +== License |
| 327 | + |
| 328 | +To be determined. |
| 329 | + |
| 330 | +== Resources |
| 331 | + |
| 332 | +- Repository: https://github.com/Hyperpolymath/anamnesis |
| 333 | +- Test Case: zotero-voyant-export (to be copied to proving-ground) |
| 334 | +- Handover Docs: `zotero-voyant-export/docs/contamination-notice/` |
| 335 | + - ANAMNESIS_HANDOVER_MINIMAL.adoc |
| 336 | + - THREAD_CONTAMINATION_WARNING.adoc |
| 337 | + |
| 338 | +--- |
71 | 339 |
|
72 | | -== Metadata |
| 340 | +*Status*: Planning phase, comprehensive research complete, initial implementation starting. |
73 | 341 |
|
74 | | -* Domain: software-development |
75 | | -* Framework: RSR (Rhodium Standard Repository) |
76 | | -* Dublin Core: link:.well-known/dc.xml[.well-known/dc.xml] |
| 342 | +*Next Immediate Steps*: Implement Milestone 1 (Claude JSON → Virtuoso RDF pipeline). |
0 commit comments