Skip to content

Commit 16fb818

Browse files
committed
Add sponsor badge
1 parent 943938a commit 16fb818

1 file changed

Lines changed: 367 additions & 0 deletions

File tree

README.md

Lines changed: 367 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,367 @@
1+
[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github)](https://github.com/sponsors/hyperpolymath)
2+
3+
= Anamnesis
4+
5+
image:https://img.shields.io/badge/License-PMPL--1.0-blue.svg[License: PMPL-1.0,link="https://github.com/hyperpolymath/palimpsest-license"]
6+
image:https://img.shields.io/badge/Philosophy-Palimpsest-indigo.svg[Palimpsest,link="https://github.com/hyperpolymath/palimpsest-license"]
7+
8+
9+
*A conversation knowledge extraction and reconciliation system for multi-LLM development workflows.*
10+
11+
== Problem
12+
13+
Modern AI-assisted development creates chaos:
14+
15+
- *Mixed-project conversations* - Multiple projects discussed in single threads
16+
- *Dead-end tangents* - Valuable ideas lost in conversation noise
17+
- *Lost context* - Decisions and rationale buried across chat histories
18+
- *Artifact fragmentation* - Code/docs created, modified, abandoned across sessions
19+
- *Multi-LLM mayhem* - Switching between Claude, ChatGPT, Mistral with no unified view
20+
21+
== Solution
22+
23+
Anamnesis parses tangled multi-LLM discussion threads, tracks artifact lifecycles (created→modified→removed→evaluated), cross-links related fragments across conversations, and generates clean summaries with content properly bucketed by project/topic.
24+
25+
== Architecture
26+
27+
Multi-language functional system leveraging each language's strengths:
28+
29+
```
30+
┌─────────────────────────────────────────────────────────────────┐
31+
│ Data Sources: Claude, ChatGPT, Mistral, Git, Browser History │
32+
└────────────────────────┬────────────────────────────────────────┘
33+
34+
35+
┌─────────────────────────────────────────────────────────────────┐
36+
│ Parser (OCaml): Angstrom combinators, type-safe format parsing│
37+
└────────────────────────┬────────────────────────────────────────┘
38+
39+
40+
┌─────────────────────────────────────────────────────────────────┐
41+
│ Orchestrator (Elixir): OTP supervision, port pools, HTTP API │
42+
└────────────────────────┬────────────────────────────────────────┘
43+
44+
┌────────────┴────────────┐
45+
│ │
46+
▼ ▼
47+
┌───────────────────────┐ ┌──────────────────────────────────────┐
48+
│ Reasoning (λProlog) │ │ Analytics (Julia) │
49+
│ - Artifact lifecycle │ │ - RDF generation (Serd.jl) │
50+
│ - Contamination │ │ - SPARQL queries (HTTP.jl) │
51+
│ - Fuzzy categorization│ │ - Reservoir computing (ESN) │
52+
│ - Meta-reasoning │ │ - KBANN (knowledge-augmented nets) │
53+
└───────────┬───────────┘ └──────────┬───────────────────────────┘
54+
│ │
55+
└────────────┬────────────┘
56+
57+
┌─────────────────────────────────────────────────────────────────┐
58+
│ Storage (Virtuoso): RDF triplestore, SPARQL 1.1, named graphs │
59+
└────────────────────────┬────────────────────────────────────────┘
60+
61+
62+
┌─────────────────────────────────────────────────────────────────┐
63+
│ Visualization (ReScript): Reagraph graphs, timeline charts, │
64+
│ fuzzy category viz, type-safe functional React components │
65+
└─────────────────────────────────────────────────────────────────┘
66+
```
67+
68+
== Technology Stack
69+
70+
| Layer | Technology | Why |
71+
|-------|-----------|-----|
72+
| *Parsing* | OCaml (Angstrom, Atdgen) | Type safety, parser combinators, ELPI integration |
73+
| *Orchestration* | Elixir/OTP | Fault tolerance, concurrency, supervision trees |
74+
| *Reasoning* | λProlog (ELPI) | Higher-order abstract syntax, meta-reasoning |
75+
| *Analytics* | Julia | Scientific computing without Python, RDF, reservoir computing |
76+
| *Storage* | Virtuoso + SPARQL | RDF native, named graphs, inference |
77+
| *Visualization* | ReScript + React | Type safety, functional paradigm, eventual JS replacement |
78+
79+
=== Critical Constraints
80+
81+
-*NO PYTHON* - Use Julia for all scientific computing
82+
-*Type safety* throughout (OCaml, ReScript, Elixir typespecs)
83+
-*Functional paradigm* - Immutability, pure functions, declarative
84+
-*Multi-LLM* from start (stub complexity initially)
85+
-*Academic rigor* - Research-grade documentation
86+
87+
== Project Structure
88+
89+
```
90+
.
91+
├── CLAUDE.md # Claude Code guidance
92+
├── README.md # This file
93+
├── parser/ # OCaml parsing components
94+
│ ├── lib/parsers/ # Format-specific parsers
95+
│ ├── bin/ # Port executable
96+
│ └── README.md
97+
├── orchestrator/ # Elixir orchestration layer
98+
│ ├── lib/anamnesis/ # Core logic
99+
│ ├── lib/anamnesis_web/ # Phoenix HTTP API
100+
│ └── README.md
101+
├── reasoning/ # λProlog meta-reasoning
102+
│ ├── lifecycle/ # Artifact state machines
103+
│ ├── categorization/ # Fuzzy membership, contamination
104+
│ ├── linking/ # Cross-conversation references
105+
│ ├── episodic/ # Memory queries
106+
│ └── README.md
107+
├── learning/ # Julia analytics
108+
│ ├── src/rdf/ # RDF manipulation
109+
│ ├── src/reservoir/ # Reservoir computing, KBANN
110+
│ ├── src/graphs/ # Graph processing
111+
│ └── README.md
112+
├── visualization/ # ReScript UI
113+
│ ├── src/components/ # React components
114+
│ ├── src/bindings/ # JS library bindings
115+
│ ├── src/types/ # Domain types
116+
│ └── README.md
117+
├── proving-ground/ # Test workspace
118+
│ └── (zotero-voyant-export copy)
119+
└── docs/
120+
├── architecture/ # System architecture
121+
├── research/ # Tech stack research
122+
├── guides/ # Integration guides
123+
└── specifications/ # RDF schemas, protocols
124+
```
125+
126+
== Getting Started
127+
128+
=== Prerequisites
129+
130+
- *OCaml* 5.0+ (via OPAM)
131+
- *Elixir* 1.15+ (via asdf or package manager)
132+
- *Julia* 1.10+ (via juliaup)
133+
- *Node.js* 18+ (for ReScript)
134+
- *Virtuoso* 7+ (via Docker or native)
135+
136+
=== Quick Start
137+
138+
```bash
139+
= 1. Install language runtimes (see component READMEs)
140+
141+
= 2. Build parser (OCaml)
142+
143+
cd parser
144+
opam install --deps-only .
145+
dune build
146+
147+
= 3. Build orchestrator (Elixir)
148+
149+
cd ../orchestrator
150+
mix deps.get
151+
mix compile
152+
153+
= 4. Build reasoning (λProlog via OCaml/ELPI)
154+
155+
cd ../reasoning
156+
dune build
157+
158+
= 5. Setup analytics (Julia)
159+
160+
cd ../learning
161+
julia --project=. -e 'using Pkg; Pkg.instantiate()'
162+
163+
= 6. Build visualization (ReScript)
164+
165+
cd ../visualization
166+
npm install
167+
npm run res:build
168+
169+
= 7. Start Virtuoso (Docker)
170+
171+
docker run -d -p 8890:8890 -p 1111:1111 \
172+
-e DBA_PASSWORD=anamnesis \
173+
--name virtuoso \
174+
openlink/virtuoso-opensource-7
175+
176+
= 8. Run Anamnesis
177+
178+
cd ../orchestrator
179+
mix phx.server
180+
= Visit http://localhost:4000
181+
182+
```
183+
184+
== Current Status
185+
186+
*Phase: Initial Implementation*
187+
188+
=== Completed ✅
189+
190+
- [x] Project architecture designed
191+
- [x] Comprehensive tech stack research (6 detailed reports)
192+
- [x] Directory structure created
193+
- [x] Component READMEs written
194+
- [x] CLAUDE.md guidance created
195+
- [x] OCaml Claude parser implemented
196+
- [x] Elixir port infrastructure
197+
- [x] Julia RDF modules (schema, serialization, SPARQL, Virtuoso)
198+
- [x] Julia reservoir computing modules (ESN, embeddings)
199+
- [x] Julia graph analysis modules
200+
- [x] ReScript domain types and color mixing
201+
202+
=== In Progress 🚧
203+
204+
- [ ] End-to-end integration testing
205+
- [ ] λProlog reasoning module implementation
206+
- [ ] Virtuoso storage integration
207+
- [ ] Additional LLM format parsers (ChatGPT, Mistral)
208+
209+
=== Phase 1 Roadmap 📋
210+
211+
1. ✅ Create Anamnesis repository
212+
2. ✅ Research tech options (Julia RDF, reservoir computing, ReScript viz, OCaml parsers, λProlog)
213+
3. ⬜ Copy proving ground (zotero-voyant-export)
214+
4. ✅ Setup basic infrastructure (build systems, test frameworks)
215+
5. 🚧 *Milestone 1*: Parse single Claude conversation JSON → Virtuoso RDF triples
216+
217+
=== Phase 2: Milestone 1 🎯
218+
219+
*Goal*: End-to-end pipeline for one conversation format
220+
221+
- [x] Claude JSON format parser (OCaml)
222+
- [x] Port communication (OCaml ↔ Elixir)
223+
- [x] Basic RDF generation (Julia)
224+
- [ ] Virtuoso storage integration
225+
- [ ] Simple graph visualization (ReScript)
226+
227+
== Key Design Concepts
228+
229+
=== Multi-Category Membership
230+
231+
Conversations don't belong to just one project. Anamnesis supports fuzzy boundaries:
232+
233+
```turtle
234+
anamnesis:conv:123 anamnesis:belongsTo anamnesis:proj:anamnesis ;
235+
anamnesis:membershipStrength 0.8 .
236+
237+
anamnesis:conv:123 anamnesis:belongsTo anamnesis:proj:rescript-evangeliser ;
238+
anamnesis:membershipStrength 0.3 .
239+
```
240+
241+
Visualized via color mixing: 80% green (Anamnesis) + 30% orange (ReScript) = blended color.
242+
243+
=== Artifact Lifecycle Tracking
244+
245+
Track state transitions over time:
246+
247+
```prolog
248+
% λProlog rules
249+
valid_transition created modified.
250+
valid_transition modified evaluated.
251+
valid_transition modified removed.
252+
253+
current_state artifact_123 timestamp_456 State.
254+
% State = modified
255+
```
256+
257+
=== Cross-Conversation Linking
258+
259+
Fragment references span conversations:
260+
261+
```sparql
262+
= SPARQL query
263+
264+
SELECT ?conv1 ?conv2 ?artifact
265+
WHERE {
266+
GRAPH ?g1 { ?conv1 anamnesis:discusses ?artifact . }
267+
GRAPH ?g2 { ?conv2 anamnesis:discusses ?artifact . }
268+
FILTER(?conv1 != ?conv2)
269+
}
270+
```
271+
272+
=== Episodic Memory
273+
274+
Named graphs partition conversations by episode:
275+
276+
```
277+
graph:conversation:uuid-1 # Conversation metadata
278+
graph:artifacts:uuid-1 # Artifact lifecycles
279+
graph:links:uuid-1 # Cross-references
280+
graph:categories:uuid-1 # Project membership
281+
```
282+
283+
== Documentation
284+
285+
- *Architecture*: `/docs/architecture/system-architecture.adoc` - Complete system design
286+
- *Research*: `/docs/research/` - Tech stack evaluations:
287+
- `julia-rdf-ecosystem.adoc` - RDF libraries (Serd.jl recommended)
288+
- `julia-reservoir-computing.adoc` - ESN, KBANN (ReservoirComputing.jl)
289+
- `puppygraph-evaluation.adoc` - Property graphs vs RDF (stick with Virtuoso)
290+
- `rescript-visualization.adoc` - Reagraph, rescript-recharts
291+
- `ocaml-parsing-elixir-bridge.adoc` - Angstrom, ports vs NIFs
292+
- `lambda-prolog-meta-reasoning.adoc` - ELPI, HOAS, integration
293+
- *Component READMEs*: Each directory has detailed setup/usage docs
294+
- *CLAUDE.md*: Guidance for Claude Code when working on this project
295+
296+
== Testing
297+
298+
=== Proving Ground
299+
300+
The `zotero-voyant-export` repository is our test case - it's contaminated with Anamnesis planning discussions mixed into Zotero export tool development. This IS the problem we're solving.
301+
302+
*Strategy*:
303+
1. Copy entire repo to `proving-ground/`
304+
2. Preserve original
305+
3. Run Anamnesis pipeline
306+
4. Verify correct untangling
307+
5. Store iterations for comparison
308+
309+
=== Test Coverage Targets
310+
311+
- Parser: 90%+ (property-based testing with qcheck)
312+
- Orchestrator: 85%+ (ExUnit + StreamData)
313+
- Analytics: 80%+ (Test.jl)
314+
- Visualization: 75%+ (Jest + React Testing Library)
315+
- Reasoning: Example-driven (λProlog test harness)
316+
317+
== Performance Targets
318+
319+
| Component | Target | Strategy |
320+
|-----------|--------|----------|
321+
| Parser throughput | 100 conversations/sec | Parallel ports, incremental parsing |
322+
| λProlog queries | <100ms | ETS caching, tabled predicates |
323+
| SPARQL queries | <50ms simple, <500ms complex | Virtuoso indexing |
324+
| Graph viz | 1000-5000 nodes interactive | WebGL (Reagraph), LOD |
325+
| RDF storage | 10M+ triples | Named graph partitioning |
326+
327+
== Related Projects
328+
329+
- *ReScript Evangeliser* - Learning curve synergies
330+
- *Zotero:NSAI* - Neurosymbolic AI integration
331+
- *Fogbinder* - Uncertainty-as-feature framework
332+
333+
== Contributing
334+
335+
This is a personal research project, but architecture decisions are documented for transparency. See component READMEs for development guidelines.
336+
337+
== Academic Context
338+
339+
Relevant to user's research/journalism work:
340+
341+
- *Agnotology* - Study of culturally-induced ignorance
342+
- *Axiology* - Theory of value
343+
- *Ethics* - Moral philosophy applications
344+
- *Knowledge representation* - Formal semantics, RDF, logic programming
345+
346+
== License
347+
348+
To be determined.
349+
350+
== Resources
351+
352+
- Repository: https://github.com/Hyperpolymath/anamnesis
353+
- Test Case: zotero-voyant-export (to be copied to proving-ground)
354+
- Handover Docs: `zotero-voyant-export/docs/contamination-notice/`
355+
- ANAMNESIS_HANDOVER_MINIMAL.adoc
356+
- THREAD_CONTAMINATION_WARNING.adoc
357+
358+
---
359+
360+
*Status*: Initial implementation phase. Core infrastructure complete, working toward Milestone 1.
361+
362+
*Next Immediate Steps*: Complete end-to-end testing and Virtuoso integration.
363+
364+
365+
== Architecture
366+
367+
See link:TOPOLOGY.md[TOPOLOGY.md] for a visual architecture map and completion dashboard.

0 commit comments

Comments
 (0)