Add scripted impression recall and memory linting

Squash merge V0.2.2 scripted impression recall, token-only engine scan, memory linting, package bins, and adoption docs.

Author: focaxisdev

CHANGELOG.md

@@ -2,6 +2,24 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.2.2] - 2026-04-21
+
+- Made engine impression scans token-only so query embeddings are deferred until chunk retrieval is needed.
+- Added CLI `--memory-root` and `--file` options and aligned `matched` with weak/strong scan levels.
+- Added `deja-vu-scan-memory` and `deja-vu-lint-memory` package binaries.
+- Added memory linting for impression schema, duplicate ids, and linked record paths.
+- Updated adoption docs, handshake, templates, and example project to make impression-first recall the default path.
+- Updated protocol naming to v0.2 and clarified project-local plain text storage.
+
+## [0.2.1] - 2026-04-21
+
+- Added a protocol-level impression layer for compact keyword-first familiarity scans.
+- Added scripted recall documentation and a default `scripts/dejavu-scan-memory.mjs` scanner.
+- Added `memory/impressions.jsonl` and `memory/events/` templates for cheap long-term continuity.
+- Updated workflow, protocol, storage, bootstrap, and AGENTS templates around impression-first recall.
+- Extended the optional TypeScript engine with `impressionTokens` and `scanImpressions()`.
+- Added tests for scanning impressions without loading deeper memory.
+
 ## [0.2.0] - 2026-04-20
 
 - Repositioned Deja Vu as a protocol-first memory system for AI agents.

README.md

@@ -21,33 +21,38 @@ It answers:
 - how memory should be stored in ordinary project files
 - when memory should be updated, compacted, or retired
 
-The minimum viable setup uses Markdown files only. No npm package, embeddings, vector search, or database is required.
+The minimum viable setup uses project-local plain text files. No npm package, embeddings, vector search, or database is required.
 
 ## Start Here
 
 If you want to adopt Deja Vu in a project without extra infrastructure:
 
 1. Read [docs/protocol.md](./docs/protocol.md).
 2. Read [docs/workflow.md](./docs/workflow.md).
-3. Copy the templates from [docs/templates](./docs/templates).
-4. Add the generated rules and memory files to your project.
+3. Read [docs/impression-layer.md](./docs/impression-layer.md).
+4. Read [docs/scripted-recall.md](./docs/scripted-recall.md).
+5. Copy the templates from [docs/templates](./docs/templates).
+6. Add the generated rules and memory files to your project.
 
 Recommended first files:
 
 - [docs/templates/AGENTS.template.md](./docs/templates/AGENTS.template.md)
 - [docs/templates/memory/index.md](./docs/templates/memory/index.md)
 - [docs/templates/memory/summary.md](./docs/templates/memory/summary.md)
+- [docs/templates/memory/impressions.jsonl](./docs/templates/memory/impressions.jsonl)
+- [docs/templates/memory/events/YYYY-MM.md](./docs/templates/memory/events/YYYY-MM.md)
 - [docs/templates/memory/decisions/decision-template.md](./docs/templates/memory/decisions/decision-template.md)
 - [docs/templates/memory/open-loops/open-loop-template.md](./docs/templates/memory/open-loops/open-loop-template.md)
 
 ## The Protocol in One Page
 
 Deja Vu follows a simple lifecycle:
 
-1. Recall before substantial planning, coding, or answering.
-2. Work using only the smallest memory slice needed.
-3. Write back only durable, reusable outcomes.
-4. Compact or supersede memories when detail becomes repetitive or stale.
+1. Scan a tiny impression index before substantial planning, coding, or answering.
+2. Load summaries only when the scan finds weak familiarity.
+3. Load detailed records only when the scan finds strong familiarity or the task requires depth.
+4. Write back durable outcomes and a cheap event trace.
+5. Compact or supersede memories when detail becomes repetitive or stale.
 
 This keeps memory project-local, readable, and easy to maintain across new conversations.
 
@@ -57,6 +62,8 @@ This keeps memory project-local, readable, and easy to maintain across new conve
 memory/
   index.md
   summary.md
+  impressions.jsonl
+  events/
   context/
     project-context.md
   decisions/
@@ -71,7 +78,7 @@ The canonical layout and field rules are specified in [docs/storage-markdown.md]
 
 - Use a single-project scope only in MVP: `project:<project-id>`.
 - Recall before substantial work.
-- Prefer summary memory first; open detailed records only when needed.
+- Prefer scripted impression scans first; open summary or detailed records only when needed.
 - Write back only durable memory:
   - decisions
   - architecture intent
@@ -132,6 +139,7 @@ The npm package provides the optional TypeScript engine. It is not required for
 const engine = new SemanticRecallEngine(config);
 
 await engine.addMemory(input);
+await engine.scanImpressions(query);
 await engine.recall(query);
 await engine.getSummary(id);
 await engine.getChunks(id);
@@ -141,6 +149,8 @@ await engine.deleteMemory(id);
 
 The public TypeScript exports remain intact for hosts that want semantic recall.
 
+`scanImpressions()` performs token-only familiarity scanning and does not load summaries or chunks.
+
 ## Examples
 
 - Protocol-first example: [examples/protocol-project](./examples/protocol-project)
@@ -175,13 +185,16 @@ deja-vu/
 npm install
 npm run build
 npm run test:src
+npm run lint:memory
 ```
 
 ## References
 
 - [docs/protocol.md](./docs/protocol.md)
 - [docs/workflow.md](./docs/workflow.md)
 - [docs/storage-markdown.md](./docs/storage-markdown.md)
+- [docs/impression-layer.md](./docs/impression-layer.md)
+- [docs/scripted-recall.md](./docs/scripted-recall.md)
 - [docs/bootstrap-instructions.md](./docs/bootstrap-instructions.md)
 - [docs/project-rules-template.md](./docs/project-rules-template.md)
 - [llms.txt](./llms.txt)

docs/agent-handshake.md

@@ -8,10 +8,11 @@ Deja Vu is a protocol-first memory system, not just a TypeScript package.
 
 An agent that discovers this repository should infer:
 
-1. the primary adoption path is rules + workflow + Markdown memory files
+1. the primary adoption path is rules + workflow + project-local memory files
 2. the project scope is `project:<project-id>` in MVP
 3. memory should stay project-local
-4. the optional TypeScript engine is an enhancement layer, not the base requirement
+4. impression-first scripted recall is the default low-cost recall path
+5. the optional TypeScript engine is an enhancement layer, not the base requirement
 
 ## Read order
 
@@ -21,7 +22,9 @@ Read these files first:
 2. `docs/protocol.md`
 3. `docs/workflow.md`
 4. `docs/storage-markdown.md`
-5. `docs/templates/AGENTS.template.md`
+5. `docs/impression-layer.md`
+6. `docs/scripted-recall.md`
+7. `docs/templates/AGENTS.template.md`
 
 Read these files only if the host wants semantic recall:
 
@@ -36,8 +39,8 @@ Read these files only if the host wants semantic recall:
 Unless the host explicitly needs engine-backed retrieval, start with:
 
 - project rules
-- Markdown memory layout
-- pre-task recall
+- memory layout
+- pre-task impression scan
 - post-task writeback
 - manual or host-driven compaction
 

docs/architecture.md

@@ -22,6 +22,8 @@ The engine layer adds:
 - chunk retrieval
 - plugin seams for embeddings, storage, and vector search
 
+This is product layering, not a reduction of the memory model.
+
 ## Engine flow
 
 1. `addMemory(input)` runs a write pipeline.
@@ -34,10 +36,12 @@ The engine layer adds:
 
 ## Layer model
 
-- Familiarity Index: fast ANN-friendly metadata and short summaries only.
+- Impression/Familiarity Index: fast ANN-friendly metadata, short summaries, and compact keyword tokens.
 - Summary Layer: compact agent-readable understanding of the memory.
 - Memory Chunks: detailed recall units scoped to a single memory id.
 
+The protocol-first path mirrors this with `memory/impressions.jsonl`, `memory/summary.md`, and detailed Markdown records.
+
 ## Plugin seams
 
 - `EmbeddingProvider`

docs/bootstrap-instructions.md

@@ -33,7 +33,9 @@ Read:
 2. `docs/protocol.md`
 3. `docs/workflow.md`
 4. `docs/storage-markdown.md`
-5. `docs/templates/AGENTS.template.md`
+5. `docs/impression-layer.md`
+6. `docs/scripted-recall.md`
+7. `docs/templates/AGENTS.template.md`
 
 ### 3. Check whether memory already exists
 
@@ -52,19 +54,28 @@ Create or update:
 - `AGENTS.md`
 - `memory/index.md`
 - `memory/summary.md`
+- `memory/impressions.jsonl`
+- `memory/events/`
 - `memory/context/project-context.md`
 - `memory/decisions/`
 - `memory/open-loops/`
+- `scripts/dejavu-scan-memory.mjs`
 
 ### 5. Apply the workflow
 
 Ensure future work follows:
 
-1. pre-task recall from the memory files
-2. minimal detailed reads
-3. selective post-task writeback
+1. pre-task recall through the impression scan script
+2. minimal summary and detailed reads based on scan strength
+3. selective post-task writeback plus a cheap event trace
 4. compaction when records become repetitive or stale
 
+After bootstrap, run:
+
+```bash
+node scripts/dejavu-lint-memory.mjs
+```
+
 ### 6. Optional engine adoption
 
 Only if the host needs semantic recall at higher scale:

docs/impression-layer.md

@@ -0,0 +1,85 @@
+# Impression Layer
+
+The impression layer is the thinnest Deja Vu memory surface.
+
+Its job is not to explain a memory. Its job is to answer one cheap question:
+
+> Does this task feel familiar enough to justify opening deeper memory?
+
+## Position in the Memory Stack
+
+Deja Vu keeps the optional engine's three-layer memory model:
+
+1. Impression or familiarity index
+2. Summary layer
+3. Detailed chunks or records
+
+The protocol-first path now mirrors that model with project files:
+
+1. `memory/impressions.jsonl`
+2. `memory/summary.md` and linked summaries
+3. detailed records under `memory/context/`, `memory/decisions/`, and `memory/open-loops/`
+
+## Impression Record
+
+Use JSON Lines so a host script can scan records without parsing the whole memory tree.
+
+Required fields:
+
+- `schema_version`
+- `id`
+- `scope`
+- `title`
+- `keywords`
+- `record_path`
+- `updated`
+
+Optional fields:
+
+- `aliases`
+- `weight`
+- `status`
+
+Example:
+
+```json
+{"schema_version":1,"id":"decision-protocol-first","scope":"project:example-project","title":"Protocol-first positioning","keywords":["protocol","workflow","markdown","memory"],"aliases":["repo-first"],"record_path":"memory/decisions/protocol-first-positioning.md","updated":"2026-04-21","weight":0.9,"status":"active"}
+```
+
+## Recall Behavior
+
+Before substantial work:
+
+1. Run the impression scan script against the current user request.
+2. If there is no match, avoid opening detailed memory by default.
+3. If there is a weak match, open only the active summary or one linked summary record.
+4. If there is a strong match, open the linked detailed record.
+5. Load chunks or raw detail only when the task explicitly needs deeper recall.
+
+## Writeback Behavior
+
+When durable memory is created or updated, update the impression record at the same time.
+
+Keep impression keywords short and reusable:
+
+- project terms
+- stable feature names
+- decision names
+- architecture nouns
+- user preference terms
+- unresolved follow-up names
+
+Do not store full summaries, transcripts, secrets, or long prose in the impression layer.
+
+## Retention Behavior
+
+The impression layer may grow over time, but it should remain cheap to scan.
+
+Ask the host whether to prune impressions, compact durable records, or archive stale records when:
+
+- too many impression records match the same query
+- many records repeat the same keyword set
+- `memory/events/` grows without promotion into durable memory
+- detailed records are no longer needed after a newer summary supersedes them
+
+Impression pruning may remove low-value keyword routes. Durable record compaction should preserve historical links through supersession or archival records.

docs/protocol.md

@@ -1,4 +1,4 @@
-# Deja Vu Protocol v0.1
+# Deja Vu Protocol v0.2
 
 Deja Vu is a protocol-first memory system for AI agents.
 
@@ -10,7 +10,7 @@ Enable any agent to maintain useful project memory using only:
 
 - project rules
 - a repeatable workflow
-- project-local Markdown files
+- project-local plain text memory files
 
 The protocol must work without a custom runtime, package install, embedding model, or vector database.
 
@@ -49,6 +49,8 @@ An implementation of Deja Vu MVP must provide these artifacts inside the project
 - `AGENTS.md` or equivalent rules file with Deja Vu rules
 - `memory/index.md`
 - `memory/summary.md`
+- `memory/impressions.jsonl`
+- `memory/events/`
 - `memory/context/project-context.md`
 - `memory/decisions/`
 - `memory/open-loops/`
@@ -62,9 +64,10 @@ These files are the canonical memory surface. Agents should treat them as the pr
 Before substantial planning, coding, or answering:
 
 1. Determine whether the task is substantial.
-2. If yes, read `memory/index.md` and `memory/summary.md`.
-3. Open only the detailed records needed for the current task.
-4. Prefer the smallest useful memory slice.
+2. If yes, run the project impression scan script when available.
+3. If the scan is weak, read `memory/summary.md` and the minimum linked records.
+4. If the scan is strong, open only the detailed records needed for the current task.
+5. If no script is available, fall back to `memory/index.md` and `memory/summary.md`.
 
 ### 2. Work
 
@@ -81,7 +84,9 @@ After meaningful work completes:
 1. Decide whether the outcome is durable.
 2. If it is durable, write or update the appropriate memory artifact.
 3. Update `memory/index.md` so future recall can find the new record quickly.
-4. Update `memory/summary.md` when the project-level understanding has changed.
+4. Update `memory/impressions.jsonl` with compact keywords for future cheap scans.
+5. Update `memory/events/` with a short trace when the work should remain discoverable.
+6. Update `memory/summary.md` when the project-level understanding has changed.
 
 ### 4. Compaction
 

docs/release-v0.2.1.md

@@ -0,0 +1,33 @@
+# Deja Vu v0.2.1
+
+Deja Vu v0.2.1 adds scripted impression-first recall.
+
+This release keeps the v0.2.0 product layering:
+
+1. protocol layer
+2. optional semantic engine layer
+
+It also makes the memory stack explicit again:
+
+1. impression or familiarity index
+2. summary layer
+3. detailed records or chunks
+
+## Highlights
+
+- Added `docs/impression-layer.md`.
+- Added `docs/scripted-recall.md`.
+- Added `scripts/dejavu-scan-memory.mjs`.
+- Added `memory/impressions.jsonl` and `memory/events/` templates.
+- Updated protocol workflow so hosts scan the impression index before loading summaries or detailed records.
+- Added `scanImpressions()` to the optional TypeScript engine.
+
+## Goal
+
+Use the smallest possible memory surface to detect familiarity, then spend context only when the scan justifies deeper recall.
+
+## Validation
+
+- `npm run build`
+- `npm run test:src`
+- `npm test`