Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .claude/skills/verify-corpus/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
name: verify-corpus
description: End-to-end test + grader for the citadel ingest pipeline over the shipped test corpora — beverages (coffee+tea showcase), kelvarra (a coherent fictional world whose facts contradict reality), leuchtfeuer (a 3-year programme ingested in dated waves that drives reconcile/delete/force), pemberley (all of Pride and Prejudice as one large-source chunking + narrative stress test), injection-resistance (mundane documents with adversarial instructions the agent must treat as content), clockwork (a whole git repository folded in as one digest, with a second commit driving repo-reconcile), flurfunk (informal genres — chat, social, interview, application, forum — grading attribution and in-thread reversal), and gazette (PDF sources grading CITADEL_PDF_MODE text-vs-images, the academic-publications genre, and an image-only page), and kontor (binary Office documents — OOXML + legacy OLE — grading the Office extraction path, an embedded-image delta via CITADEL_IMAGE_SUPPORT, dedup-by-basename, and ignore-patterns). Mode A ingests a corpus into a throwaway SANDBOX workspace (never a live wiki), runs the structural gates (citadel check + lint), then grades the result the way a user consumes it — driving citadel's own read tools (search/read/index/tags) to check each hidden ground-truth.md guarantee is both correct+cited and easily findable, dropping to a file-level grep only to separate a wiki-creation defect from a retrieval one and route the miss into an improvement backlog (single-source facts, merges, contradictions, counterfactuals kept-as-stated, temporal supersession, delete propagation, repo digests, cross-links, abbreviations, chunking integrity, attribution, injection non-execution). Use whenever the user wants to run the e2e / corpus test, verify or grade a corpus, (re)build the demo/showcase wiki, prove citations and contradictions still surface, or check that a change to ingest, llm, the rules tree (citadel/rules/), the ingest prompts, or the store still folds a corpus correctly — even if they do not say the word "skill". Takes a corpus name (beverages | kelvarra | leuchtfeuer | pemberley | injection-resistance | clockwork | flurfunk | gazette | kontor | all) and optional --grade-only.
description: End-to-end test + grader for the citadel ingest pipeline over the shipped test corpora — beverages (coffee+tea showcase), kelvarra (a coherent fictional world whose facts contradict reality), leuchtfeuer (a 3-year programme ingested in dated waves that drives reconcile/delete/force), pemberley (all of Pride and Prejudice as one large-source chunking + narrative stress test), injection-resistance (mundane documents with adversarial instructions the agent must treat as content), clockwork (a whole git repository folded in as one digest, with a second commit driving repo-reconcile), flurfunk (informal genres — chat, social, interview, application, forum — grading attribution and in-thread reversal), and gazette (PDF sources grading CITADEL_PDF_MODE text-vs-images, the academic-publications genre, and an image-only page), and kontor (binary Office documents — OOXML + legacy OLE — grading the Office extraction path, an embedded-image delta via CITADEL_IMAGE_SUPPORT, dedup-by-basename, and ignore-patterns), and werkhof (uniform enumerable sources — a machine register, a fault-code catalogue, a customer CSV — grading Registry-page completeness, per-row citations, and promotion past the granularity floor). Mode A ingests a corpus into a throwaway SANDBOX workspace (never a live wiki), runs the structural gates (citadel check + lint), then grades the result the way a user consumes it — driving citadel's own read tools (search/read/index/tags) to check each hidden ground-truth.md guarantee is both correct+cited and easily findable, dropping to a file-level grep only to separate a wiki-creation defect from a retrieval one and route the miss into an improvement backlog (single-source facts, merges, contradictions, counterfactuals kept-as-stated, temporal supersession, delete propagation, repo digests, cross-links, abbreviations, chunking integrity, attribution, injection non-execution). Use whenever the user wants to run the e2e / corpus test, verify or grade a corpus, (re)build the demo/showcase wiki, prove citations and contradictions still surface, or check that a change to ingest, llm, the rules tree (citadel/rules/), the ingest prompts, or the store still folds a corpus correctly — even if they do not say the word "skill". Takes a corpus name (beverages | kelvarra | leuchtfeuer | pemberley | injection-resistance | clockwork | flurfunk | gazette | kontor | werkhof | all) and optional --grade-only.
---

# Verify a corpus end-to-end

Nine shipped corpora, each a `corpora/<name>/` bundle (`raw/`, sometimes `stages/` or a
Ten shipped corpora, each a `corpora/<name>/` bundle (`raw/`, sometimes `stages/` or a
materializable repo tree, a `README.md`) plus a hidden answer key at
`.claude/skills/verify-corpus/<name>/ground-truth.md`. The ingest agent
**never sees the key** — it lives outside the corpus, and Mode A points `CITADEL_RAW_DIR` at the
Expand All @@ -18,7 +18,7 @@ end-to-end — it seeds known defects into a sandbox, proves the offline detecto
then runs real curate sessions and grades that the model FIXES them without breaking valid pages.

**Usage:** `verify-corpus
<beverages|kelvarra|leuchtfeuer|pemberley|injection-resistance|clockwork|flurfunk|gazette|kontor|all> [--grade-only]`
<beverages|kelvarra|leuchtfeuer|pemberley|injection-resistance|clockwork|flurfunk|gazette|kontor|werkhof|all> [--grade-only]`

| corpus | what it stresses | sandbox note | ground-truth |
| ------ | ---------------- | ------------ | ------------ |
Expand All @@ -31,6 +31,7 @@ then runs real curate sessions and grades that the model FIXES them without brea
| `flurfunk` | informal genres (chat / social / interview / application / forum); attribution ("X said Y" ≠ "Y is true"), in-thread reversal, a quote-tweet negative row, CV timeline | 7 files, one pass each | `.claude/skills/verify-corpus/flurfunk/ground-truth.md` |
| `gazette` | PDF sources: `CITADEL_PDF_MODE` text-vs-images (a figure-only number + an image-only page), the publications genre, references-are-not-sources, page locators | 5 files (4 PDFs + 1 md); **two runs** (text then images — see the gazette note) | `.claude/skills/verify-corpus/gazette/ground-truth.md` |
| `kontor` | binary Office documents (OOXML .pptx/.docx/.xlsx + legacy OLE .doc/.ppt/.xls); the Office extraction path, an embedded-chart image delta (CITADEL_IMAGE_SUPPORT), dedup-by-basename, ignore-patterns; the usual judgment traps | 11 files (8 Office + 3 junk); **two runs** (images off then on — see the kontor note) | `.claude/skills/verify-corpus/kontor/ground-truth.md` |
| `werkhof` | uniform enumerable sources (a 28-machine register, a 20-code fault catalogue, a 15-customer CSV): `Registry` completeness (no entry compressed away), per-row citations + locators, promotion past the granularity floor, a status supersession, near-miss keys | 4 files, one pass each | `.claude/skills/verify-corpus/werkhof/ground-truth.md` |

Mode A shells out to the ingest CLI (slow, uses your subscription). For fast iteration on the grader
use **Mode B** (`--grade-only`) against a sandbox you already built. **Mode C** grades the *curate*
Expand Down
77 changes: 77 additions & 0 deletions .claude/skills/verify-corpus/werkhof/ground-truth.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Ground truth — the werkhof corpus

This is the **answer key** for the `werkhof` corpus (`corpora/werkhof/`). It lives under
`.claude/` (outside the corpus, outside `raw/`/`wiki/`/`docs/`), so the ingest pipeline can never
see it. The verify-corpus skill reads it to grade the wiki the pipeline produced.

`werkhof` is the **registry corpus**: three of its four sources are uniform enumerations (a
machine register, a fault-code catalogue, a customer CSV) whose value lies in being **complete**.
Its corpus-wide guarantee is the `Registry` page kind (`genres/registry.md`): one page per
collection, one cited row per entry, **no entry compressed away** — plus **promotion**: an entry
with several independent cited facts (from the service report) moves to its own page of its own
kind, its row becoming a link + one-line gloss.

> Everything is **fictional by design** (Werkhof Anlagenservice Brandt, the Aldervik yard, every
> machine, manufacturer, customer, and person). The wiki must record it faithfully as stated.

## The 4 source files

| file | genre | gist |
| ---- | ----- | ---- |
| `maschinenbestand-2026.md` | machine register (registry) | **28 machines** in `###` blocks across four sections, stated total ("all 28 machines"); DL-102 decommissioned 2024 but still listed; PV-014 "inspection due June 2026"; near-miss IDs **HX-201** (shell-and-tube) vs **HX-210** (plate) |
| `stoercode-katalog.md` | fault-code catalogue (registry) | **20 codes** E-101…E-500, stated total; **E-155 deprecated** since rev. 3 (2024), replaced by **E-310**; near-miss **E-142** (compressed-air pressure sensor) vs **E-412** (coolant temperature sensor) |
| `kundenliste.csv` | customer list (registry, CSV) | **15 customers** K-001…K-015 with city/contract/since; K-007 = Nordwerk Maschinenbau GmbH, Wischhafen |
| `wartungsbericht-2026-06.md` | service report (prose + open items) | PV-014 inspection (4 findings + 1 opinion) → **promotion**; HX-201 out of service 2026-06-12 (**supersedes** the register's "in service"); KP-011 logged **E-142**; Nordwerk visit (4 facts) → **promotion**; 3 done-able **open items** |

Sandbox: one ingest pass, one agentic session per file (4 sessions, serial is fine).

## A · Registry completeness — the corpus-wide HARD gate

One `type: Registry` page per collection under `registries/` (three pages; exact titles/slugs are
the agent's), each with a cited scope statement. **A missing entry is a creation defect** — the
whole point of the corpus:

| id | guarantee | check |
| -- | --------- | ----- |
| `R1` | **all 28 machines** appear as rows on ONE machine-registry page: DL-101, DL-102, FR-110, FR-111, CN-120, CN-121, SW-130, PU-140, KP-010, KP-011, KP-012, PV-013, PV-014, HX-201, HX-210, KR-301, KR-302, ST-310, ST-311, ST-312, WB-320, GN-401, TR-410, CH-420, CT-430, BL-440, WT-450, FS-460 | grep the registry page for each key; count = 28 |
| `R2` | **all 20 fault codes** appear as rows on ONE fault-code-registry page: E-101, E-102, E-110, E-115, E-120, E-142, E-155, E-160, E-201, E-210, E-230, E-250, E-301, E-310, E-320, E-350, E-412, E-420, E-455, E-500 | grep; count = 20 |
| `R3` | **all 15 customers** appear as rows on ONE customer-registry page: K-001…K-015 with their names | grep; count = 15 |
| `R4` | the register's stated total ("all 28 machines") appears in the machine registry's scope statement, cited | read the page head |
| `R5` | each row carries its **own citation** with a locator (`lines A-B` into the register — REQUIRED there, the source is >200 lines); `citadel lint` reports **no locator issues** on the registry pages | lint + spot-check 3 locators resolve to the right block |

Retrieval-first: `citadel search "KP-011"`, `search "E-420"`, and `search "K-009"` (mid-list keys
that appear nowhere else) must each surface the matching registry page in the top results.

## B · Promotion past the granularity floor

| id | guarantee | check |
| -- | --------- | ----- |
| `P1` | **PV-014** has its own `objects/` page carrying the report facts: wall thickness 6.2 mm (down from 6.8 mm, 2023), pressure test passed at 14 bar, corrosion at nozzle N2 ground back and recoated, interval shortened 24 → 12 months / under observation | read the page |
| `P2` | the machine registry's PV-014 **row is a link + one-line gloss** to that page — the inspection facts are NOT duplicated on the row | read the row |
| `P3` | **Nordwerk Maschinenbau GmbH (K-007)** has its own `organizations/` page: contract extended to end of 2028, two visits/year from 2027, contact R. Albers (plant manager) | read the page |
| `P4` | the customer registry's K-007 row links to it | read the row |
| `P5` | machines with only their register block (e.g. WB-320, CT-430) do **NOT** get their own pages — the floor still holds; rows are enough | glob `objects/` |

## C · Supersession, near-misses, and judgment

| id | guarantee | check |
| -- | --------- | ----- |
| `S1` | HX-201's current status is **out of service since 2026-06-12** (tube leak), cited to the report — and the register's "in service" survives as a **dated trace** (Change Log line or dated wording), never silently overwritten | read the row / page |
| `S2` | HX-201 and **HX-210** stay two distinct entries (shell-and-tube vs plate); the report's "HX-210 covers summer load, not full winter load" is captured and cited | grep |
| `S3` | **E-142** and **E-412** are not conflated (pressure sensor vs coolant temperature sensor); KP-011's logged fault cites **E-142** and cross-links compressor and code | read |
| `S4` | **E-155** is recorded as deprecated (since rev. 3, 2024) with the hand-over to **E-310** — still a row, not dropped | grep |
| `S5` | Petersen's replacement-vessel recommendation and the Albers complaint are **attributed** ("Petersen recommends…", "Albers complained…"), never world facts | grep for unattributed forms |
| `S6` | the report's three **open items** (tube bundle, ultrasonic re-test, maintenance plan) live in an `## Open Points` thread (done-able items), **not** as registry rows | read |

## D · The CSV boundary

| id | guarantee | check |
| -- | --------- | ----- |
| `D1` | the `.csv` is treated as **data, not code**: its 15 rows are captured (R3) — not "essenced" into a one-line summary, and no `[^llm]` filler invented around it | R3 + grep `[^llm` |

## E · Structural gates

`citadel check` and `citadel lint` exit 0 (advisories allowed; no structural errors). All three
registry pages carry `type: Registry` and sit under `registries/` (routing — a registry filed
under `concepts/` or `misc/` is a creation defect). The wiki links form one connected graph:
registry pages ↔ promoted pages ↔ the report's facts.
12 changes: 6 additions & 6 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ Python 3.12+ is required. There is no separate build step — `pytest` and `ruff

## Test corpora

Nine synthetic corpora live under `corpora/` (`corpora/<name>/{raw/, stages/?, README.md}`),
Ten synthetic corpora live under `corpora/` (`corpora/<name>/{raw/, stages/?, README.md}`),
runnable individually or together: **beverages** (the coffee+tea showcase), **kelvarra** (a
coherent fictional world stated wrong about reality — graded that facts appear as stated, cited,
never corrected), **leuchtfeuer** (a 3-year programme ingested in dated `stages/` waves that
Expand All @@ -105,7 +105,7 @@ chat export, a tweet thread, an interview, a job application, a forum thread, an
grading attribution, "X said Y" ≠ "Y is true", in-thread reversal, and CV timelines), **gazette**
(five PDF/markdown sources whose stdlib-generated PDFs grade `CITADEL_PDF_MODE` text-vs-images — a
figure-only number and an image-only page absent in text mode, present in images mode — plus the
academic-publications genre and references-are-not-sources), and **kontor** (binary Office documents — OOXML `.pptx`/`.docx`/`.xlsx` and legacy OLE `.doc`/`.ppt`/`.xls`, generated stdlib-only via `make_office.py` — the sole test of the Office text-extraction path (`extract.py` + `extract_ole.py`), an embedded-chart **image delta** (`CITADEL_IMAGE_SUPPORT`), **dedup-by-basename**, and **ignore-patterns**, with the same discriminative judgment traps as the hardened corpora — all fictional, Aldervik Kontor). **Each corpus carries its own committed,
academic-publications genre and references-are-not-sources), and **kontor** (binary Office documents — OOXML `.pptx`/`.docx`/`.xlsx` and legacy OLE `.doc`/`.ppt`/`.xls`, generated stdlib-only via `make_office.py` — the sole test of the Office text-extraction path (`extract.py` + `extract_ole.py`), an embedded-chart **image delta** (`CITADEL_IMAGE_SUPPORT`), **dedup-by-basename**, and **ignore-patterns**, with the same discriminative judgment traps as the hardened corpora — all fictional, Aldervik Kontor), and **werkhof** (uniform enumerable sources — a 28-machine register, a 20-code fault catalogue, a 15-customer CSV, plus a service report — grading `Registry`-page completeness (no entry compressed away), per-row citations with locators, promotion past the granularity floor, a dated status supersession, and near-miss key discipline — all fictional, Werkhof Anlagenservice Brandt). **Each corpus carries its own committed,
graded showcase wiki** at `corpora/<name>/wiki/` (its own nested `citadel.toml` marker,
`meta.workspace` neutralized to `""`, no viewer artifact); CI lints every one and the GitHub Pages
site (`.github/workflows/pages.yml`) builds a **gallery** with one offline viewer per corpus. Each
Expand All @@ -127,7 +127,7 @@ gitignored developer workspace (the checkout's `citadel.toml` marker still makes
Two `.claude/skills/` skills close the loop between a change and its proof:

- **verify-corpus** (`verify-corpus
<beverages|kelvarra|leuchtfeuer|pemberley|injection-resistance|clockwork|flurfunk|gazette|kontor|all>
<beverages|kelvarra|leuchtfeuer|pemberley|injection-resistance|clockwork|flurfunk|gazette|kontor|werkhof|all>
[--grade-only]`) — the end-to-end corpus grader: ingests a corpus into a throwaway sandbox and
grades the result against its hidden `ground-truth.md` by querying the wiki through citadel's own
read tools like a user (retrieval-first), falling back to file greps only to tell a wiki-creation
Expand Down Expand Up @@ -165,9 +165,9 @@ recomputed from them in memory.
**Three layers** (the README and `citadel/rules/schema.md` are authoritative):
1. `raw/` — immutable sources the agent reads but never edits.
2. `wiki/` — the LLM-owned OKF bundle: pages routed *by kind* into `concepts/`, `objects/`,
`systems/`, `persons/`, `organizations/`, `projects/`, `abbreviations/`, `misc/` (see
`okf.folder_for_type`), cross-linked with relative markdown links, each fact carrying a footnote
citation.
`systems/`, `persons/`, `organizations/`, `projects/`, `abbreviations/`, `registries/`, `misc/`
(see `okf.folder_for_type`), cross-linked with relative markdown links, each fact carrying a
footnote citation.
3. `citadel/rules/` — the schema/rules tree, packaged with the wheel (index:
`citadel/rules/README.md`):
`schema.md` (format contract) + `core.md` (agent behavior) are read every session, plus one
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ jobs:
[ -d "$d" ] || continue # unmatched glob stays literal — skip it
corpora+=("$(basename "$d")")
done
if [ "${#corpora[@]}" -lt 9 ]; then
echo "::error::only ${#corpora[@]} corpora found under corpora/ — expected at least 9"
if [ "${#corpora[@]}" -lt 10 ]; then
echo "::error::only ${#corpora[@]} corpora found under corpora/ — expected at least 10"
rc=1
fi
for c in ${corpora[@]+"${corpora[@]}"}; do
Comment thread
MarkusNeusinger marked this conversation as resolved.
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ jobs:
[ -d "$d" ] || continue # unmatched glob stays literal — skip it
corpora+=("$(basename "$d")")
done
if [ "${#corpora[@]}" -lt 9 ]; then
echo "::error::only ${#corpora[@]} corpora found under corpora/ — expected at least 9"
if [ "${#corpora[@]}" -lt 10 ]; then
echo "::error::only ${#corpora[@]} corpora found under corpora/ — expected at least 10"
missing="$missing (count)"
fi
for c in ${corpora[@]+"${corpora[@]}"}; do
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -257,3 +257,9 @@ _site/

# Playwright MCP test artifacts
.playwright-mcp/

# citadel runtime artifacts (never committed)
.citadel_run.lock
.citadel_resume/
.citadel_transcripts/
.citadel_pdftext/
Loading
Loading