From 4c60f3e438fbe00690925f802176f8b42d7fddbd Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Tue, 14 Jul 2026 15:56:30 -0400 Subject: [PATCH 1/8] [harness] Add documentation-factory FSM engine and strict-build gate TECH.md YAML-FSM helpers (_fsm/next_phase/set_phase) ported from the HyperShell factory and adapted to python3; strict_check.py gates 'mkdocs build --strict' against strict-baseline.txt (7 known pre-existing warnings) so only regressions block. --- .agents/factory/bin/_fsm.py | 209 ++++++++++++++++++++++++++++ .agents/factory/bin/next_phase.py | 80 +++++++++++ .agents/factory/bin/set_phase.py | 122 ++++++++++++++++ .agents/factory/bin/strict_check.py | 106 ++++++++++++++ .agents/factory/strict-baseline.txt | 31 +++++ 5 files changed, 548 insertions(+) create mode 100644 .agents/factory/bin/_fsm.py create mode 100755 .agents/factory/bin/next_phase.py create mode 100755 .agents/factory/bin/set_phase.py create mode 100755 .agents/factory/bin/strict_check.py create mode 100644 .agents/factory/strict-baseline.txt diff --git a/.agents/factory/bin/_fsm.py b/.agents/factory/bin/_fsm.py new file mode 100644 index 00000000..1a10faca --- /dev/null +++ b/.agents/factory/bin/_fsm.py @@ -0,0 +1,209 @@ +"""Shared helpers for the RCAC-Docs documentation-factory FSM scripts. + +The finite-state machine for a documentation job (a `feature`/`fix`/`refactor`) +lives in the YAML frontmatter of its ``spec//TECH.md`` roadmap. These +helpers read, validate, mutate, and re-serialize that frontmatter so the +*scripts* (not the model) own the fragile YAML arithmetic — model in-context +YAML editing is the primary FSM-corruption risk (see +``.agents/factory/methodology.md``). + +Requires PyYAML, which is a declared project dependency (``requirements.txt``) +and lives in the project virtual environment — NOT the system interpreter. Run +these scripts with the project env active, e.g. from the repo root:: + + source .venv/bin/activate # or: conda activate rcac-docs + python3 .agents/factory/bin/ + + + + + +
+
+ RCAC docs · factory + +
+
+ +
+
+

RCAC Docs · Agentic Authoring · Getting Started

+

From chatbots to a documentation factory

+

A guide to agentic documentation authoring for the RCAC docs team — what an + agent really is, how a harness turns context into action, and how a lightweight, + auditable methodology (the .agents/factory) makes authoring and maintaining this + site repeatable, reviewable, and safe.

+
+ ~20 min read + Executive orientation ~5 min + Audience: contributors & leadership + Self-contained · works offline +
+
+
+ +
+
+ + +
+

Executive orientation

+

Why this, why now #

+

For a decade our tools have answered questions. In the last two years they + began to do work. That is the whole story — and it changes how technical documentation + gets written and maintained.

+ +

A chatbot produces text. An agent is given a goal, a set of tools, and the + latitude to use them in a loop: it reads the existing pages, builds the site, checks that links + resolve and pages render, edits Markdown, and inspects the result — narrating and course-correcting + as it goes. The thing that made this leap possible is not a bigger model alone; it is the + harness — the software (Claude Code and peers) that assembles a critical mass + of context and grants the model the ability to act. Context plus action, in a + loop, is where the wellspring of productivity comes from.

+ +

That power needs a channel. Left unstructured, an agent will happily write the wrong page quickly, + or the right page in a way you cannot review — or, worse for a user guide, document a + #SBATCH flag that does not exist. The discipline that channels it is + method — and the good parts of agile practice (especially 37signals' + Shape Up) map cleanly onto agent work. This site packages that method as a small, + transparent toolkit in .agents/: five skills and a set of reference + documents that carry a change from a shaped goal, through a researched plan, through authoring, + through an adversarial review, to a pull request — with a durable, committed record at every step.

+ +

What changes for the docs team #

+

The level of abstraction rises. People spend less time on boilerplate scaffolding, nav wiring, and + consistency chores, and more time doing the things only people can do well: shaping + what is worth documenting, judging whether what came back is accurate and clear, + and owning the result. The agent executes; the human sets direction and holds the + bar — and for HPC instructions people run on real clusters, that bar is high.

+ +
+
+

What the factory is

+
    +
  • A repeatable lifecycle: shape → plan → draft → review → ship.
    • +
  • Plain Markdown + git + skills — no new platform, no lock-in.
    • +
  • An auditable trail: intent, design, and content, committed together.
    • +
  • Human judgement at every gate that matters — and review before production.
    • +
+
+
+

What it is not

+
    +
  • Not "the AI writes everything and we hope." Every step is reviewable.
    • +
  • Not a replacement for writers or RSEs — it raises what they work on.
    • +
  • Not a heavyweight process forced onto a typo fix (it scales down).
    • +
  • Not a frozen spec: the plan is a living artifact, the goal is the contract.
    • +
+
+
+ +

For institutional leadership, at a glance #

+

The four questions leaders ask most, answered in one line each — with a fuller treatment in + Notes for institutional leadership.

+
+
+
Governance & guardrails
+

Human sign-off gates, high-impact-file controls, and an independent adversarial review step are + engineered into the lifecycle — critical because a merge to main deploys straight + to production.

+
+
+
Cost & effort economics
+

Ceremony scales to the size of the change (an appetite), so a typo fix does not pay + for a new user guide's process. You spend rigor where it earns its keep.

+
+
+
Reproducibility & auditability
+

Every job leaves a committed record — goal, design, and content, traceable requirement → + commit. A paper trail for why a page says what it says.

+
+
+
Workforce
+

This augments the docs team. Humans shape, judge, and own; agents execute. The scarce skill + becomes good judgement about accuracy and clarity, which your people already have.

+
+
+ + +
+ + +
+

Practitioner guide · 1

+

Agents, not chatbots #

+

A chatbot is a function from prompt to text. You ask; it answers; the transaction ends. Useful, + but it does nothing — you still paste the Markdown, build the site, find the broken link, + and come back.

+ +

An agent closes that loop itself. Given a goal and a set of tools, it + runs a cycle:

+ +
+
01
Perceive
read pages, build the site, inspect state
+
→
+
02
Reason
decide the next concrete action
+
→
+
03
Act
edit Markdown, update nav, run mkdocs
+
→
+
04
Observe
read the result, then loop
+
+ +

The difference is not eloquence; it is agency. Because it observes the outcome of + each action, an agent can recover from a broken build, chase down a bad link, or notice that the + page it just wrote never made it into the nav. This is why "it drafted some text" (2022) and "it + shipped a reviewed page" (2026) are different eras.

+ +
+
+

Chatbot

+ 
# one shot, no hands
+you   ❯ write a Running Jobs page
+bot   ❯ here's some Markdown…
+you   ❯ # you paste it, the build
+        # warns, the link 404s…
+
+
+

Agent

+ 
agent ❯ read userguides/gautschi/run_jobs
+agent ❯ edit: add a #SBATCH example
+agent ❯ run: mkdocs build --strict
+agent ❯ # warn → fix link → clean → commit
+
+
+ +
+
The catch
+

Agency cuts both ways. An agent can write the wrong thing, or an inaccurate thing, just as fast + as the right thing well. Everything after this section is about channelling that agency — + giving it the right context, the right method, and the right gates.

+
+
+ + +
+

Practitioner guide · 2

+

The harness: where context becomes action #

+

The model is the engine; the harness is the vehicle. A coding/authoring harness + (Claude Code and peers) does two jobs at once, and their combination is the productivity + unlock:

+ +
+
+ Context in +

A critical mass of context

+

The harness feeds the model far more than your prompt:

+
    +
  • The repository — every page, read on demand, searched, navigated.
    • +
  • Project memory — an AGENTS.md + that states the conventions and invariants of this site.
    • +
  • Tools — the shell, mkdocs, git, the web.
    • +
  • Skills — packaged procedures the agent can invoke by name.
    • +
+
+
+ Action out +

The ability to act

+

And it lets the model do things, under permission control:

+
    +
  • Edit and create pages across the site; wire up the nav.
    • +
  • Run commands — build the site, check links, regenerate breadcrumbs.
    • +
  • Use git; open pull requests.
    • +
  • Spawn sub-agents to research in parallel.
    • +
+
+
+ +

Put a loop around those two and you have the whole machine: gather context → decide → act → + observe → repeat. The art — "context engineering" — is getting the right context in + front of the model at the right moment, because a model's attention is finite and its session + memory is not durable across restarts.

+ +
+
The load-bearing idea
+

Because a fresh session starts blank, durable state must live in files and git, not in + the chat. That single fact is why the factory writes everything down — the goal, the + plan, the roadmap, the review — and reads it back fresh each time. It is what lets a draft resume + next week on another machine, and lets a reviewer start with no memory of how the page was + written.

+
+ +

Skills: procedures the agent can run #

+

A skill is a Markdown file (SKILL.md) with a little front-matter and + a body of instructions. When you type its name as a slash command — say /docs-draft — + the harness loads those instructions, injects live facts (the current branch, the git status), and + the agent follows the procedure. Skills are how you capture "the way we author here" once, so nobody + re-explains it every time. The five skills in this guide are exactly that: the RCAC-Docs way to take + a change from idea to merge.

+
+ + +
+

Practitioner guide · 3

+

Method: Shape Up meets agents #

+

Give a capable agent no method and you get motion without direction. The good news: much of what + teams already learned about directing work applies — you just keep the thinking tools and + drop the calendar rituals.

+ +

The sharpest fit is Shape Up (Ryan Singer / 37signals). Its core ideas translate + almost directly to a solo-author-plus-agent pipeline:

+ +
+ + + + + + + + + + + + + + + + +
Shape Up ideaIn the factory
Appetite — fix the time, vary the scope.An appetite: small|big field sizes how much research and how many phases a job gets. The plan fits the appetite, not the reverse.
Shaping — rough, solved, and bounded, at the right altitude.The GOAL.md is shaped: concrete enough to de-risk, abstract enough to leave design freedom.
No-gos — say what you will not do.An explicit non-goals list keeps scope from creeping ("not documenting cluster X").
Rabbit holes — name the scary unknowns and settle them first.Each research brief investigates one unknown — often an HPC spec to verify against an authoritative source.
Hill — uphill (figuring out) vs downhill (just writing).Each phase carries a hill signal; a phase stuck "uphill" is a raised hand.
Circuit breaker — don't extend indefinitely; stop and re-shape.Draft iterations and review bounce-backs are capped; on trip, stop and re-plan.
+
+ +

What we deliberately discard: the betting table, six-week cycles, cool-down, and + separate shaper/builder/QA roles — those exist to synchronize a human team and dissolve + for a serial solo-plus-agent flow.

+ +
+
One place Shape Up must not be followed
+

Shape Up assumes scope is always cuttable and most defects can wait. That is + false for documentation people run on real clusters: a wrong #SBATCH + flag, a partition that doesn't exist, an inaccessible table, or a broken link that silently ships + to production is not a deferrable defect. The factory marks HPC technical accuracy, WCAG + accessibility, and build integrity as hammerable: false — the reviewer may never trim + them to "fit the appetite."

+
+ +

The broader movement this belongs to — committing a specification alongside + the content and letting the spec drive the work — is often called spec-driven + development. Tools like GitHub Spec Kit and AWS Kiro converge on the same + what/why → how → ordered-work spine the factory uses. Our version is deliberately + lightweight: Markdown, git, and skills, anchored on the site's own AGENTS.md.

+
+ + +
+

Practitioner guide · 4

+

The .agents/factory lifecycle #

+

One documentation job flows through five skills, on its own git branch, leaving a committed trail + under spec/<slug>/. The whole pipeline:

+ +
+
/docs-feature
Shape
GOAL.md — what & why
+
→
+
/docs-plan
Design
research → PLAN → TECH
+
→
+
/docs-draft
Author
one phase at a time ⟲
+
→
+
/docs-review
Judge
blind QA ⟲
+
→
+
/docs-publish
Ship
PR → main
+
+

The ⟲ steps loop: review sends confirmed + problems back to draft until the change is clean, then publish opens the pull request. Merging to + main auto-deploys to production within ~5 minutes.

+ +

The artifact spine #

+

Each stage reads the previous stage's committed files and writes its own. This is the durable + substrate from §2 in practice:

+
+ + + + + + + + + + + + + + +
FileWritten byWhat it is
GOAL.md/docs-featureThe locked contract: problem, appetite, non-goals, and testable + acceptance criteria with stable IDs (R1, R2…).
PLAN.md/docs-planThe authoritative design: archetype, page structure, nav placement, + macros/snippets, mapped back to each requirement ID.
research/*.md/docs-planBriefs from parallel read-only sub-agents — one per rabbit hole (big appetite only).
TECH.md/docs-planThe phased roadmap — a small state machine (see below) the draft resumes from.
REVIEW.md/docs-reviewSeverity-ranked findings + a requirement→evidence matrix.
+
+ +

TECH.md is a finite-state machine #

+

The roadmap's YAML front-matter is the resume point. It lists phases (publishable units of + work — a page or a coherent section), each with the requirements it satisfies, a verify command, and + honesty signals. The draft skill reads it fresh every time, does the next actionable phase, and + writes the state back — so authoring is fully resumable and git history becomes the checkpoint log.

+
spec/<slug>/TECH.md — front-matter (excerpt)
+ 
appetite: big              # small|big → caps phases & iterations
+status: in_progress
+current_phase: P2
+phases:
+  - id: P1
+    name: "Scaffold: index + overview"
+    satisfies: [R1]             # traces back to GOAL requirement IDs
+    hammerable: false          # HPC accuracy — review may not cut this
+    hill: downhill            # uphill=figuring out · downhill=writing
+    status: done
+    verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py"
+

A pair of small helper scripts (next_phase.py / + set_phase.py) read and update this block, so the agent + never hand-edits fragile YAML.

+ +
+
Why --strict
+

Production CI builds with mkdocs build --verbose, so a broken internal link or a + page missing from the nav ships silently. The factory's verify gate runs + mkdocs build --strict and blocks only on warnings not already in + strict-baseline.txt — catching regressions before + they reach readers.

+
+ +

Build in the open #

+

All of this — the skills, the reference docs, the per-job spec/ folders — is committed + to the repository. The methodology is not a hidden prompt; it is version-controlled, reviewable, and + improvable like any other part of the site. The full rationale lives in + methodology.md.

+
+ + +
+

Practitioner guide · 5

+

Hands-on: the five skills #

+

Each is a slash command you invoke deliberately (none fire on their own). They guard their own + preconditions — a wrong branch or a dirty tree stops them cold — so the process is hard to run out + of order.

+ +
+
/docs-feature — shape the goal
+

From a clean main, creates a feature/<slug>, + fix/<slug>, or refactor/<slug> branch and refines your idea + (or a hand-written GOAL) into a shaped GOAL.md: appetite, non-goals, and testable + criteria. Shaping only — no research, no drafting. Asks you when something is ambiguous rather + than guessing.

+
reads your prompt / an untracked GOAL.md · writes spec/<slug>/GOAL.md · + skill docs-feature/SKILL.md
+
+ +
+
/docs-plan — research & design
+

Checks the idea against the site's invariants, fans out read-only research sub-agents (scaled to + appetite; existing-site-first), then writes the PLAN.md design and the + TECH.md phased roadmap. A small fix can skip the research entirely.

+
reads GOAL.md, AGENTS.md, the site · writes research/, PLAN.md, TECH.md · + skill docs-plan/SKILL.md
+
+ +
+
/docs-draft — author the roadmap
+

The resume-and-write driver. Reads the FSM, authors the next phase, runs that phase's verify + command (it proves the work by building and rendering the site, not by ticking a box), + and makes one atomic commit of content + state. It may revise the plan as reality demands, but a + contradiction with the locked GOAL stops it for you.

+
reads TECH.md, PLAN.md, the pages · writes docs/, nav, TECH.md state · + skill docs-draft/SKILL.md
+
+ +
+
/docs-review — adversarial QA
+

Independent quality assurance. It delegates to a fresh reviewer with no memory of how the + page was written, who sees only the goal, the diff, and the invariants — not the + author's own plan. It verifies by building and rendering the site, and separates what it can prove + by command (broken links, nav, front-matter) from what needs a human (HPC accuracy, prose + accessibility). Confirmed issues loop back to draft; high-impact files always require a human + sign-off.

+
reads GOAL.md + diff + invariants (blind to PLAN/TECH) · writes REVIEW.md, findings · + skill docs-review/SKILL.md
+
+ +
+
/docs-publish — ship it
+

Opens a pull request to main with a rich, artifact-linked description (or does a + local squash-merge). This is the one irreversible, outward step — a merge deploys to production — + so it always confirms with you first and never targets dev.

+
reads all artifacts + git log · writes a PR (or a local merge) · + skill docs-publish/SKILL.md
+
+ +

A day in the life #

+
a new user-guide page, start to finish
+ 
/docs-feature add a Python examples page to the Gautschi user guide
+# → branch feature/gautschi-python, spec/gautschi-python/GOAL.md — you review it
+
+/docs-plan
+# → research/, PLAN.md, TECH.md — you approve the design (archetype, nav spot, macros)
+
+/docs-draft          # phase P1: scaffold + nav … commit
+/docs-draft          # phase P2: draft the page … commit  (repeat until done)
+
+/docs-review         # fresh, blind QA → REVIEW.md (approved / changes-requested)
+
+/docs-publish        # opens the PR to main, links every artifact
+
+
Scales down
+

A typo fix or a one-line correction does not need all of this. Set appetite: small + (the default for fix/ branches) and /docs-plan collapses to a lean plan + with no research fan-out. A truly trivial change can skip the lifecycle altogether. Rigor is a + dial, not a toll.

+
+
+ + +
+

For institutional leadership

+

Notes for institutional leadership #

+

The at-a-glance panel above, expanded. These are the concerns that decide whether the + docs team can adopt agentic authoring responsibly.

+ +

Governance & guardrails #

+

Autonomy is bounded by design, not by hope:

+
    +
  • Human gates at the decisions that matter. A human approves the goal before + the planning, the design before pages are written, and the review before anything ships.
    • +
  • Independent review. The QA step runs in a fresh context, blind to the author's + reasoning, and verifies by building and rendering the site — a direct countermeasure to a model + rubber-stamping its own work. Because there is no staging environment, this review is the last + line of defense before production.
    • +
  • High-impact-file controls. Any confirmed problem touching a shared surface + (mkdocs.yml, the macros in main.py, the catalog generators, the theme + overrides, CI) forces a mandatory human sign-off.
    • +
  • Encoded invariants. The site's non-negotiable rules — branch/deploy model, + the generated-content firewall, HPC accuracy, WCAG accessibility — live in + invariants.md and are checked at both planning and + review; violating one is an automatic top-severity finding.
    • +
+ +

Cost & effort economics #

+

Agent work consumes tokens (and therefore money and time), most of all during broad research + fan-out. The factory treats spend as a dial:

+
    +
  • Appetite scaling. Small changes skip research and collapse the paperwork; only + genuinely large jobs (a new cluster guide, a tutorial series) pay for parallel investigation.
    • +
  • Parallel where it pays, serial where it must. Research fans out (large, + well-understood gains); authoring stays single-threaded (parallel authors make conflicting + decisions about voice, structure, and nav).
    • +
  • Circuit breakers. A stuck phase or a review that keeps bouncing trips a stop, + rather than burning budget in a loop — a spend ceiling with a human off-ramp.
    • +
+ +

Reproducibility & auditability #

+

Every job leaves a committed, dated record:

+
    +
  • A traceable chain from requirement ID → design element → implementing commit → + review evidence. You can answer "why does this page say this, and who decided?" from the + repository alone.
    • +
  • An ADR-style trail. The spec/<slug>/ folder is retained on + merge as a point-in-time record of intent and design — not a living doc to maintain forever.
    • +
  • The site remains ground truth. Specs record intent; the rendered site and + AGENTS.md are authoritative. This avoids the classic trap of stale documentation + pretending to be the source of truth.
    • +
+ +

Workforce #

+

The honest framing: this raises the level of abstraction at which the docs team + works. The routine scaffolding is increasingly automated; the durable, human contributions become + more valuable, not less:

+
    +
  • Shaping — deciding what is worth documenting and bounding it well.
    • +
  • Judgement — knowing whether a page is accurate, clear, and accessible.
    • +
  • Ownership — standing behind what ships to users.
    • +
+

These are exactly the skills experienced writers and RSEs already have. The change is that they + spend more of their day exercising them and less on boilerplate. It is a force multiplier for a + small docs team, not a substitute for one.

+
+ + +
+

Practitioner guide · 6

+

Reference: every material, linked #

+

Everything the factory is made of, with what it is and how it is used. All paths are relative to + .agents/factory/ (this file's home).

+ +

The constitution #

+
+ + + + + + +
MaterialWhat it is / how it's used
AGENTS.md (repo root)The site's operating manual and the factory's constitution: architecture, + content conventions, the branch/deploy model, the invariants. Skills reference it; they + never duplicate it. (CLAUDE.md is a symlink to it.)
+
+ +

The five skills #

+
+ + + + + + + + + +
SkillRole
docs-featureShape a goal on a fresh branch.
docs-planResearch, design, and lay out the phased roadmap.
docs-draftAuthor the roadmap, one verified phase at a time.
docs-reviewIndependent, blind, evidence-based QA.
docs-publishOpen the pull request to main (or merge locally).
+
+ +

Reference documents #

+
+ + + + + + + + + + + + + + +
DocumentWhat it is / how it's used
methodology.mdThe full lifecycle rationale — the "why" behind the skills, and the Shape Up mapping. + Read this to understand the flow.
invariants.mdThe curated checklist of load-bearing rules, drawn from AGENTS.md. Used as + the planning gate and the review footgun list.
ears.mdTemplates for writing testable acceptance criteria (the EARS convention), recast for + documentation outcomes. Used by /docs-feature.
style-guide.mdThe house style and per-archetype conventions (front-matter, macros/snippets, admonitions, + links, assets, nav). Used by /docs-plan and /docs-draft.
review-rubric.mdThe reviewer's operating manual — what it may see, severity levels, the CONFIRMED vs + PLAUSIBLE split, and when a human must sign off.
+
+ +

Templates & state scripts #

+
+ + + + + + + + + + + + + +
FileWhat it is / how it's used
templates/GOAL.mdSkeleton for the shaped goal.
templates/PLAN.mdSkeleton for the design.
templates/TECH.mdSkeleton for the phased FSM roadmap (with the field reference).
templates/REVIEW.mdSkeleton for the review report.
bin/next_phase.pyReads & validates a TECH.md; prints the next actionable phase as JSON.
bin/set_phase.pyAdvances FSM state by regenerating the front-matter (no fragile hand-edits).
bin/strict_check.pyGates a mkdocs --strict log against the warning baseline (fails only on regressions).
bin/_fsm.pyShared parse / validate / serialize helpers for the scripts above.
strict-baseline.txtThe known pre-existing --strict warnings the build gate ignores.
+
+ +

Further reading #

+ +
+ +
+
+ + + + + + diff --git a/.agents/settings.json b/.agents/settings.json new file mode 100644 index 00000000..182186b6 --- /dev/null +++ b/.agents/settings.json @@ -0,0 +1,19 @@ +{ + "$comment": "Shared, version-controlled permission allowlist for the RCAC-Docs documentation factory. These are safe, read-only-or-idempotent commands the docs-* skills run repeatedly; pre-approving them reduces interactive prompts. Personal overrides go in .agents/settings.local.json (gitignored). Keep this least-privilege — do not add broad wildcards.", + "permissions": { + "allow": [ + "Bash(mkdocs build:*)", + "Bash(.venv/bin/mkdocs build:*)", + "Bash(python3 .agents/factory/bin/next_phase.py:*)", + "Bash(python3 .agents/factory/bin/strict_check.py:*)", + "Bash(python3 -c \"import yaml, mkdocs\":*)", + "Bash(python tools/generate_breadcrumbs.py:*)", + "Bash(git status:*)", + "Bash(git diff:*)", + "Bash(git log:*)", + "Bash(git branch:*)", + "Bash(git rev-parse:*)", + "Bash(grep:*)" + ] + } +} diff --git a/spec/README.md b/spec/README.md new file mode 100644 index 00000000..46e7a45c --- /dev/null +++ b/spec/README.md @@ -0,0 +1,22 @@ +# `spec/` — documentation-factory design records + +Each documentation job run through the factory (see [`../.agents/factory/methodology.md`](../.agents/factory/methodology.md)) +gets a `spec/{slug}/` directory holding its committed, dated design record: + +``` +spec/{slug}/ + GOAL.md # what & why — the locked contract (stable R-IDs), by /docs-feature + PLAN.md # the design (archetype, structure, nav, macros, a11y), by /docs-plan + TECH.md # the phased YAML-FSM the draft step resumes from, by /docs-plan + REVIEW.md # blind adversarial QA findings + verdict, by /docs-review + research/ # read-only research briefs (big appetite only), by /docs-plan +``` + +These are an **immutable, point-in-time record of intent** ("build in the open"), retained on +merge like an architecture decision record — not living documents to maintain forever. The +rendered site and the source tree remain ground truth; `AGENTS.md` is the constitution. + +This directory sits **outside `docs/`**, so MkDocs does not build or publish it. + +The `{slug}` matches the job's branch (`feature/{slug}`, `fix/{slug}`, or `refactor/{slug}`). +Start a new job with `/docs-feature`. From 544119065cd3bc64b0e8068cc2ce659b1917e330 Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Tue, 14 Jul 2026 15:56:31 -0400 Subject: [PATCH 6/8] [harness] Evolve AGENTS.md into the documentation-factory constitution Restructure into operating manual + load-bearing invariants + a 'working as an agent' factory section; drop the Co-Authored-By convention; correct stale Read-the-Docs/S3 references. CLAUDE.md symlinks AGENTS.md. --- AGENTS.md | 434 ++++++++++++++++++++++++------------------------------ CLAUDE.md | 1 + 2 files changed, 197 insertions(+), 238 deletions(-) create mode 120000 CLAUDE.md diff --git a/AGENTS.md b/AGENTS.md index 6a530daa..857ac888 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,244 +1,202 @@ -# RCAC Documentation Website -## Project Overview -This is the MkDocs-based documentation website for the **Rosen Center for Advanced Computing (RCAC)** at Purdue University. It provides comprehensive user guides, tutorials, reference material, software catalogs, datasets, blog articles, and workshop materials for RCAC's HPC systems (Anvil and Gautschi). - -**Primary URL**: https://docs.rcac.purdue.edu (served from the Geddes Kubernetes cluster) -**Source Repository**: https://github.com/PurdueRCAC/RCAC-Docs -### Reference Sites -The following HPC documentation sites use a similar MkDocs Material framework and serve as design/content references: -- https://docs.tacc.utexas.edu/ -- https://docs.alcf.anl.gov/ -- https://docs.ncsa.illinois.edu/en/latest/ -- https://docs.jetstream-cloud.org/ -- https://docs.hpc.ucdavis.edu/ -## Repository Structure -``` -RCAC-Docs/ -├── .github/ -│ ├── ISSUE_TEMPLATE/ # Bug/feature request templates -│ └── workflows/ # GitHub Actions automation -├── docs/ # Main documentation content -│ ├── assets/ # Images, CSS, JS, fonts, data files -│ ├── blog/ # Blog posts -│ ├── datasets/ # Dataset documentation -│ ├── snippets/ # Reusable content snippets -│ ├── software/ # Software catalog pages (auto-generated) -│ ├── stylesheets/ # extra.css and theme overrides -│ ├── userguides/ # User guides (Anvil, Gautschi) -│ ├── workshops/ # Workshop materials (HPC Exchange, Matplotlib, etc.) -│ ├── contact.md # Contact page -│ ├── faq.md # FAQ page -│ ├── index.md # Homepage -│ ├── tags.md # Tag index page -│ └── templates.md # Template examples -├── hooks/ # MkDocs hooks (e.g., social media) -├── k8s/geddes-prod/ # Kubernetes manifests for production deployment -├── modulefiles/ # HPC module files (software installations) -├── overrides/ # Material theme customizations (main.html, partials) -├── tools/ # Automation scripts for software catalog -├── .gitignore -├── .readthedocs.yaml # Read the Docs configuration (legacy/secondary) -├── Dockerfile.geddes-prod # Image built and pushed to GHCR for Geddes -├── main.py # MkDocs macros plugin definitions -├── mkdocs.yml # MkDocs configuration -├── README.md -└── requirements.txt # Python dependencies -``` -## Technology Stack -### Core Technologies -- **MkDocs**: Static site generator (v1.6.1) -- **Material for MkDocs**: Theme (v9.6.16) -- **Python 3.x**: Runtime environment (Dockerfile uses `python:3.11-slim`) -- **GitHub Actions**: CI/CD automation -- **Docker / GHCR**: Production image at `ghcr.io/purduercac/rcac-docs-prod:latest` (served by `nginx:alpine`) -- **Kubernetes (Geddes)**: Production hosting in the `rcac-docs` namespace -### Key MkDocs Plugins -- `mkdocs-material`: Material Design theme -- `mkdocs-material-extensions` -- `material/blog` (built-in blog plugin): Blog functionality with categories and pagination -- `material/tags` (built-in tags plugin): Content tagging -- `mkdocs-git-revision-date-localized-plugin`: Git-based timestamps -- `mkdocs-macros-plugin`: Dynamic content macros (see `main.py`) -- `search`: Built-in search -### Markdown Extensions -- `pymdownx` (details, blocks.caption, emoji, snippets, highlight, inlinehilite, superfences, tabbed) -- `admonition`: Callout boxes -- `tables`: Table support -- `attr_list` and `md_in_html`: HTML attributes / inline HTML in markdown -- `toc`: Table of contents with permalinks -## Content Organization -### Major Sections -1. **HPC User Guides**: Comprehensive guides for Anvil and Gautschi clusters (plus external links to Bell, Negishi, Gilbreth, Scholar) -2. **Storage User Guides**: External links to Data Depot, Fortress, Scratch, Home, Box -3. **Blog**: News and updates categorized by system/topic (Anvil, Gautschi, Software, Slurm, Workflows) -4. **Software Catalog**: Auto-generated catalog of available software -5. **Datasets**: AI, Hydrological, Meteorological, Geospatial, Covariates, GeoAI, Igenomes -6. **Workshops**: HPC Exchange (weeks 1–4), Anvil Kubernetes, Scientific Visualization with Matplotlib -7. **FAQs** and **Contact** pages -### Software Catalog System -The software catalog is **automatically generated** from HPC module files: -**Process Flow**: -1. `modulefiles/` → Module files for installed software -2. `tools/update_apps_inventory.sh` → Extracts app names, versions, clusters → `rcac_apps_inventory.json` -3. `tools/generate_apps_topics.py` → Categorizes apps by topic → `apps_topics.json` -4. `tools/gen_apps_descriptions.py` and `tools/update_apps_descriptions_from_inventory.sh` → Fetches descriptions from Spack / Wikipedia / PyPI → `apps_descriptions.json` -5. `tools/generate_md.sh` → Creates individual markdown files under `docs/software/apps_md/` -6. `tools/update_apps_catalog.sh` → Updates catalog index page `docs/software/app_catalog.md` -**Topics**: MPI, Compilers, Audio/Visual, Climate, Chemistry, Fluid Dynamics, Geoscience, Library, Material Science, Math/Stat, Engineering, Programming, Utilities, Workflow, Misc. -## Automation & Workflows -### Branches -> **Heads-up for agents and new contributors**: despite its name, `dev` is **not** a generic development/integration branch. Do **not** treat it the way you would a `dev` branch in a conventional dev→main flow. Read the section below before pushing, merging, or opening PRs involving `dev`. - -- `main`: **trunk and the only production branch.** All normal content and feature work targets `main` directly via PR. `main` is what deploys to https://docs.rcac.purdue.edu. -- `dev`: a **long-lived, permanently-divergent WCAG 2.1 accessibility remediation branch.** It carries an experimental Material theme override layer (ARIA labels, dedup'd form labels, contrast fixes) that is **not yet ready for production**. It is served separately from its own Geddes environment (`k8s/geddes-dev/`, `Dockerfile.geddes`, `deploy_to_geddes_dev.yml`) for WAVE re-audits. See `dev.md` on the `dev` branch for the remediation plan and phase history. -- Data flow is **main → dev**, not the other way around. `sync_main_to_dev.yml` forward-merges `main` into `dev` on every push so the a11y preview stays current with content, while preserving the dev-only framework layer. -- Direct PRs from `dev` into `main` are **blocked** by CI (`block_dev_to_main.yml`). When the a11y work is complete and accepted, the expected endgame is a deliberate, curated PR of the theme layer into `main` — not an automatic merge. - -#### Guidance for agents -- Default working branch for any content, nav, workflow, infra, or macro change is **`main`** (usually via a short-lived feature/`wip` branch PR'd into `main`). -- Do **not** cherry-pick, rebase, or PR from `dev` into `main` without an explicit human request tied to the a11y rollout. -- When editing files in the dev-only a11y layer (`docs/stylesheets/extra.css`, `overrides/partials/toc.html`, `docs/assets/js/a11y.js`, the `a11y.js` entry in `mkdocs.yml`, `overrides/base.html`, `overrides/partials/{header,logo,nav,nav-item,palette,search}.html`, `dev.md`, `Dockerfile.geddes`, `k8s/geddes-dev/`, `deploy_to_geddes_dev.yml`, `wcag_audit/`), work on `dev` — changes made on `main` to these files can be overwritten or clobbered by the sync workflow's "preserve dev" logic. -- The sync workflow's list of preserved paths is **hardcoded** in `sync_main_to_dev.yml`; if a11y work expands to new files, that workflow must be updated or those files will be overwritten on the next sync. -### GitHub Actions -#### 1. `deploy_to_geddes_prod.yml` — **primary production deployment** -- **Triggers**: Push to `main` (excluding `modulefiles/**`, `tools/**`, `wcag_audit/**`); also manual `workflow_dispatch` -- **Process**: - 1. Logs into GHCR with `GITHUB_TOKEN` - 2. Builds `Dockerfile.geddes-prod` (multi-stage: `python:3.11-slim` → `mkdocs build` → `nginx:alpine`) - 3. Pushes `ghcr.io/purduercac/rcac-docs-prod:latest` -- **Rollout**: The in-cluster `rcac-docs-prod-restart` CronJob (every 5 minutes) runs `kubectl rollout restart deployment/rcac-docs-prod -n rcac-docs`, which re-pulls `:latest` (because `imagePullPolicy: Always`) -#### 2. `publish_to_s3.yml` — secondary/legacy S3 publish -- **Triggers**: Completion of the catalog rebuild workflow, push to `main`/`auto_publish` (excluding `modulefiles/**` and `tools/**`), or `workflow_dispatch` -- **Process**: Builds the site and syncs to the Ceph S3 bucket `s3://rcac-docs/` at `https://s3.rcac.purdue.edu` -- **Status**: No longer the canonical production target; retained for mirror/backup purposes -#### 3. `rebuild_on_module_tools_change.yml` -- **Triggers**: Push to `main` or `dev` when `modulefiles/**` or `tools/**` change -- **Purpose**: Automatically regenerate the software catalog -- **Process**: - 1. Runs `make` in `tools/` - 2. Stages/rebases/commits updated catalog files (`docs/software/apps_md/`, `app_catalog.md`, `index.md`) - 3. Pushes using GitHub App token (`purduercac-docs-bot`) -#### 4. `update_breadcrumbs.yml` -- **Triggers**: Push to `main` or `dev` when `mkdocs.yml` changes -- **Purpose**: Regenerate navigation breadcrumbs -- **Process**: Runs `tools/generate_breadcrumbs.py`, updates `docs/assets/data/breadcrumbs.json`, commits via App token -- **Logic**: Only includes parent titles if parent has an `index.md`; root `/` is excluded -#### 5. `sync_main_to_dev.yml` -- **Triggers**: Push to `main`; manual `workflow_dispatch` -- **Purpose**: Forward-merge `main` into the a11y remediation branch `dev` so it stays current with production content, while **hardcoded-preserving** the dev-only a11y framework layer: `docs/stylesheets/`, `overrides/partials/toc.html`, and the `a11y.js` entry in `mkdocs.yml` (re-added via `sed` if absent). -- **Caveat**: the preserved-paths list is hardcoded. If the a11y work adds or moves files outside that list, this workflow must be updated or those files will be clobbered on the next sync. -#### 6. `block_dev_to_main.yml` -- **Triggers**: Pull requests targeting `main` -- **Purpose**: Fail CI if the PR source branch is `dev`. This is a guardrail against accidentally shipping the experimental a11y theme layer to production — **not** a statement that the workflow is inverted. See the **Branches** section above for the intended model. -### Kubernetes Deployment (Geddes) -Manifests live in `k8s/geddes-prod/` and are applied to the `rcac-docs` namespace on the Geddes cluster: -- `deployment.yaml`: `rcac-docs-prod` Deployment, 1 replica, image `ghcr.io/purduercac/rcac-docs-prod:latest`, `imagePullPolicy: Always`, `imagePullSecrets: ghcr-secret`, modest CPU/memory limits -- `service.yaml`: `ClusterIP` Service exposing port 80 -- `ingress.yaml`: `ingressClassName: public`, TLS host `docs.rcac.purdue.edu` (secret `rcac-docs-prod-tls`) -- `cronjob.yaml`: `rcac-docs-prod-restart` CronJob running every 5 minutes; uses `bitnami/kubectl` (from `geddes-registry.rcac.purdue.edu/docker-hub-cache/`) with a mounted `kubeconfig-secret` to `kubectl rollout restart` the deployment so new `:latest` images are pulled -### GitHub App Bot -- **Name**: `purduercac-docs-bot` -- **Purpose**: Automated commits that push to `main`/`dev` without triggering recursive workflows -- **Configuration**: `PURDUERCAC_DOCS_BOT_APP_ID` and `PURDUERCAC_DOCS_BOT_PRIVATE_KEY` in repository secrets; commits authored as `purduercac-docs-bot@users.noreply.github.com` -### Read the Docs Integration -- `.readthedocs.yaml` is still present (Ubuntu 24.04, Python 3, `pip install -r requirements.txt`) but is **no longer the primary deployment path**. The Geddes/Kubernetes pipeline above serves `docs.rcac.purdue.edu`. -## Development Workflow -### Local Development Setup -```bash -# Clone repository -git clone git@github.com:PurdueRCAC/RCAC-Docs.git -cd RCAC-Docs +# AGENTS.md -# Create Python environment (conda or venv) -conda create -n rcac-docs python -conda activate rcac-docs +Guidance for agents (Claude Code and others) working in this repository. `CLAUDE.md` is a +symlink to this file — edit `AGENTS.md`, never a separate copy. (`.claude` is likewise a +symlink to `.agents`, so Claude Code discovers the documentation-factory skills and settings +through it.) -# Install dependencies -python -m pip install -r requirements.txt +This document is the operating manual and **constitution**: the architecture, the +load-bearing invariants, and the process rules an agent needs to make correct, safe changes +to this documentation site without rediscovering them each session. **When something here +disagrees with the rendered site or the source tree, the site is ground truth — fix this +file.** The curated `.agents/factory/invariants.md` is a strict subset of this file, kept in +lockstep; if it drifts, this file wins. + +--- + +## Project + +This is the MkDocs Material documentation website for the **Rosen Center for Advanced +Computing (RCAC)** at Purdue University — user guides, references, software/dataset catalogs, +blog articles, and workshop/training material for RCAC's HPC clusters and storage services. + +- **Production URL:** (served from the Geddes Kubernetes cluster). +- **Source repo:** . +- **Design/content reference sites** (same MkDocs-Material lineage): docs.tacc.utexas.edu, + docs.alcf.anl.gov, docs.ncsa.illinois.edu, docs.jetstream-cloud.org, docs.hpc.ucdavis.edu. + +## Environment & working rules + +- **Setup:** create a Python environment and install pinned deps. + ```bash + python -m venv .venv && source .venv/bin/activate # or: conda create -n rcac-docs python && conda activate rcac-docs + python -m pip install -r requirements.txt + ``` + (The local `.venv` may be uv-created and lacks `pip`; `uv pip install -r requirements.txt` + syncs it. The project itself is pip/`requirements.txt`-managed — there is no `pyproject.toml` + and no `uv.lock`.) +- **Always run tooling from the repo root.** The macros module (`main.py`) and the factory + scripts resolve paths relative to the current directory. +- **Commit only when explicitly asked.** When you do: branch off **`main`** (never commit + straight to `main`; never touch `dev` outside an a11y rollout — see Branch & deploy model). + Commit subjects follow `[category] Imperative summary` — categories include `feature`, `fix`, + `refactor`, `docs`, `content`, `catalog`, `ci`, `harness` (the `.agents/` factory); the set is + not closed. +- **No `Co-Authored-By` trailer** on commits — a deliberate clean-log convention. End PR + **bodies** with the generation line instead. (Automated commits are authored as the GitHub + App bot `purduercac-docs-bot`.) +- **Use `Closes #NN`** to auto-close an issue — it works here because `main` is the default branch. + +## Commands -# Start local server -mkdocs serve # Default: http://localhost:8000 -mkdocs serve -a localhost:8080 # Custom port -``` -### Local Production-Image Build (optional) -```bash -docker build -f Dockerfile.geddes-prod -t rcac-docs-prod:local . -docker run --rm -p 8080:80 rcac-docs-prod:local -``` -### Making Changes -#### Content Changes -1. Edit markdown files in `docs/` -2. Test locally with `mkdocs serve` -3. Open a PR into `main` from a short-lived feature or `wip` branch. Do **not** PR from `dev` into `main` — it is blocked by CI, and `dev` is the a11y remediation branch, not a staging branch for content. -4. On merge to `main`, `deploy_to_geddes_prod.yml` builds and pushes the image; the Geddes CronJob rolls the deployment within ~5 minutes. `sync_main_to_dev.yml` separately forwards the merge onto `dev` for the a11y preview. -#### Software Catalog Updates -1. Add/modify module files in `modulefiles/` -2. Push/merge to `main` (or `dev`) -3. `rebuild_on_module_tools_change.yml` runs `make` and bot-commits the regenerated catalog -#### Navigation Changes -1. Edit `nav:` in `mkdocs.yml` -2. Push/merge to `main` (or `dev`) -3. `update_breadcrumbs.yml` regenerates `docs/assets/data/breadcrumbs.json` -### Manual Catalog Generation ```bash -cd tools -make # Runs all steps in correct order - -# Or run individually: -./update_apps_inventory.sh -v -./generate_apps_topics.py -./update_apps_descriptions_from_inventory.sh -./generate_md.sh -./update_apps_catalog.sh +mkdocs serve # local preview at http://localhost:8000 +mkdocs serve -a localhost:8080 # custom port +mkdocs build # build the static site into ./site (gitignored) +mkdocs build --strict # promote link/nav warnings to errors — USE THIS to verify (CI does not) + +cd tools && make # regenerate the software & dataset catalogs (see Automation) +python tools/generate_breadcrumbs.py # regenerate docs/assets/data/breadcrumbs.json after a nav change + +# Optional: build the production image locally +docker build -f Dockerfile.geddes-prod -t rcac-docs-prod:local . && docker run --rm -p 8080:80 rcac-docs-prod:local ``` -## Special Features -### MkDocs Macros (`main.py`) -Reusable content snippets defined as Python functions (via `mkdocs-macros-plugin`): -- `login_snippet(host, cluster)`: SSH login instructions -- `account_snippet(host, cluster)`: Account creation info -- `ssh_keys_snippet(resource)`: SSH key setup guide (macOS/Linux/Windows/PuTTY) -- `ssh_x11_snippet(resource)`: X11 forwarding setup -- `thinlinc_snippet(resource)`: ThinLinc client instructions -### MkDocs Hooks -- `hooks/socialmedia.py`: Social-media share button support (currently commented out in `mkdocs.yml`) -### Theme Customizations -- **Logo/Favicon**: Purdue branding (`docs/assets/purdue.png`, `docs/assets/purdue_favicon.png`) -- **Color Schemes**: Auto / light / dark (Material palette toggles) -- **Fonts**: Roboto (text), Consolas (code) -- **Custom CSS**: `docs/stylesheets/extra.css` -- **Custom JS**: `docs/assets/js/` — `tablefilter.js`, `init-tablefilter.js`, `breadcrumbs.js`, `external-links.js` (plus `a11y.js` on `dev`) -- **Theme overrides**: `overrides/main.html` and `overrides/partials/` -### Analytics & Feedback -- Google Analytics (property: `G-3D80JBWGP0`) -- Page feedback widget (thumbs up/down) with link to a feedback issue template -## Best Practices -### Content -- Use admonitions for important notes/warnings -- Include code blocks with proper syntax highlighting -- Link to related pages within user guides -- Tag blog posts with appropriate categories (Anvil, Gautschi, Software, Slurm, Workflows) -### Version Control -- Land all content/feature/workflow changes on `main` via PR; `sync_main_to_dev.yml` will forward them to the a11y `dev` branch automatically -- Touch `dev` directly **only** for WCAG/a11y remediation work on the override layer; never backport `dev` to `main` outside of an explicit, curated a11y rollout -- Oz-assisted commits include trailer: `Co-Authored-By: Oz ` -- Bot commits are authored as: `purduercac-docs-bot@users.noreply.github.com` -### Testing -- Always test locally with `mkdocs serve` before pushing -- Optionally validate the production image with `docker build -f Dockerfile.geddes-prod` -- Verify breadcrumbs render correctly after `mkdocs.yml` changes -- Verify software catalog after module file changes -## Key Files Reference -- `mkdocs.yml`: Main configuration, navigation, plugins, markdown extensions -- `requirements.txt`: Python dependencies (pinned) -- `main.py`: MkDocs macros -- `hooks/socialmedia.py`: Optional social-share hook -- `tools/Makefile` and `tools/README.md`: Software catalog build automation -- `Dockerfile.geddes-prod`: Production image for Geddes (builder + nginx serve stage) -- `k8s/geddes-prod/*.yaml`: Kubernetes Deployment, Service, Ingress, CronJob -- `.github/workflows/`: CI/CD automation (deploy to Geddes, S3 mirror, catalog, breadcrumbs, branch sync/guards) -- `.readthedocs.yaml`: Read the Docs build config (secondary) + +**Stack:** MkDocs 1.6.1 · Material for MkDocs 9.6.16 (custom_dir `overrides`) · Python 3.11 in +the prod image · pinned in `requirements.txt`. Plugins: `search`, `blog`, `tags`, +`git-revision-date-localized`, `rss`, `macros` (see `main.py`). Markdown extensions: `toc` +(permalink), `tables`, `attr_list`, `md_in_html`, `admonition`, and `pymdownx.*` (`details`, +`blocks.caption`, `emoji`, `snippets`, `highlight`, `inlinehilite`, `superfences`, `tabbed`). + +## Repository map + +| Path | Responsibility | +|------|----------------| +| `docs/` | All published content (the only directory MkDocs builds). | +| `docs/userguides//` | Per-cluster/storage guides (anvil, bell, gautschi, gilbreth, negishi, scholar, geddes, hammer; storage: depot, fortress, boxfolder, reedfolder). Consistent chapter set across clusters. | +| `docs/blog/posts/` | Blog posts (Material blog plugin); authors in `docs/blog/.authors.yml`. | +| `docs/software/`, `docs/datasets/` | **Auto-generated** catalog pages (see Automation) — do not hand-edit. | +| `docs/workshops/`, `docs/lifesciences/` | Training material, organized as series with an `index.md` hub. | +| `docs/snippets/` | Reusable includes (`--8<--`); `exclude_docs` — not pages. | +| `docs/assets/` | Images, JS, CSS, data (`assets/data/breadcrumbs.json` is generated), PDFs, fonts. | +| `mkdocs.yml` | Config, theme, plugins, markdown extensions, and the **manually curated `nav:`**. | +| `main.py` | `mkdocs-macros` module — the `{{ … }}` macros/snippets (SSH, accounts, Slurm, storage, containers, …). | +| `overrides/`, `docs/stylesheets/` | Material theme overrides + custom CSS (much of it dev-only a11y layer — see below). | +| `tools/` | Catalog generators (`Makefile`, `generate_*`/`update_*`) and `generate_breadcrumbs.py`. | +| `modulefiles/` | HPC module files that seed the software-catalog generator. | +| `hooks/` | MkDocs hooks (`socialmedia.py`, currently disabled in `mkdocs.yml`). | +| `Dockerfile.geddes-prod`, `k8s/geddes-prod/` | Production image (builder → nginx) and Kubernetes manifests. | +| `.github/workflows/` | CI/CD (deploy, catalog rebuild, breadcrumbs, branch sync/guard). | +| **`.agents/`** | The spec-driven **documentation factory** (`docs-*` skills + `factory/` methodology, invariants, style guide, EARS, review rubric, templates, and `bin/` FSM scripts). `.claude` symlinks here. | +| **`spec/{slug}/`** | The committed, dated per-job design records (`GOAL/PLAN/TECH/REVIEW.md`) the factory produces and retains on merge. Outside `docs/`, so not published. | + +## Content conventions & archetypes + +Full detail in `.agents/factory/style-guide.md`; `docs/templates.md` is the live extension +reference. In brief: + +- **Archetypes:** cluster user-guide article · catalog entry (mostly generated) · blog post · + tutorial/workshop lesson · reference/hub/landing. Each has a distinct front-matter contract. +- **Front-matter:** user guides carry `tags`/`authors`/`resource`/`search.boost:2`; blog posts + carry `date.created`/`categories` (from the allowed set)/`slug`/`authors` + a `` + excerpt marker; landing/catalog pages use `hide: [footer(, toc)]`. +- **Links relative, assets absolute:** page-to-page markdown links are relative + (`[Overview](overview.md)`); images/PDFs/JS/CSS use absolute site-root paths (`/assets/…`). +- **Reuse over duplication:** cluster-variable content comes from `main.py` macros + (`{{ ssh_keys_snippet(resource) }}`, inline `{{ resource }}`) and `--8<--` snippet includes — + don't hardcode what a macro supplies. Escape literal `{{`/`{%` in prose with `{% raw %}`. +- **Voice:** professional, instructional, second person; imperative for steps; admonitions for + notes/warnings; one `#` H1 per page; cross-link and end subpages with a back-link. + +## Invariants (load-bearing) + +The curated footgun checklist is `.agents/factory/invariants.md` (used by the plan gate and the +review). The headlines: + +1. **Branch & deploy model** — `main` is trunk *and* production; merging auto-deploys. `dev` is + the a11y branch; `dev → main` is CI-blocked. +2. **The dev-only a11y layer is untouchable on `main`** (the sync workflow clobbers it). +3. **Generated-content firewall** — never hand-edit catalogs or `breadcrumbs.json`; regenerate. +4. **Nav is manual** — add every new page to `mkdocs.yml` `nav:` or it is orphaned. +5. **Links relative, assets absolute.** +6. **Front-matter per archetype** (blog `categories` from the allowed set; ``). +7. **Macros/snippets + Jinja escaping.** +8. **Build integrity** — `mkdocs build --strict` introduces no new warnings vs the baseline + (`hammerable: false`). +9. **Content-level WCAG 2.1 accessibility** (`hammerable: false`). +10. **Per-cluster parallelism** — keep the shared chapter set/order. +11. **HPC technical accuracy** — commands, flags, partitions, paths, specs must be correct + (`hammerable: false`). +12. **Commit/PR conventions** (this file's "Environment & working rules"). + +## Branch & deploy model + +- **`main`** — trunk and the **only production branch**. All content/feature/nav/infra work + targets `main` via a PR from a short-lived `feature/`|`fix/`|`refactor/` branch. **A merge to + `main` auto-deploys to production within ~5 minutes — there is no staging gate.** +- **`dev`** — a long-lived, permanently-divergent **WCAG 2.1 accessibility-remediation** branch + carrying an experimental theme/override layer not yet ready for production. Served from its own + Geddes environment for WAVE re-audits. +- **Data flow is main → dev.** `sync_main_to_dev.yml` forward-merges `main` into `dev` on every + push, force-preserving a hardcoded list of dev-only files (`docs/stylesheets/`, + `overrides/partials/toc.html`, the `a11y.js` nav entry, …). Editing those files on `main` risks + being clobbered — do that work on `dev`. +- **`dev → main` PRs are hard-blocked** by `block_dev_to_main.yml`. When the a11y work is + accepted, the endgame is a deliberate, curated PR of the theme layer — never an automatic merge. + +**Deploy chain:** push/merge to `main` → `deploy_to_geddes_prod.yml` builds `Dockerfile.geddes-prod` +(`python:3.11-slim` → `mkdocs build --verbose` → `nginx:alpine`) and pushes +`ghcr.io/purduercac/rcac-docs-prod:latest` → the in-cluster `rcac-docs-prod-restart` CronJob +(every 5 min) rolls the deployment (`imagePullPolicy: Always`). Manifests live in `k8s/geddes-prod/` +(`deployment/service/ingress/cronjob/nginx.conf`). + +## Automation & generated content + +- **Software/dataset catalogs** are generated from `modulefiles/` + `tools/*.json` by + `tools/Makefile` (`make` runs `update_apps_inventory.sh` → `generate_apps_topics.py` → + `update_apps_descriptions_from_inventory.sh` → `generate_md.sh` → `update_apps_catalog.sh`, plus + the NGC/ROCm steps). Output lands in `docs/software/**` and `docs/datasets/dataset_catalog.md` + with a `` marker. `rebuild_on_module_tools_change.yml` + regenerates and bot-commits these when `modulefiles/**` or `tools/**` change. +- **Breadcrumbs** — `python tools/generate_breadcrumbs.py` writes `docs/assets/data/breadcrumbs.json` + from the nav; `update_breadcrumbs.yml` regenerates and bot-commits it when `mkdocs.yml` changes. +- **CI is not `--strict`.** The prod build uses `mkdocs build --verbose`, so broken internal links + and nav orphans deploy silently. The factory's verify gate runs `--strict` to catch them; see + `.agents/factory/strict-baseline.txt` for the known pre-existing warnings. + +## Special features + +- **Macros (`main.py`):** `mkdocs-macros` functions for reusable, cluster-variable content — + SSH/keys/X11/ThinLinc, accounts, Slurm overviews, storage/scratch/quota, transfers + (scp/sftp/globus/htar/hsi), containers (apptainer/biocontainers/rocm), compile, recovery, etc. + Read `main.py` for the current list and signatures before hardcoding anything. +- **Theme:** Purdue branding, light/dark/auto palette, custom CSS in `docs/stylesheets/extra.css`, + custom JS in `docs/assets/js/` (tablefilter, breadcrumbs, external-links, outage-widget). +- **Analytics & feedback:** Google Analytics; a thumbs up/down page-feedback widget. + +## Working on this repo as an agent — the documentation factory + +- **Use the factory for non-trivial work.** A documentation job (`feature`/`fix`/`refactor`) flows + through the `.agents/` spec-driven lifecycle — `/docs-feature` (shape `GOAL.md`) → `/docs-plan` + (research + `PLAN.md`/`TECH.md`) → `/docs-draft` (execute phases) → `/docs-review` (blind, + externally-verified QA) → `/docs-publish` (squash PR to `main`), each on a + `feature/`|`fix/`|`refactor/` branch with artifacts committed under `spec/{slug}/`. + `.agents/factory/methodology.md` is the *why*; `getting-started.html` is the onboarding. + **Ceremony scales to appetite** — a one-sentence fix may skip the lifecycle entirely. +- **Verify by rendering, not asserting.** After a change, run + `mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py` (no new warnings), + confirm the page is in `nav:`, and `mkdocs serve` to eyeball the render — the concurrency and + link/nav behavior are where documentation bugs hide. +- **Put content where it belongs:** new pages under `docs/…` + a `nav:` entry; cluster-variable + content via `main.py` macros (extend the macro, don't fork the prose); generated catalogs via + `tools/`, never by hand. +- **Never guess HPC specifics.** A wrong `#SBATCH` flag, partition, module, or path misdirects real + users. Verify against a gold-standard page or an authoritative source, or emit + `[NEEDS CLARIFICATION]`. +- **This file is the map, but it drifts.** For a deep change, re-verify the specific invariant + against the site/source before relying on it, and update this file when things move. + ## Contact -- **Organization**: Rosen Center for Advanced Computing -- **Email**: rcac-help@purdue.edu -- **GitHub**: https://github.com/PurdueRCAC -- **Discord**: https://discord.gg/RmtKZmaQW9 + +- **Org:** Rosen Center for Advanced Computing · **Email:** rcac-help@purdue.edu +- **GitHub:** · **Discord:** diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file From 1fbbf742757e0decba97d7c7f792548af2c14f27 Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Tue, 14 Jul 2026 15:56:31 -0400 Subject: [PATCH 7/8] [harness] Wire factory discovery via .claude symlink and gitignore .claude -> .agents so Claude Code discovers the skills; un-ignore .claude and ignore only .agents/settings.local.json. --- .claude | 1 + .gitignore | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) create mode 120000 .claude diff --git a/.claude b/.claude new file mode 120000 index 00000000..c0ca4685 --- /dev/null +++ b/.claude @@ -0,0 +1 @@ +.agents \ No newline at end of file diff --git a/.gitignore b/.gitignore index eb621ad3..d37e4447 100644 --- a/.gitignore +++ b/.gitignore @@ -46,7 +46,9 @@ htmlcov/ .env.local # Assistant / tooling scratch -.claude/ +# `.claude` is a tracked symlink -> `.agents` (the documentation factory), so it is NOT +# ignored. Only genuinely personal Claude Code settings are ignored: +.agents/settings.local.json .warp/ # Project-local drafts not for publication From 81bc59fefa691d45dc0646c931a067babf7e6cab Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Tue, 14 Jul 2026 15:56:31 -0400 Subject: [PATCH 8/8] [ci] Skip prod deploy for documentation-factory bookkeeping paths Add spec/**, .agents/**, AGENTS.md, CLAUDE.md to deploy_to_geddes_prod paths-ignore -- they never affect the built site. --- .github/workflows/deploy_to_geddes_prod.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/deploy_to_geddes_prod.yml b/.github/workflows/deploy_to_geddes_prod.yml index 0820cb16..f0010842 100644 --- a/.github/workflows/deploy_to_geddes_prod.yml +++ b/.github/workflows/deploy_to_geddes_prod.yml @@ -8,6 +8,11 @@ on: - 'modulefiles/**' - 'tools/**' - 'wcag_audit/**' + # Documentation-factory bookkeeping never affects the built site (all outside docs/). + - 'spec/**' + - '.agents/**' + - 'AGENTS.md' + - 'CLAUDE.md' workflow_dispatch: permissions: