From 4b859e27aceed8f6b3259086f6c2725324630a2f Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Tue, 14 Jul 2026 23:33:26 -0400 Subject: [PATCH 01/32] [feature] Shape agentic-ai-docs goal --- spec/agentic-ai-docs/GOAL.md | 206 +++++++++++++++++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 spec/agentic-ai-docs/GOAL.md diff --git a/spec/agentic-ai-docs/GOAL.md b/spec/agentic-ai-docs/GOAL.md new file mode 100644 index 00000000..e8b1ac54 --- /dev/null +++ b/spec/agentic-ai-docs/GOAL.md @@ -0,0 +1,206 @@ +# GOAL — Agentic AI on RCAC: guidance, MCP tooling, and shared context + +> **Origin spec.** The *what* and *why* — the locked contract `docs-review` grades against. +> The *how* lives in [`PLAN.md`](PLAN.md) and [`TECH.md`](TECH.md) (written by `docs-plan`). + +- **slug:** agentic-ai-docs +- **kind:** feature +- **appetite:** big · *multi-pillar new section; expect `docs-plan` to split into several phases and + to scope-hammer nice-to-haves against this contract.* + +## Problem + +Researchers on RCAC's clusters are already using agentic coding tools — Claude Code, OpenAI +Codex, Gemini CLI, opencode, Warp — with or without our endorsement, both directly on login +nodes and from their laptops targeting the cluster over SSH. Today the RCAC documentation site +has **zero** public guidance on this: no acceptable-use rules, no setup instructions, no mention +of our MCP servers, and no published account of the shared context we inject into agents. The +result is the failure mode described in our PEARC'26 paper *Hello Computer: HPC in the Agentic +Era* — users arrive at the help desk with confident, plausible, and wrong AI output (a Slurm +script for a scheduler we don't run), and the cluster is exposed to preventable mistakes because +nobody shaped the context in which the interaction happened. + +RCAC's chosen strategy is **proactive engagement, not prohibition**: system-wide `/etc/agents.d` +context files, purpose-built MCP servers, and honest documentation that helps users *verify* +rather than *forbidding* the tools. Two of those three pillars exist in prototype (the MCP +servers); the documentation pillar does not exist at all, and the shared context/settings +artifacts the other pillars are meant to consume have not yet been authored. This feature builds +the documentation pillar **and**, as a shared exercise, constructs the first real version of the +shared context and per-harness settings — starting with the Gautschi cluster. + +## Outcome / vision + +A new top-level **Agentic AI** section on `docs.rcac.purdue.edu`, plus a cross-linked **Using AI +Agents** chapter in the Gautschi user guide. After reading, a researcher can: + +- Understand RCAC's stance and what is expected of them: acceptable use, etiquette, the real + limitations, and best practices (using an agent *for research* vs *for operations*; verifying + output; augmenting rather than outsourcing their own expertise; the safety "don't cross the + streams" cautions). +- **Set up their chosen harness** (any of the five) correctly for Gautschi, in whichever mode + they work — running the agent *on the cluster* (login nodes) or *locally, targeting the + cluster* over MCP + SSH. +- Learn what our **MCP servers** are, why they matter, and how to connect them: the HPC + operations server (`rcac-mcp`, generalizing toward a plugin architecture), the docs-search + server (`rcac-docs-mcp`, hosted at `docs.rcac.purdue.edu/mcp`), and `globus-mcp` for transfers. +- **See exactly what context and permissions RCAC gives their agents** — the actual Gautschi + `/etc/agents.d` context files (`unix.md`, `filesystems.md`, `lmod.md`, `slurm.md`, + `policies.md`) and the per-harness settings files, published verbatim — and know how to send + feedback and corrections. + +The published context/settings files are the **canonical single source of truth**: cluster +config management (Puppet) copies them out to `/etc/agents.d` on the hosts, and the HPC MCP +server reads them over SSH and injects them as agent context. This is early, evolving work; the +docs say so. + +## Acceptance criteria (the contract) + +Reader-facing, observable outcomes. Stable IDs survive squash-merge. + +**Section & stance** + +- **R1** — A top-level **Agentic AI** section SHALL appear in `mkdocs.yml` `nav:` with an + `index.md` hub that states RCAC's stance (proactive engagement, not prohibition; verify, don't + forbid) and links to every subpage; it SHALL resolve without a `--strict` nav/link warning. + +**Acceptable use, etiquette, limitations, best practices** + +- **R2** — WHEN a reader opens the acceptable-use/etiquette page, it SHALL state the rules for + running agents on RCAC systems (no heavy compute on login nodes; jobs go through Slurm with + time limits; no sensitive data in world-readable locations; destructive actions require + confirmation) consistently with RCAC's existing Acceptable Use policy, and link to the Purdue + IT policy of record. +- **R3** — A best-practices/limitations page SHALL distinguish using an agent *for research* from + using one *for operations*, and SHALL cover context engineering, verification of output + ("augmented, not outsourced"; ask *why*, not just *what*), and the caution/blast-radius risks + (destructive commands, allocation exhaustion, credential/secret leakage). + +**Running agents (organized by deployment mode; all five harnesses)** + +- **R4** — The "Running agents" material SHALL be organized primarily by deployment mode — + (a) *on the cluster* (login nodes) and (b) *locally, targeting the cluster* via MCP + SSH — and + each mode SHALL cover Claude Code, Codex, Gemini CLI, opencode, and Warp. +- **R5** — The on-cluster guidance SHALL state the login-node constraints that apply to agents + (no heavy/parallel/long compute on front-ends; submit work via Slurm), consistent with the + Gautschi Running Jobs guidance. +- **R6** — The local (MCP + SSH) guidance SHALL explain the local-first architecture: the agent + runs in the user's own environment and executes remotely through their existing SSH access — no + new credentials and no hosted infrastructure required. + +**MCP servers** + +- **R7** — An MCP page SHALL document RCAC's MCP servers with their current focus and link each to + its public GitHub repository: the HPC operations server (`rcac-mcp`, generalizing toward a + plugin architecture, e.g. `cluster-mcp[slurm,lmod,…]`); the docs-search server + (`rcac-docs-mcp`, hosted at `docs.rcac.purdue.edu/mcp`); and `globus-mcp` (data transfers). It + SHALL explain *why* MCP matters (context that knows our clusters) and present the tooling as + evolving/prototype. +- **R8** — The MCP page SHALL explain that the HPC MCP server reads the host's `/etc/agents.d` + markdown context (a configurable location) over SSH and injects it into the agent's context. +- **R9** — Any install/connection commands or tool names shown for the MCP servers SHALL be + verified against the live public repositories at draft time — never invented. + +**Shared context & settings (build + publish the canonical source of truth)** + +- **R10** — The section SHALL publish the actual Gautschi shared-context files verbatim + (`unix.md`, `filesystems.md`, `lmod.md`, `slurm.md`, `policies.md`); their content SHALL be + HPC-accurate for Gautschi (Rocky Linux 9; Lmod; partitions `cpu`/`ai`/`highmem`; QOS + `normal`/`standby`/`preemptible`; correct filesystem paths and quotas). +- **R11** — The section SHALL publish per-harness settings/permission files for all five harnesses + (Claude Code, Codex, Gemini CLI, opencode, Warp), each in that harness's native settings + format, wiring in the shared context and encoding a starting-point set of enforced permissions + (e.g. deny destructive operations and login-node heavy compute). +- **R12** — A page SHALL explain how `/etc/agents.d` context and the harness settings fit together + (context injected; permissions enforced), state that these files are the canonical source of + truth copied into cluster config management, and tell readers how to contribute feedback and + corrections. + +**Gautschi integration** + +- **R13** — A **Using AI Agents** chapter SHALL be added to the Gautschi user guide (in the + Gautschi `nav:` sub-tree) and be cross-linked with the top-level Agentic AI section, giving + Gautschi-specific setup pointers. + +**Build, accessibility, accuracy (hammerable: false — never trimmed to fit appetite)** + +- **R14** — Every new page SHALL be included in `mkdocs.yml` `nav:`, and `mkdocs build --strict` + SHALL introduce no new warnings versus `strict-baseline.txt`. +- **R15** — Page-to-page links SHALL be relative and static assets absolute; every content image + SHALL carry meaningful `alt` text; headings SHALL descend without skipping; tables SHALL use + header rows (content-level WCAG 2.1). +- **R16** — All HPC specifics on the new pages (partitions, QOS, modules, filesystem paths, SSH + hostnames, commands) SHALL be correct for Gautschi, verified against gold-standard Gautschi + pages or authoritative sources; no invented flags or paths. + +## Non-goals (no-gos) + +- **Other clusters.** Gautschi only. Anvil, Bell, Gilbreth, Negishi, Scholar, Geddes, Hammer, and + the storage services are an explicit follow-up feature (per-cluster parallelism handled then). +- **Building the MCP servers.** We document and link `rcac-mcp` / `globus-mcp` / `rcac-docs-mcp`; + their source code lives in their own repos and is not shipped or vendored here. +- **The config-management (Puppet) deployment pipeline.** We author the canonical `/etc/agents.d` + and settings files; wiring them into Puppet and onto the hosts is out of scope for this feature. +- **A security audit / formal threat model** of MCP or agentic tooling — practical caution only, + mirroring the paper's non-goal (confinement discussed, not proven). +- **Model or vendor benchmarking / endorsement.** Guidance is vendor-neutral across the five + harnesses; no performance claims. +- **Theme / CSS / JS or accessibility-*theme*-layer changes.** Content-level a11y only; the theme + a11y layer is a `dev`-branch concern (invariants §2). +- **Generated content.** No hand-edits to catalogs, the software/dataset generators, or + breadcrumbs (breadcrumbs regenerate from the nav change via the normal workflow). +- **A companion announcement blog post** — deferred to a possible follow-up so it doesn't expand + this already-large feature. + +## Clarifications + +Resolved with the human during shaping on 2026-07-14. + +- **Q:** Where should the material live, and how prominent? — **A:** Hybrid: a new **top-level + "Agentic AI" section** for the general/conceptual/MCP/context material, **plus** a cross-linked + **"Using AI Agents"** chapter in the Gautschi user guide. +- **Q:** How should the built context files and harness settings be surfaced? — **A:** Publish + them **verbatim** in this repo as the **canonical single source of truth**. Puppet copies them + out to `/etc/agents.d` on the cluster hosts, and the HPC MCP server injects them over SSH; users + do **not** hand-install them. (A downloadable copy is a nice-to-have, not required.) +- **Q:** Which harnesses get authored settings/permission files? — **A:** **All five** (Claude + Code, Codex, Gemini CLI, opencode, Warp). All five also get usage guidance. +- **Q:** Primary organizing axis for the run-agents guidance? — **A:** **By deployment mode** + (on-cluster login nodes vs. local via MCP + SSH), with harnesses covered within each. +- **Q (architecture, from the human):** MCP servers are being re-architected — record the current + intent. — **A:** `rcac-mcp` is refocusing on **HPC-only operations** with a **generalized + plugin architecture** (e.g. `cluster-mcp[slurm,lmod,…]`); `rcac-docs-mcp` is a **standalone** + service hosted at **`docs.rcac.purdue.edu/mcp`** (full-text search/load over this site); + `globus-mcp` handles data transfers. The HPC server, over SSH, reads `/etc/agents.d` (a + configurable path) on the host and injects those markdown files as MCP context. Docs present all + of this as **prototype/evolving** ("likely to change over time"). +- **Q:** What enforced-permission policy do the settings encode? — **A:** This is a deliberate + **shared construction exercise**; author a reasonable **starting-point** deny/allow policy + derived from the paper (no login-node heavy compute; require Slurm time limits; no secrets in + world-readable paths; confirm destructive operations) and iterate later. Not a blocker. +- **Voice/stance (adopted from the paper, recorded):** proactive engagement not prohibition; + "mostly harmless"; verify don't forbid; augmented not outsourced; "don't cross the streams" + caution. Professional, instructional, second person (house style). + +*No unresolved `[NEEDS CLARIFICATION]` markers remain; the GOAL is ready for `docs-plan`.* + +## Related materials + +- **PEARC'26 paper** (the motivating source, three pillars, and Discussion framing to mirror for + voice): — *Hello Computer: HPC in the + Agentic Era* (Lentner & Ashish, 2026). +- **MCP repositories:** · + · + (hosted at `docs.rcac.purdue.edu/mcp`). +- **Gold-standard pages to mirror / reuse:** + - Hub archetype: `docs/lifesciences/index.md` (tagged hub + grid cards). + - Policy archetype: `docs/userguides/anvil/policies.md` + `docs/snippets/resourceuse.md` + (`resource_use` macro) — Gautschi currently has **no** `policies.md`. + - External-tool → cluster prior art: `docs/lifesciences/guides/vscode.md` (Remote-SSH / + ProxyJump into a compute node). + - Gautschi facts: `docs/userguides/gautschi/{overview,software,storage}.md`, + `run_jobs/{index,queues,directives}.md`. + - Reusable macros (`main.py`): `ssh_keys_snippet`, `module_system`, `slurm_general_jobs`, + `storage_quota`, `scratch_space`, `resource_use`. +- **Reference sites** (same MkDocs-Material lineage): docs.tacc.utexas.edu, docs.alcf.anl.gov, + docs.ncsa.illinois.edu. +- Issue: *(none yet — create and back-link if desired).* From fbf6fe7069d2b9d53d4534bee7f477796a5de878 Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Wed, 15 Jul 2026 08:16:23 -0400 Subject: [PATCH 02/32] [feature] Plan agentic-ai-docs: design + phased roadmap --- spec/agentic-ai-docs/PLAN.md | 267 +++++++++++ spec/agentic-ai-docs/TECH.md | 334 +++++++++++++ spec/agentic-ai-docs/research/00-digest.md | 174 +++++++ .../research/01-site-architecture.md | 438 ++++++++++++++++++ .../research/02-macros-and-jinja.md | 321 +++++++++++++ .../research/03-gautschi-facts.md | 323 +++++++++++++ .../research/04-mcp-servers.md | 354 ++++++++++++++ .../research/05-harness-configs.md | 415 +++++++++++++++++ .../research/06-context-and-conventions.md | 309 ++++++++++++ 9 files changed, 2935 insertions(+) create mode 100644 spec/agentic-ai-docs/PLAN.md create mode 100644 spec/agentic-ai-docs/TECH.md create mode 100644 spec/agentic-ai-docs/research/00-digest.md create mode 100644 spec/agentic-ai-docs/research/01-site-architecture.md create mode 100644 spec/agentic-ai-docs/research/02-macros-and-jinja.md create mode 100644 spec/agentic-ai-docs/research/03-gautschi-facts.md create mode 100644 spec/agentic-ai-docs/research/04-mcp-servers.md create mode 100644 spec/agentic-ai-docs/research/05-harness-configs.md create mode 100644 spec/agentic-ai-docs/research/06-context-and-conventions.md diff --git a/spec/agentic-ai-docs/PLAN.md b/spec/agentic-ai-docs/PLAN.md new file mode 100644 index 00000000..91d2e431 --- /dev/null +++ b/spec/agentic-ai-docs/PLAN.md @@ -0,0 +1,267 @@ +# PLAN — Agentic AI on RCAC: guidance, MCP tooling, and shared context + +> **Status:** Draft for review · **Last updated:** 2026-07-15 +> **Authoritative design.** The *how*. Vision/contract is [`GOAL.md`](GOAL.md); the phased +> executable roadmap is [`TECH.md`](TECH.md). Backing detail is in [`research/`](research/). +> Every design element traces to a GOAL R-ID. + +## 1. Summary + +Build a net-new top-level **Agentic AI** section (hub + guidance + MCP + a mode-organized +"Running Agents" area + a "Shared Context & Settings" subdir) plus a cross-linked **Using AI +Agents** chapter in the Gautschi user guide. As a shared exercise, we also *construct* the first +real Gautschi `/etc/agents.d` context files and per-harness settings, storing them as byte-exact +canonical files under `docs/snippets/agentic-ai/` and publishing them verbatim via `--8<--` fenced +includes (proven Jinja-safe). Ten publishable phases, scaffold → content → integration; HPC +accuracy, accessibility, and build integrity are `hammerable: false`. + +## 2. Design + +### Archetypes + +Three archetypes are in play (see [`style-guide.md`](../../.agents/factory/style-guide.md)): + +- **Reference / hub / landing** — `docs/agentic-ai/index.md` and the two subdir `index.md`s. + Front-matter: `title` + `tags: [Agentic AI]` (mirror `docs/lifesciences/index.md`); grid-cards + body. (Hubs may add `hide: [toc]` if they end up card-only.) +- **Cluster user-guide article** — the Gautschi `using_ai_agents.md` chapter. Front-matter: + `tags: [Gautschi]`, `authors: […]`, and `{% set resource = "gautschi" %}` where it reuses + cluster macros. +- **Policy page** (a lighter variant, mirrored from `docs/userguides/anvil/policies.md`) — the + acceptable-use page: `tags` + `authors`, one H1, hand-written sections + `{{ resource_use(…) }}`. + +A **novel publishing pattern** (not a new archetype, but new to this repo) carries the verbatim +context/settings artifacts — see *Reuse* below. + +### Page structure (files to create) + +**New content pages** under `docs/agentic-ai/`: + +| File | H1 / purpose | R-IDs | +|------|--------------|-------| +| `index.md` | *Agentic AI at RCAC* — stance (proactive engagement, not prohibition; verify, don't forbid), grid-card nav to subpages | R1 | +| `acceptable_use.md` | *Acceptable Use & Etiquette* — rules for agents on RCAC systems; reuse `{{ resource_use(resource) }}`; link Purdue IT AUP | R2 | +| `best_practices.md` | *Best Practices & Limitations* — agent *for research* vs *for operations*; context engineering; verification ("augmented, not outsourced"); caution/blast-radius | R3 | +| `mcp_servers.md` | *RCAC MCP Servers* — why MCP; the three servers with verified connect commands; `/etc/agents.d` injection | R7, R8, R9, R16 | +| `running_agents/index.md` | *Running Agents* — the two deployment modes explained; card links | R4 | +| `running_agents/on_cluster.md` | *On the Cluster (Login Nodes)* — five harnesses on login nodes; login-node constraints; Warp = local-only note | R4, R5, R16 | +| `running_agents/local.md` | *Local, Targeting the Cluster (MCP + SSH)* — local-first architecture; connect each harness to the MCP servers | R4, R6 | +| `shared_context/index.md` | *Shared Context & Settings* — how `/etc/agents.d` + settings fit; canonical source-of-truth → Puppet + MCP injection; **how to contribute feedback** | R12 | +| `shared_context/context_files.md` | *Context Files (`/etc/agents.d`)* — the five Gautschi context files, published verbatim | R10, R16 | +| `shared_context/settings.md` | *Harness Settings & Permissions* — the five per-harness settings, published verbatim | R11 | + +**New Gautschi chapter:** `docs/userguides/gautschi/using_ai_agents.md` (R13, R16). + +**New canonical artifact files** (byte-exact source of truth; under `docs/snippets/agentic-ai/`, +excluded from nav by `exclude_docs: snippets/`, `--8<--`-included — never nav pages): + +``` +docs/snippets/agentic-ai/ + agents.d/unix.md agents.d/filesystems.md agents.d/lmod.md + agents.d/slurm.md agents.d/policies.md + agents.d/AGENTS.md # concatenated canonical context (the assembled file harnesses read) + claude/settings.json codex/config.toml gemini/settings.json + opencode/opencode.json warp/AGENTS.md # + warp profile-denylist note in prose +``` + +### Navigation placement (exact) + +Insert a **top-level section after Life Sciences** (`mkdocs.yml:361`, before the commented Tags line): + +```yaml + - Agentic AI: + - agentic-ai/index.md + - Acceptable Use & Etiquette: agentic-ai/acceptable_use.md + - Best Practices & Limitations: agentic-ai/best_practices.md + - MCP Servers: agentic-ai/mcp_servers.md + - Running Agents: + - agentic-ai/running_agents/index.md + - On the Cluster (Login Nodes): agentic-ai/running_agents/on_cluster.md + - Local (MCP + SSH): agentic-ai/running_agents/local.md + - Shared Context & Settings: + - agentic-ai/shared_context/index.md + - Context Files (/etc/agents.d): agentic-ai/shared_context/context_files.md + - Harness Settings & Permissions: agentic-ai/shared_context/settings.md +``` + +Insert the Gautschi chapter **between Compiling Source Code (`mkdocs.yml:144`) and FAQs (`:145`)**: + +```yaml + - Using AI Agents: userguides/gautschi/using_ai_agents.md +``` + +**Nav is added incrementally, in lockstep with page creation** (a nav line pointing at a missing +file, or a page absent from nav, both fail `--strict`). Reading/nav order == phase order. Breadcrumbs +regenerate at the integration phase (`python tools/generate_breadcrumbs.py`). + +### Reuse (macros, snippets, gold-standard pages) + +- **Verbatim-publishing pattern** (research 02, proven): each artifact is a real file under + `docs/snippets/agentic-ai/…`, embedded in a fenced block: + ````markdown + ```json title="~/.claude/settings.json" + --8<-- "docs/snippets/agentic-ai/claude/settings.json" + ``` + ```` + `--8<--` content is spliced **after** the Jinja pass, so literal `{{`/`{%` never breaks the build. + Single source of truth, syntax-highlighted. Fence languages: `json` (Claude/Gemini/opencode), + `toml` (Codex), `markdown` (agents.d/*, Warp AGENTS.md). +- **Macros for background prose** (call, don't hardcode): `{{ resource_use(resource) }}` (AUP block, + acceptable-use page), `{{ module_system(resource) }}`, `{{ slurm_general_overview(resource) }}`, + `{{ ssh_keys_snippet(resource) }}`, `{{ storage_quota(resource) }}`, `{{ scratch_purge(resource) }}` + — chiefly on the Gautschi chapter and where HPC background is summarized. **Do not** source the + *context files* from these macros (they emit generic RCAC text; the context files must be authored + from verified Gautschi facts — research 03). +- **Gold-standard pages to mirror:** `docs/lifesciences/index.md` (hub + grid cards); + `docs/userguides/anvil/policies.md` (policy page); `docs/lifesciences/guides/vscode.md` + (external-tool → login → compute prior art, for the local mode). + +### Assets + +None required for a first cut (text + code blocks + admonitions + grid cards). If an architecture +diagram is added later for the local-first MCP+SSH flow, it goes under +`docs/assets/images/agentic-ai/` with an absolute path and descriptive alt text (deferred; not a +GOAL requirement). + +### Cross-links & tags + +- Tag new pages `Agentic AI`; the Gautschi chapter `Gautschi`. +- Hub grid-cards → each subpage; each subpage ends with a back-link to its hub. +- Cross-links: `running_agents/local.md` ↔ `mcp_servers.md`; `mcp_servers.md` ↔ + `shared_context/context_files.md` (the `/etc/agents.d` injection); `shared_context/settings.md` ↔ + `running_agents/*` (where to install each config); Gautschi chapter ↔ the top-level section. + **Sibling cross-links are added in the integration phase** (P10) so every intermediate build is + `--strict`-clean; each content phase only back-links to an already-existing hub. + +### Accessibility plan (R15) + +One `#` H1 per page; headings descend without skipping; grid-card links use descriptive text (not +"click here"); every table has a header row; no meaning by color alone; any future image gets +meaningful alt text. Code blocks carry a `title=` label. This is content-level a11y (owned on +`main`); no theme/CSS/`overrides` changes (invariants §2). + +### Requirement → design map + +| R-ID | Design element(s) that satisfy it | +|------|-----------------------------------| +| R1 | `docs/agentic-ai/index.md` hub + top-level `Agentic AI` nav section | +| R2 | `agentic-ai/acceptable_use.md` (policy archetype + `{{ resource_use }}` + Purdue AUP link) | +| R3 | `agentic-ai/best_practices.md` (research-vs-ops, context engineering, verification, caution) | +| R4 | `running_agents/{index,on_cluster,local}.md` organized by deployment mode, five harnesses each | +| R5 | `running_agents/on_cluster.md` login-node constraints (mirrors Gautschi Running Jobs) | +| R6 | `running_agents/local.md` local-first architecture (no new credentials/hosted infra) | +| R7 | `mcp_servers.md` — three servers, focus + repo links, "why MCP" | +| R8 | `mcp_servers.md` — HPC server reads `/etc/agents.d` over SSH, injects as context | +| R9 | `mcp_servers.md` — install/connect commands verified against the live repos (research 04) | +| R10 | `docs/snippets/agentic-ai/agents.d/*.md` (Gautschi-accurate) + `shared_context/context_files.md` | +| R11 | `docs/snippets/agentic-ai/{claude,codex,gemini,opencode,warp}/…` + `shared_context/settings.md` | +| R12 | `shared_context/index.md` — how it fits, source-of-truth → Puppet + MCP, feedback channels | +| R13 | `docs/userguides/gautschi/using_ai_agents.md` + Gautschi nav entry + cross-links | +| R14 | Every phase adds its nav line; every `verify:` runs `mkdocs build --strict` | +| R15 | a11y plan above; final sweep in P10 | +| R16 | context files/MCP commands/Gautschi facts authored from research 03/04; verified P4/P7/P9/P10 | + +## 3. Invariant gate (constitution check) + +Checked against [`invariants.md`](../../.agents/factory/invariants.md) before research and again +after this design. Touched sections and how the design honors each: + +- **§1 branch/deploy** — all work on `feature/agentic-ai-docs`; squash PR to `main` at publish. No + direct `main`/`dev` edits. +- **§2 dev-only a11y layer** — **untouched.** No edits to `docs/stylesheets/`, `overrides/`, + `a11y.js`, or any force-preserved path. Content-level a11y only. +- **§3 generated-content firewall** — no hand-edits to catalogs or `breadcrumbs.json`; breadcrumbs + are **regenerated** via `tools/generate_breadcrumbs.py` after the nav change. `docs/snippets/ + agentic-ai/*` are hand-authored source, not generated outputs. +- **§4 nav-is-manual** — every new page gets a `mkdocs.yml` `nav:` entry in the same phase; snippet + files are correctly *not* in nav (excluded, `--8<--`-only). +- **§5 links relative / assets absolute** — page-to-page links relative; any asset absolute. +- **§6 front-matter per archetype** — hub (`title`+`tags`), policy page (`tags`+`authors`), + userguide chapter (`tags`+`authors`+`{% set resource %}`). +- **§7 macros / Jinja escaping** — verbatim config is `--8<--`-included (post-Jinja, immune); + background prose reuses macros. No literal `{{`/`{%` pasted into page source unescaped. +- **§8 build integrity** — each phase gates on `mkdocs build --strict | strict_check.py` (zero new + warnings vs the 7-line baseline). +- **§9 content WCAG** — a11y plan above; finalized P10. +- **§10 per-cluster parallelism** — a Gautschi-only "Using AI Agents" chapter creates deliberate + per-cluster asymmetry; **explicitly scoped** as a pilot in GOAL non-goals (other clusters follow). + See deviation table. +- **§11 HPC accuracy** — context files/MCP/Gautschi facts authored strictly from verified research; + the two known Gautschi doc-errors (`--partition=a10`, `-A standby`) are excluded by design; + volatile numbers (quotas) are deferred to `myquota`/`slist` rather than hardcoded. +- **§12 commit/PR** — `[feature] …` subjects, no `Co-Authored-By`; squash PR; `Closes #NN` if an + issue is filed. + +### Deviation justifications + +| Deviation | Why needed | Simpler alternative rejected because | +|-----------|-----------|--------------------------------------| +| Gautschi-only `Using AI Agents` chapter (per-cluster asymmetry, §10) | GOAL scopes this feature to Gautschi as the pilot; the cross-cluster home is the top-level section | Adding the chapter to all 8 clusters now blows the appetite and would duplicate not-yet-verified per-cluster facts; scoped as a follow-up in GOAL non-goals | +| New verbatim-config publishing pattern (`--8<--` fenced includes from `docs/snippets/agentic-ai/`) | R10/R11 require publishing real, byte-exact config as the single source of truth | Hand-pasting config into pages needs `{% raw %}` and duplicates content (drift); this pattern is proven `--strict`-safe (research 02) and stays within §7 (snippets are a sanctioned mechanism) | + +## 4. Rabbit holes (resolved) + +- **Will publishing verbatim config with literal `{{`/`{%` break the macros build?** → No. + `--8<--` splices files *after* the Jinja pass; proven with an isolated `mkdocs build --strict` + (exit 0) ([`research/02-macros-and-jinja.md`](research/02-macros-and-jinja.md)). +- **What are the *real* Gautschi partitions/QOS/paths/commands?** → Extracted with citations, plus + an `[UNCONFIRMED]` list and two doc-errors to avoid + ([`research/03-gautschi-facts.md`](research/03-gautschi-facts.md)). +- **Do the MCP servers exist, and what are the real connect commands/tools?** → Yes; all three + public, `uvx`-installed, verified configs + tool lists + ([`research/04-mcp-servers.md`](research/04-mcp-servers.md)). +- **What is each harness's settings/permissions/MCP/context-file contract?** → Full per-harness + table, incl. system/managed enforcement points and Warp's GUI-only caveat + ([`research/05-harness-configs.md`](research/05-harness-configs.md)). +- **What should the `/etc/agents.d` files contain, and how do they reach each harness?** → Per-file + outlines + the concatenate-to-`AGENTS.md` mapping + ([`research/06-context-and-conventions.md`](research/06-context-and-conventions.md)). +- **Where exactly does this live in the site?** → Top-level after Life Sciences + Gautschi chapter; + exact nav lines ([`research/01-site-architecture.md`](research/01-site-architecture.md)). + +## 5. Risks & open questions + +- **⚠ `cluster-mcp` / HPC-only refocus is [UNVERIFIED] publicly** (research 04): the public + `rcac-mcp` is a prototype that still bundles docs tools; the plugin architecture + (`cluster-mcp[slurm,lmod,…]`) has no public trace. **Mitigation:** document `rcac-mcp` as the + current prototype with its real tools; present the plugin/refocus direction as *planned/evolving*, + clearly labeled. **Human check before publish** that this framing is acceptable. +- **Gautschi `[UNCONFIRMED]` facts** — exact home/scratch quotas (illustrative only), Depot mount + path, OpenMPI version, default shell, availability of `sinfo`/`sacct`/`jobinfo`, a Purdue AUP + direct URL. **Mitigation:** context files tell the agent to run `myquota`/`findscratch`/`slist` + instead of hardcoding; state only verified facts; use the AUP block from `resource_use` / link + Purdue IT Policy V.4.1 (from `resourceuse.md`). +- **Warp is a local desktop GUI, not a headless CLI** — cannot be installed on a login node + (research 05). **Mitigation:** on-cluster coverage for Warp = "run on your workstation, SSH in"; + no server-side Warp settings file — represent Warp's "settings" as `AGENTS.md` + a documented + Agent-Profile denylist, and note the Run-until-completion denylist bypass. +- **Enforced-permissions policy is a v0 starting point** (GOAL clarification) — label it as such and + invite feedback. OS sandboxes (bwrap/Docker) are often disabled on shared nodes → lean on the + permission/approval layer; the real config-management enforcement points are Claude + `/etc/claude-code/managed-settings.json` and Gemini `/etc/gemini-cli/settings.json`. +- **`check_paths:false`** — a mistyped `--8<--` path fails **silently** (empty block, no `--strict` + error). **Mitigation:** each context/settings phase greps the built `site/` for a sentinel token. +- **Feedback channel** for context corrections — default to the RCAC-Docs GitHub issues + + rcac-help@purdue.edu + Discord (from AGENTS.md contact); confirm at draft. + +## 6. Verification strategy + +Seeds each phase's `verify:` in `TECH.md`: + +- **Build integrity:** `.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py` + (no new warnings vs `strict-baseline.txt`) — every phase. +- **Nav:** `grep` the phase's page path in `mkdocs.yml` (no "not in nav"/"missing file" warning). +- **Verbatim includes (P4/P7/P8):** `grep -rq site/agentic-ai/…` to defeat the + `check_paths:false` silent-empty footgun (e.g. `RCAC_SCRATCH`, `mcpServers`). +- **Render:** `mkdocs serve` and eyeball layout (grid cards, tabs, admonitions, macro expansion, + code blocks non-empty) — in the phase steps where layout matters. +- **Front-matter:** matches the archetype (hub `title`+`tags`; policy/userguide `tags`+`authors`). +- **Accessibility:** images (if any) have alt text; headings descend; tables have header rows; + descriptive link text — final sweep P10. +- **HPC accuracy:** every factual claim checked against a gold-standard Gautschi page or + `research/03`; MCP commands against `research/04`; **no `--partition=a10`, no `-A standby`**; + volatile numbers deferred to runtime commands. + +--- + +*Backing research: [`research/00-digest.md`](research/00-digest.md).* diff --git a/spec/agentic-ai-docs/TECH.md b/spec/agentic-ai-docs/TECH.md new file mode 100644 index 00000000..9537d9c5 --- /dev/null +++ b/spec/agentic-ai-docs/TECH.md @@ -0,0 +1,334 @@ +--- +slug: agentic-ai-docs +title: "Agentic AI on RCAC: guidance, MCP tooling, and shared context" +kind: feature +appetite: big +status: in_progress +branch: feature/agentic-ai-docs +base: main +current_phase: P1 +last_updated: "2026-07-15" +phases: + - id: P1 + name: "Scaffold: section hub + top-level nav anchor" + status: pending + satisfies: [R1] + depends_on: [] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'agentic-ai/index.md' mkdocs.yml" + - id: P2 + name: "Acceptable Use & Etiquette page" + status: pending + satisfies: [R2] + depends_on: [P1] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'acceptable_use.md' mkdocs.yml" + - id: P3 + name: "Best Practices & Limitations page" + status: pending + satisfies: [R3] + depends_on: [P1] + parallel: false + hammerable: true + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'best_practices.md' mkdocs.yml" + - id: P4 + name: "MCP Servers page (rcac-mcp / globus-mcp / rcac-docs-mcp)" + status: pending + satisfies: [R7, R8, R9, R16] + depends_on: [P1] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'mcp_servers.md' mkdocs.yml" + - id: P5 + name: "Running Agents: overview + on-cluster (login nodes)" + status: pending + satisfies: [R4, R5] + depends_on: [P1] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'running_agents/on_cluster.md' mkdocs.yml" + - id: P6 + name: "Running Agents: local, targeting the cluster (MCP + SSH)" + status: pending + satisfies: [R4, R6] + depends_on: [P1, P5] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'running_agents/local.md' mkdocs.yml" + - id: P7 + name: "Shared context: hub + build/publish the 5 /etc/agents.d files" + status: pending + satisfies: [R10, R12, R16] + depends_on: [P1] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'shared_context/context_files.md' mkdocs.yml && grep -rq RCAC_SCRATCH site/agentic-ai/shared_context/" + - id: P8 + name: "Build/publish per-harness settings & permissions (all five)" + status: pending + satisfies: [R11, R12] + depends_on: [P1, P7] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'shared_context/settings.md' mkdocs.yml && grep -rq mcpServers site/agentic-ai/shared_context/" + - id: P9 + name: "Gautschi 'Using AI Agents' chapter" + status: pending + satisfies: [R13, R16] + depends_on: [P1, P4, P5, P6, P7] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'using_ai_agents.md' mkdocs.yml" + - id: P10 + name: "Integration: hub cards, cross-links, breadcrumbs, a11y + accuracy sweep" + status: pending + satisfies: [R14, R15, R16] + depends_on: [P1, P2, P3, P4, P5, P6, P7, P8, P9] + parallel: false + hammerable: false + hill: uphill + verify: ".venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && python tools/generate_breadcrumbs.py && grep -q 'Agentic AI' mkdocs.yml" +review: + last_reviewed_commit: "" + verdict: none + blocked_reason: "" +--- + +# TECH.md — Agentic AI on RCAC + +The **context engine and finite-state machine** for authoring this job. The YAML frontmatter above +is the resume ground-truth (read it with +`python3 .agents/factory/bin/next_phase.py spec/agentic-ai-docs/TECH.md`); the per-phase checklists +below are the work. `docs-draft` executes the next actionable phase, runs its `verify:`, updates +state via `set_phase.py`, and makes one atomic content+state commit. Run from the repo root with the +project env active. + +- **Vision / requirements (locked):** [`GOAL.md`](GOAL.md) — R-IDs are the contract. +- **Authoritative design:** [`PLAN.md`](PLAN.md). +- **Backing research:** [`research/00-digest.md`](research/00-digest.md) + the six briefs. + +## Conventions (apply to every phase) + +- Voice/archetype/load-bearing rules from [`../../AGENTS.md`](../../AGENTS.md), + [`style-guide.md`](../../.agents/factory/style-guide.md), and + [`invariants.md`](../../.agents/factory/invariants.md). +- One phase per `docs-draft` invocation; one atomic commit with **both** content and the `TECH.md` + state change. Subjects: `[feature] Draft agentic-ai-docs P: …`. **No `Co-Authored-By`.** +- **Adding a page updates `mkdocs.yml` `nav:` in the same commit** (else orphaned / missing-file + `--strict` failure). Add nav lines in reading order (see PLAN §2). Regenerate breadcrumbs at P10 + (`python tools/generate_breadcrumbs.py`). +- **Verbatim artifacts** (`docs/agentic-ai` pages that show config) use the proven pattern: a real + file under `docs/snippets/agentic-ai/…` embedded in a fenced block via `--8<--` (post-Jinja, so + literal `{{`/`{%` is safe). `check_paths:false` fails silently → each such phase greps the built + `site/` for a sentinel and eyeballs `mkdocs serve`. +- **HPC accuracy (hammerable:false):** author from [`research/03-gautschi-facts.md`](research/03-gautschi-facts.md) + and [`research/04-mcp-servers.md`](research/04-mcp-servers.md). **Never** emit `--partition=a10` or + `#SBATCH -A standby`. Defer volatile numbers (quotas) to `myquota`/`slist`; don't invent + `[UNCONFIRMED]` specifics. +- Never hand-edit generated files (catalogs, breadcrumbs) — regenerate via `tools/`. +- Each content page ends with a back-link to its hub; **sibling cross-links are deferred to P10** so + intermediate builds stay `--strict`-clean. + +--- + +## Phase P1 — Scaffold: section hub + nav anchor +**Satisfies:** R1 · **Depends on:** — · **hammerable:** false +**Goal:** the top-level `Agentic AI` section exists with a hub `index.md` stating RCAC's stance, is +wired into nav, and builds clean. + +- [ ] Create `docs/agentic-ai/index.md` (hub archetype, mirror `docs/lifesciences/index.md`): + `title: Agentic AI`, `tags: [Agentic AI]`; one H1; a stance intro (proactive engagement, not + prohibition; "mostly harmless"; verify, don't forbid — from GOAL/paper). Grid-card links are + **deferred to P10** (children don't exist yet) — for now a short "in this section" prose list + with **no dead links**, or cards pointing only to pages that exist. +- [ ] Insert the top-level `Agentic AI` nav block after Life Sciences (`mkdocs.yml:361`), initially + containing only `- agentic-ai/index.md`. +- **Verify:** `.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q 'agentic-ai/index.md' mkdocs.yml` +- **Touches:** `docs/agentic-ai/index.md`, `mkdocs.yml`. + +## Phase P2 — Acceptable Use & Etiquette +**Satisfies:** R2 · **Depends on:** P1 · **hammerable:** false +**Goal:** a page stating the rules for running agents on RCAC systems, consistent with the existing +AUP, with a link to the Purdue IT policy of record. + +- [ ] Create `docs/agentic-ai/acceptable_use.md` (policy archetype, mirror + `docs/userguides/anvil/policies.md`): `tags`+`authors`; one H1; hand-written sections — no + heavy compute on login nodes (quote/echo the Gautschi Running-Jobs rule), jobs via Slurm with + `--time`, no sensitive data in world-readable paths, no secrets/credentials in context, + destructive actions require confirmation, agent output is the user's responsibility. +- [ ] Reuse the canonical AUP block: `{% set resource = "gautschi" %}` then + `{{ resource_use(resource) }}` (or link Purdue IT Acceptable Use Policy V.4.1 directly if a + single-cluster `resource` reads oddly for cross-cluster policy — see research 02). +- [ ] Add nav line under the section; back-link to the hub. +- **Verify:** `… strict_check.py && grep -q 'acceptable_use.md' mkdocs.yml` +- **Touches:** `docs/agentic-ai/acceptable_use.md`, `mkdocs.yml`. + +## Phase P3 — Best Practices & Limitations +**Satisfies:** R3 · **Depends on:** P1 · **hammerable:** true +**Goal:** a page distinguishing agents *for research* vs *for operations*, covering context +engineering, verification, and caution/blast-radius. + +- [ ] Create `docs/agentic-ai/best_practices.md`: research-vs-operations distinction; context + engineering ("Tea, Earl Grey, hot" — specificity); verification ("augmented, not outsourced"; + ask *why* not just *what*; the expertise paradox); caution/blast-radius (destructive commands, + allocation exhaustion, credential/secret leakage; existing HPC confinement contains blast + radius). Frame from the paper's Discussion; professional/instructional voice. +- [ ] Add nav line; back-link to hub. +- **Verify:** `… strict_check.py && grep -q 'best_practices.md' mkdocs.yml` +- **Touches:** `docs/agentic-ai/best_practices.md`, `mkdocs.yml`. + +## Phase P4 — MCP Servers +**Satisfies:** R7, R8, R9, R16 · **Depends on:** P1 · **hammerable:** false +**Goal:** document the three RCAC MCP servers accurately (why MCP; current focus; verified connect +commands; the `/etc/agents.d` injection), presenting the tooling as prototype/evolving. + +- [ ] Create `docs/agentic-ai/mcp_servers.md`: "why MCP" (context that knows our clusters); a + per-server section for **rcac-mcp** (HPC ops, prototype, stdio via `uvx`, runs over the user's + existing SSH, tool list, `rcac://context`), **globus-mcp** (transfers, beta, user's Globus + OAuth), **rcac-docs-mcp** (hosted HTTP at `docs.rcac.purdue.edu/mcp`, `doc_search`/`doc_load`). + Link each public repo. Use the **verified** connect blocks from research 04 (inline fenced + JSON/TOML — short, no Jinja triggers). +- [ ] State that the HPC server reads the host's `/etc/agents.d` markdown over SSH and injects it as + context (R8). +- [ ] Frame the `cluster-mcp[slurm,lmod,…]` plugin/HPC-refocus direction as **planned/evolving**, + clearly labeled — NOT as shipped fact (research 04 marks it `[UNVERIFIED]`). ⚠ Surface to the + human at review/publish. +- [ ] Add nav line; back-link to hub. +- **Verify:** `… strict_check.py && grep -q 'mcp_servers.md' mkdocs.yml` +- **Touches:** `docs/agentic-ai/mcp_servers.md`, `mkdocs.yml`. + +## Phase P5 — Running Agents: overview + on-cluster +**Satisfies:** R4, R5 · **Depends on:** P1 · **hammerable:** false +**Goal:** the `Running Agents` area exists (mode overview) plus the on-cluster (login-node) page +covering all five harnesses with the login-node constraints. + +- [ ] Create `docs/agentic-ai/running_agents/index.md` (subdir hub): explain the two deployment + modes (on-cluster vs local MCP+SSH); card/links to the two mode pages (local link deferred to + P10 if `local.md` not yet created — or ordered so on_cluster exists here and local added P6). +- [ ] Create `docs/agentic-ai/running_agents/on_cluster.md`: install/run each of Claude Code, Codex, + Gemini CLI, opencode on a login node (Linux); the login-node compute constraint (mirror the + Gautschi Running-Jobs rule); point writable work at `$RCAC_SCRATCH`; note OS sandboxes are + often unavailable on shared nodes. **Warp:** state honestly it is a local desktop GUI and + **cannot** run on a login node — run it on your workstation and SSH in (see local page). +- [ ] Add the `Running Agents` nav sub-section (index + On the Cluster); back-links to hub. +- **Verify:** `… strict_check.py && grep -q 'running_agents/on_cluster.md' mkdocs.yml` +- **Touches:** `docs/agentic-ai/running_agents/{index,on_cluster}.md`, `mkdocs.yml`. + +## Phase P6 — Running Agents: local (MCP + SSH) +**Satisfies:** R4, R6 · **Depends on:** P1, P5 · **hammerable:** false +**Goal:** the local-mode page — agent runs in the user's own environment, executes remotely over +existing SSH via the MCP servers. + +- [ ] Create `docs/agentic-ai/running_agents/local.md`: the local-first architecture (no new + credentials, no hosted infra — "if you can SSH to the cluster, your agent can too"); how to + register `rcac-mcp` (and globus-mcp / rcac-docs-mcp) in each harness (reference research 05 + MCP-config forms); cite `docs/lifesciences/guides/vscode.md` as prior art for external-tool → + cluster tunneling. Back-link to hub; forward links to `mcp_servers.md` deferred to P10. +- [ ] Add the `Local (MCP + SSH)` nav line under Running Agents. +- **Verify:** `… strict_check.py && grep -q 'running_agents/local.md' mkdocs.yml` +- **Touches:** `docs/agentic-ai/running_agents/local.md`, `mkdocs.yml`. + +## Phase P7 — Shared context: hub + build/publish the 5 context files +**Satisfies:** R10, R12, R16 · **Depends on:** P1 · **hammerable:** false +**Goal:** author the five Gautschi `/etc/agents.d` context files (byte-exact, HPC-accurate) and +publish them verbatim; the shared-context hub explains the model + how to give feedback. + +- [ ] Author canonical files under `docs/snippets/agentic-ai/agents.d/`: `unix.md`, `filesystems.md`, + `lmod.md`, `slurm.md`, `policies.md` — each *fact → correct command/path → "do not…" → + rationale* (outlines in research 06), grounded in verified Gautschi facts (research 03). Also + write `agents.d/AGENTS.md` (the concatenated canonical context harnesses read). **Accuracy + gate:** partitions cpu/ai/highmem/smallgpu/profiling; QOS normal/standby/preemptible; `-A` + (via `slist`) + `-q` + `-p` mandatory; `$RCAC_SCRATCH`, 60-day purge, `myquota`/`findscratch`; + GCC 14.1.0 + OpenMPI; **no `--partition=a10`, no `-A standby`**; defer quota numbers to + `myquota`. +- [ ] Create `docs/agentic-ai/shared_context/index.md` (subdir hub): what `/etc/agents.d` is, that + these files are the **canonical single source of truth** copied into cluster config management + (Puppet) and injected by the MCP server over SSH, and **how to contribute feedback/corrections** + (RCAC-Docs GitHub issues + rcac-help@purdue.edu + Discord). +- [ ] Create `docs/agentic-ai/shared_context/context_files.md`: publish each context file verbatim + via `--8<--` fenced `markdown` blocks with `title=` labels. +- [ ] Add the `Shared Context & Settings` nav sub-section (index + Context Files); back-links to hub. +- [ ] `mkdocs serve` — confirm the included blocks render non-empty (check_paths footgun). +- **Verify:** `… strict_check.py && grep -q 'shared_context/context_files.md' mkdocs.yml && grep -rq RCAC_SCRATCH site/agentic-ai/shared_context/` +- **Touches:** `docs/snippets/agentic-ai/agents.d/*.md`, `docs/agentic-ai/shared_context/{index,context_files}.md`, `mkdocs.yml`. + +## Phase P8 — Build/publish per-harness settings & permissions +**Satisfies:** R11, R12 · **Depends on:** P1, P7 · **hammerable:** false +**Goal:** author and publish verbatim the per-harness settings/permission files for all five +harnesses, wiring in the shared context and encoding a starting-point enforced-permission policy. + +- [ ] Author canonical files under `docs/snippets/agentic-ai/`: `claude/settings.json` + (`permissions.deny` for `rm -rf`/`sudo`; note managed `/etc/claude-code/managed-settings.json` + as the enforcement point; `mcpServers` for rcac-mcp; CLAUDE.md→AGENTS.md bridge), + `codex/config.toml` (`approval_policy`/`sandbox_mode`; `[mcp_servers.rcac]`; AGENTS.md native), + `gemini/settings.json` (`tools.exclude` `run_shell_command(rm)`/`(sudo)`; note system + `/etc/gemini-cli/settings.json` as the enforcement point; `mcpServers`), `opencode/opencode.json` + (`permission` bash-pattern map, catch-all first; `mcp` local), `warp/AGENTS.md` (Warp has no + server-side settings file — represent it as the repo AGENTS.md + a documented Agent-Profile + denylist; note Run-until-completion bypasses the denylist). Use research 05 for exact schema; + mark any `[UNVERIFIED]` key honestly. +- [ ] Create `docs/agentic-ai/shared_context/settings.md`: publish each settings file verbatim via + `--8<--` fenced blocks (json/toml/markdown); explain each harness's permission model briefly, + that this is a **v0 starting point** inviting feedback, and where config-management enforcement + lives (managed/system files). Note OS-sandbox weakness on shared nodes. +- [ ] Add the `Harness Settings & Permissions` nav line; back-link to hub. +- [ ] `mkdocs serve` — confirm blocks render non-empty. +- **Verify:** `… strict_check.py && grep -q 'shared_context/settings.md' mkdocs.yml && grep -rq mcpServers site/agentic-ai/shared_context/` +- **Touches:** `docs/snippets/agentic-ai/{claude,codex,gemini,opencode,warp}/…`, `docs/agentic-ai/shared_context/settings.md`, `mkdocs.yml`. + +## Phase P9 — Gautschi "Using AI Agents" chapter +**Satisfies:** R13, R16 · **Depends on:** P1, P4, P5, P6, P7 · **hammerable:** false +**Goal:** a Gautschi user-guide chapter giving cluster-specific setup pointers, cross-linked with the +top-level section. + +- [ ] Create `docs/userguides/gautschi/using_ai_agents.md` (userguide archetype): `tags: [Gautschi]`, + `authors`, `{% set resource = "gautschi" %}`; Gautschi-specific quick-start (login host + `gautschi.rcac.purdue.edu`, `slist`/`myquota`, partitions/QOS reminders), and links into the + top-level Agentic AI pages (MCP servers, running-agents, shared context). Reuse macros for + SSH/module/Slurm background where helpful. +- [ ] Insert the `Using AI Agents` nav line between Compiling Source Code (`mkdocs.yml:144`) and FAQs. +- **Verify:** `… strict_check.py && grep -q 'using_ai_agents.md' mkdocs.yml` +- **Touches:** `docs/userguides/gautschi/using_ai_agents.md`, `mkdocs.yml`. + +## Phase P10 — Integration: cards, cross-links, breadcrumbs, sweep +**Satisfies:** R14, R15, R16 · **Depends on:** P1–P9 · **hammerable:** false +**Goal:** the section reads as one coherent whole; every internal link resolves; breadcrumbs are +current; a11y and HPC accuracy pass a final review. + +- [ ] Populate the hub grid-cards (`agentic-ai/index.md`) linking every subpage; add the + Running-Agents index card to `local.md`; add sibling cross-links (running-agents ↔ mcp_servers + ↔ shared_context ↔ Gautschi chapter). +- [ ] `python tools/generate_breadcrumbs.py` (nav changed). +- [ ] a11y sweep: one H1/page, descending headings, header rows on tables, descriptive link text, + alt text on any image. +- [ ] HPC-accuracy final pass against research 03/04: partitions, QOS, paths, commands, MCP connect + strings; confirm **no `--partition=a10` / `-A standby`** anywhere in the new pages/snippets. +- [ ] Full `mkdocs serve` eyeball of the whole section. +- **Verify:** `… strict_check.py && python tools/generate_breadcrumbs.py && grep -q 'Agentic AI' mkdocs.yml` +- **Touches:** `docs/agentic-ai/**`, `docs/userguides/gautschi/using_ai_agents.md`, `docs/assets/data/breadcrumbs.json`, `mkdocs.yml`. + +--- + +## How `docs-draft` drives this + +1. `next_phase.py spec/agentic-ai-docs/TECH.md` prints the next actionable phase (statuses are + authoritative). +2. Pre-flight: clean tree, on `feature/agentic-ai-docs`, `main` reachable, env active + (`python3 -c "import yaml, mkdocs"`). +3. Execute every `[ ]` in the phase (consult `PLAN.md` / `research/` / `style-guide.md`). +4. Run the phase's `verify:` — never advance on a checkbox alone. +5. Amend this file if reality diverges (`set_phase.py`; note in the commit body). STOP and escalate + only on a **`GOAL.md` contradiction** or a phase stuck `uphill` across drafts (raised hand — esp. + the `cluster-mcp` framing and any newly-surfaced `[UNCONFIRMED]` HPC spec). +6. Mark the phase `done`, advance `current_phase`, `--touch`; one `[feature]` commit; stop & report. diff --git a/spec/agentic-ai-docs/research/00-digest.md b/spec/agentic-ai-docs/research/00-digest.md new file mode 100644 index 00000000..12d1be67 --- /dev/null +++ b/spec/agentic-ai-docs/research/00-digest.md @@ -0,0 +1,174 @@ +# Research digest — agentic-ai-docs + +Synthesis of the six research briefs in this directory. Cited detail lives in each brief; this is +the decision-ready summary that `PLAN.md`/`TECH.md` build on. + +- [`01-site-architecture.md`](01-site-architecture.md) — IA, nav insertion, hub archetype, breadcrumbs. +- [`02-macros-and-jinja.md`](02-macros-and-jinja.md) — the verbatim-publishing pattern (the #1 risk), macros to reuse, policy archetype. +- [`03-gautschi-facts.md`](03-gautschi-facts.md) — citable Gautschi facts + `[UNCONFIRMED]` list + two doc-errors to avoid. +- [`04-mcp-servers.md`](04-mcp-servers.md) — the three public MCP servers, verified connect commands, tool lists. +- [`05-harness-configs.md`](05-harness-configs.md) — per-harness config/permission/MCP/context-file reference. +- [`06-context-and-conventions.md`](06-context-and-conventions.md) — `/etc/agents.d` content design, context→harness mapping, peer prior art. + +## 1. Information architecture (settled) + +- New **top-level `Agentic AI`** nav section, inserted **after Life Sciences** (`mkdocs.yml:361`), + before the commented Tags line. Life Sciences is the exact structural precedent (a domain hub + with its own `index.md`). Contact/FAQs stay last. +- New **Gautschi chapter** `Using AI Agents`, inserted **between Compiling Source Code + (`mkdocs.yml:144`) and FAQs (`:145`)** so FAQs stays last per cluster convention. +- Hub archetype = `docs/lifesciences/index.md`: `title`+`tags` front-matter, `
` blocks, `## :material-…:` headers. A section subdir needs a bare `index.md` + first child (`navigation.indexes` enabled, `mkdocs.yml:18`). +- Breadcrumbs regenerate from nav (`python tools/generate_breadcrumbs.py`; CI also runs on + `mkdocs.yml` change). Only 7 pre-existing `--strict` warnings in `strict-baseline.txt`, none in + our files → the bar is **zero new warnings**. +- **No existing agentic-AI docs** — net-new. Closest analogs: `docs/userguides/anvil/anvilgpt.md` + (a hosted chatbot, not a coding agent) and `docs/lifesciences/guides/vscode.md` (external + tool → login → compute via ProxyJump; useful structural precedent for the local mode). + +## 2. The verbatim-config pattern (the #1 technical risk — SOLVED) + +The `mkdocs-macros` plugin evaluates Jinja on every page's own source **before** Markdown parsing; +`pymdownx.snippets` (`--8<--`) splices files in **during** parsing, **after** Jinja. **Therefore +`--8<--`-included content is never Jinja-evaluated** — literal `{{`/`{%` passes through untouched +(proven empirically, `mkdocs build --strict` exit 0). + +**Pattern:** keep each context/settings artifact as a real byte-exact file under +`docs/snippets/agentic-ai/…` (excluded from nav by `exclude_docs: snippets/`, still `--8<--`-includable +and present in the prod build) and embed it in a fenced code block: + +````markdown +```json title="~/.claude/settings.json" +--8<-- "docs/snippets/agentic-ai/claude/settings.json" +``` +```` + +Single source of truth, syntax-highlighted, zero escaping, page keeps `{{ macro() }}` access. +**Footgun:** `check_paths:false` — a mistyped include path yields a silent **empty** block (no +`--strict` error). Mitigate by grepping the built `site/` for a sentinel token in each phase's +`verify:` and eyeballing the render. + +**Macros to reuse** (call, don't hardcode; `{% set resource = "gautschi" %}` then `{{ macro(resource) }}`): +`resource_use` (Purdue AUP block), `module_system`, `slurm_general_overview`, `slurm_general_jobs`, +`ssh_keys_snippet`, `storage_quota`, `scratch_space`, `scratch_purge`, `environment_variables`, +`accounts_md_snippet`. **Caveat:** Gautschi's own pages do *not* invoke `module_system`/ +`slurm_general_jobs` (they emit generic RCAC text) — fine for background prose, but the **context +files must be authored from the verified Gautschi facts (§4), not from these generic macros.** + +**Policy archetype** = `docs/userguides/anvil/policies.md` (Gautschi has none): `tags`+`authors` +front-matter, one H1, `{% set resource %}`, hand-written sections + `{{ resource_use(resource) }}` / +`{{ scratch_purge(resource) }}`. + +## 3. MCP servers (verified against the public repos) + +All three are public, MIT, Python/FastMCP, installed for the desktop path via `uvx git+https://…`. + +- **`rcac-mcp`** (PurdueRCAC/rcac-mcp, **prototype**) — HPC cluster+storage tools. Local **stdio** + subprocess; runs commands on the cluster over the user's **existing SSH** (`~/.ssh/config`), no + new credential. Verified config: + ```json + {"mcpServers":{"rcac":{"command":"uvx","args":["git+https://github.com/purduercac/rcac-mcp","--ssh-host","cluster.rcac.purdue.edu"]}}} + ``` + Tools: `run_command`, `list_directory`, `read_file`/`write_file`, `upload_file`/`download_file`, + `myquota`, `storage_paths`, `jobinfo`/`jobcmd`/`jobenv`/`jobscript`, `showpartitions`, + `average_wait`, `sbatch`/`squeue`/`scancel`/`sacct`/`sinfo`, `scontrol_show_job`/`_node`, `slist`, + `sfeatures`, `doc_search`, `doc_load`. **Confirmed:** reads `/etc/agents.d/*.md` over SSH and + exposes it as resource `rcac://context`. Optional hosted HTTP with JWT/OIDC delegation. +- **`globus-mcp`** (**BETA**, banner-warned) — wraps Globus CLI + Compute SDK; user's own Globus + OAuth (browser login). stdio via `uvx git+…`. Tools: `whoami`, `endpoint_search/show`, `ls/stat/ + mkdir/rename/rm/delete`, `transfer/transfer_batch`, `task_*`, `compute_*`. +- **`rcac-docs-mcp`** — hosted **no-auth HTTP at `docs.rcac.purdue.edu/mcp`** (add as a remote HTTP + MCP server; exact client JSON `[UNVERIFIED]`), or local stdio via `uvx git+…`. Two tools: + `doc_search(query, category=None)`, `doc_load(path)` over an FTS5/BM25 index. + +**⚠ [UNVERIFIED]:** the "HPC-only refocus / `cluster-mcp[slurm,lmod,…]` plugin architecture" (from +GOAL R7) has **zero public mentions** — the public `rcac-mcp` still bundles docs tools. **Docs must +present `rcac-mcp` as the current prototype and frame the plugin direction as planned/evolving, +clearly labeled — not as shipped fact. Confirm with the human before publish.** Also noted: private +`agentbase` (internal-KB agentic search) — out of scope. + +## 4. Gautschi facts (safety-critical — see brief 03 for citations) + +- **System:** Rocky Linux 9; login `gautschi.rcac.purdue.edu`; 8 front-ends (192-core EPYC 9654, + 768 GB); auth = SSH keys or Purdue Login MFA (**not** "BoilerKey" — that term is absent). +- **Slurm:** every job needs 4 parts and must pass **both `-A/--account=` and `-p/--partition=`** + (+ optional `-q/--qos=`). Partitions: `cpu` (192c, ~2 GB/core, normal+standby, 2wk/4h), + `ai` (112c, 8× H100 80 GB, normal+preemptible, 2wk), `highmem` (192c, 1.5 TB, 24h, >48-core rule), + `profiling` (whole-node, 24h), `smallgpu` (128c, 2× L40S, 24h). QOS: `normal` (default, charged), + `standby` (idle, free, 4h max), `preemptible` (ai only, 0.25 GPU-hr, killable). List accounts with + **`slist`**. Verified commands: `sbatch`, `squeue -u`, `scontrol show/hold/release`, `scancel`, + `sinteractive`, `srun`, `slist`, `sfeatures`, `myquota`, `findscratch`, `purgelist`. +- **Filesystems:** home `/home/$USER` (`$HOME`, GPFS, snapshots); scratch `/scratch/gautschi/$USER` + via **`$RCAC_SCRATCH`** (NOT `$CLUSTER_SCRATCH`), **60-day** purge, not backed up; Depot/Fortress + for long-term; check with **`myquota`** / `findscratch`. **Quotas (25 GB home / 100 TB scratch) + are illustrative `myquota` output, not official — context files should tell the agent to RUN + `myquota`, not hardcode numbers.** +- **Modules:** environment modules (Lmod — implied by `module spider`); `module avail/spider/load`; + recommended GCC 14.1.0 + OpenMPI (exact OpenMPI version unstated). +- **Login-node rule (quote):** "Do NOT run large, long, multi-threaded, parallel, or CPU-intensive + jobs on a front-end login host… Always use SLURM to submit your work as a job." + (`run_jobs/index.md:16-17`). +- **🚫 Two doc-errors to NOT propagate:** `--partition=a10` (invalid — use `ai`/`smallgpu`) and + `#SBATCH -A standby` (standby is a QOS `-q standby`, not an account). +- **`[UNCONFIRMED]`** (do not invent; run-command or omit): exact quotas, Depot mount path, OpenMPI + version, default login shell, `sinfo`/`sacct`/`jobinfo` on Gautschi, a Purdue AUP direct URL, + data-classification policy. + +## 5. The five harnesses (see brief 05 for the full table) + +| harness | config path(s) | format | permission mechanism | MCP | context file | +|---|---|---|---|---|---| +| **Claude Code** | `~/.claude/settings.json`; project `.claude/settings.json`; **managed `/etc/claude-code/managed-settings.json`** | JSON | `permissions.allow/deny/ask` + `defaultMode` + OS sandbox | `claude mcp add` / `.mcp.json` (stdio+http/sse) | `CLAUDE.md` (no native AGENTS.md — bridge via `@import`/symlink) | +| **Codex CLI** | `~/.codex/config.toml`; `.codex/config.toml` | TOML | `approval_policy` + `sandbox_mode` (no per-cmd list; Linux sandbox needs **bubblewrap**) | `[mcp_servers.NAME]` / `codex mcp add` (stdio+HTTP) | `AGENTS.md` (native) | +| **Gemini CLI** | **`/etc/gemini-cli/settings.json` (system, final say)**; `~/.gemini/settings.json`; `.gemini/settings.json` | JSON | `tools.core` (allow) / `tools.exclude` (deny) `run_shell_command()`; blocklist wins; chains split | `mcpServers` (stdio+SSE+HTTP) | `GEMINI.md`; configurable `context.fileName` (can add AGENTS.md) | +| **opencode** | `~/.config/opencode/opencode.json`; project `opencode.json`; `/etc/opencode/` | JSON | `permission` per-bash-pattern map (**last-match-wins → catch-all first**); no OS sandbox | `mcp` `type:"local"`/`"remote"` | `AGENTS.md` (native) + `instructions` | +| **Warp** | **Desktop-app UI** (Settings > Agents); MCP JSON; Warp Drive rules | UI/JSON | Agent-Profile autonomy + regex allow/deny (Run-until-completion **ignores denylist**) | Settings > Agents > MCP (stdio+remote) | `AGENTS.md` (ALL-CAPS; `WARP.md` legacy) | + +**Cross-cutting HPC realities:** OS sandboxes (bwrap/Docker/namespaces) are frequently **disabled on +shared login nodes** → treat the **permission/approval layer as primary**, point writable roots at +`/scratch/$USER`, deny `rm -rf`/`sudo`. The **enforcement points** for config management are the +*system/managed* files: Claude `/etc/claude-code/managed-settings.json`, Gemini +`/etc/gemini-cli/settings.json`. **Warp is the outlier — a local desktop GUI, cannot be installed on +a login node**; on-cluster coverage for Warp = "run it on your workstation and SSH in" (document +honestly, don't fabricate a login-node path). + +## 6. Context-file design + mapping + prior art + +- **`/etc/agents.d/*.md` = the DRY source of truth.** Shape per file: *fact → correct command/path + → "do not…" list → rationale* (mirrors the paper's own `rules/`+`tips/`). Per-file outlines: + - **unix.md** — Rocky 9 / bash; shared front-ends for edit/compile/submit only; no heavy compute + on login nodes; no sudo; SSH-inherited access; process hygiene. + - **filesystems.md** — home vs `$RCAC_SCRATCH` (not backed up, ~60-day purge) vs `/tmp` vs + Depot/Fortress; `myquota`/`findscratch`; no secrets in world-readable paths; confirm bulk deletes. + - **lmod.md** — software via environment modules, not apt/yum; `module avail/spider/load`; GCC + 14.1.0+OpenMPI; never invent module names. + - **slurm.md** — job needs resources + `-A` (via `slist`) + `-q` + `-p` (mandatory); partitions + cpu/ai/highmem/smallgpu/profiling; QOS normal/standby/preemptible; always set `--time`; no + PBS/LSF flags; **never** `--partition=a10` or `-A standby`. + - **policies.md** — Purdue AUP; sensitive-data & credential bans; resource stewardship; + destructive-op confirmation; report to rcac-help@purdue.edu. +- **Mapping context → harnesses:** concatenate `/etc/agents.d/*.md` into one canonical **`AGENTS.md`**; + symlink/bridge `CLAUDE.md` (Claude has no native AGENTS.md); Codex/opencode/Warp/Gemini read + `AGENTS.md` (Gemini via `context.fileName`). **Local mode:** `rcac-mcp` injects the files over SSH + (`rcac://context`). **Permissions are NOT portable** — hand-translate one policy into each harness's + native settings format. +- **Prior-art verdict — early but not empty:** NERSC has a full [AI Coding Tools page](https://docs.nersc.gov/development/coding-agents/) + ("verification is the whole game"; login-node + secrets rules) — the closest analog; TACC has + [AI Tools on TACC Resources](https://docs.tacc.utexas.edu/basics/conduct/#ai) (compute-nodes-only, + user-accountable). ALCF/NCSA — nothing agent-facing found. **No peer publishes an MCP/Slurm server + or `/etc/agents.d` context** → RCAC's genuine white space. (A broad community "slurm-mcp" sweep was + org-policy-blocked mid-research; not required for this feature.) + +## 7. Net implications for the plan + +1. One top-level `docs/agentic-ai/` section (hub + guidance + MCP + running-agents + shared-context + subdir) + one Gautschi chapter; canonical artifacts under `docs/snippets/agentic-ai/`. +2. Publish verbatim via `--8<--` fenced includes; verify by grepping built `site/` (check_paths + footgun). +3. Author context files from verified Gautschi facts; instruct agents to run `myquota`/`slist` + rather than hardcoding volatile numbers; never emit the two doc-errors. +4. Present MCP `rcac-mcp` as prototype; the `cluster-mcp` plugin direction is planned/evolving and + needs human confirmation before publish. +5. Warp is local-only; be honest about the on-cluster gap. Enforcement lives in the system/managed + settings files (Claude `/etc/claude-code/`, Gemini `/etc/gemini-cli/`). diff --git a/spec/agentic-ai-docs/research/01-site-architecture.md b/spec/agentic-ai-docs/research/01-site-architecture.md new file mode 100644 index 00000000..b1f150af --- /dev/null +++ b/spec/agentic-ai-docs/research/01-site-architecture.md @@ -0,0 +1,438 @@ +# Research 01 — Site Architecture & Navigation Mechanics + +Scope: how to add (a) a new top-level **Agentic AI** nav section and (b) a **Using AI Agents** +chapter inside the Gautschi user guide. All citations are to files under the repo root +`/Users/geoffrey/Software/github.com/purduercac/RCAC-Docs`. Read-only investigation; no site +files were modified. + +--- + +## 1. `mkdocs.yml` nav — top-level sections and exact insertion points + +The `nav:` block is `mkdocs.yml:96-364`. The **top-level sections, in order**, are: + +1. `HPC User Guides:` — `mkdocs.yml:97` +2. `Storage User Guides:` — `mkdocs.yml:206` +3. `RCAC Resources:` — `mkdocs.yml:245` +4. `Life Sciences:` — `mkdocs.yml:332` +5. `# - Tags: tags.md` (commented out) — `mkdocs.yml:362` +6. `- Contact us: contact.md` — `mkdocs.yml:363` +7. `- FAQs: faqs.md` — `mkdocs.yml:364` + +`Life Sciences` is the closest structural precedent for `Agentic AI`: it is a top-level +*domain hub* (its own `index.md` + `Getting Started` / `Guides` / `Tutorials` subsections), +not a cluster or storage guide. See `mkdocs.yml:332-361`: + +```yaml + - Life Sciences: + - lifesciences/index.md + - Getting Started: + - HPC Orientation for Biologists: lifesciences/guides/hpc-orientation.md + ... + - Tutorials: + - Gene Prediction: + - BRAKER3: lifesciences/tutorials/braker.md + ... + - Juicer: lifesciences/tutorials/juicer.md + # - Tags: tags.md + - Contact us: contact.md + - FAQs: faqs.md +``` + +### RECOMMENDED insertion point for top-level "Agentic AI" + +**Insert immediately after the `Life Sciences` section ends (after `mkdocs.yml:361`) and +before the commented `# - Tags` line (`mkdocs.yml:362`).** This groups the two domain hubs +(Life Sciences, Agentic AI) together and keeps the utility items (`Contact us`, `FAQs`) last — +consistent with the current layout. (Alternative acceptable slot: after `Storage User Guides` +ends at `mkdocs.yml:244`, before `RCAC Resources`. The Life-Sciences-adjacent slot is +preferred for domain-hub parallelism.) + +The last two lines of the Life Sciences block to anchor against are `mkdocs.yml:360-361`: + +```yaml + - Hi-C Analysis: + - Juicer: lifesciences/tutorials/juicer.md +``` + +**PASTE-READY snippet** (new lines to add directly below `mkdocs.yml:361`, at the same +2-space indent as `- Life Sciences:`). A section that owns a direct `index.md` child gets a +breadcrumb parent (see §4), so include one: + +```yaml + - Agentic AI: + - agentic-ai/index.md + - Overview: agentic-ai/overview.md + - Using AI Agents on RCAC: agentic-ai/using-ai-agents.md + # ...additional pages, each also created under docs/agentic-ai/ + linked from the hub +``` + +Notes for the drafter: +- The exact content dir/filenames above are a proposal; whatever paths you use must exist + under `docs/` and each must appear here or the page is orphaned (invariant §4; + `.agents/factory/style-guide.md:153`). +- `docs_dir` is unset, so it defaults to `docs/` (see §5) — paths are relative to `docs/`. +- `Agentic AI` is **not** in `BREADCRUMB_TITLE_OVERRIDES` + (`tools/generate_breadcrumbs.py:39-55`), so it renders verbatim as "Agentic AI" in + breadcrumbs. Add an override there only if you want a longer label. + +### Gautschi sub-tree block + +The Gautschi cluster block is `mkdocs.yml:135-145` (nested under `HPC User Guides` → +2nd-level, 4-space indent for the `- Gautschi:` key, 6-space for its children): + +```yaml + - Gautschi: + - userguides/gautschi/index.md + - Gautschi Overview: userguides/gautschi/overview.md + - Biography of Gautschi: userguides/gautschi/biography.md + - Accounts: userguides/gautschi/accounts.md + - Software: userguides/gautschi/software.md + - Running Jobs: userguides/gautschi/run_jobs/index.md + - File Storage and Transfer: userguides/gautschi/storage.md + - Gateway (Open OnDemand): userguides/gautschi/gateway.md + - Compiling Source Code: userguides/gautschi/compile.md + - Frequently Asked Questions: userguides/gautschi/faqs.md +``` + +### RECOMMENDED insertion point for the Gautschi "Using AI Agents" chapter + +**Insert between `Compiling Source Code` (`mkdocs.yml:144`) and `Frequently Asked Questions` +(`mkdocs.yml:145`).** Across every cluster, `Frequently Asked Questions` is the last chapter; +keeping FAQs last preserves that convention. So the new line becomes the second-to-last child. + +PASTE-READY — the Gautschi block after the edit (new line marked; single flat page): + +```yaml + - Gautschi: + - userguides/gautschi/index.md + - Gautschi Overview: userguides/gautschi/overview.md + - Biography of Gautschi: userguides/gautschi/biography.md + - Accounts: userguides/gautschi/accounts.md + - Software: userguides/gautschi/software.md + - Running Jobs: userguides/gautschi/run_jobs/index.md + - File Storage and Transfer: userguides/gautschi/storage.md + - Gateway (Open OnDemand): userguides/gautschi/gateway.md + - Compiling Source Code: userguides/gautschi/compile.md + - Using AI Agents: userguides/gautschi/using_ai_agents.md # <-- NEW + - Frequently Asked Questions: userguides/gautschi/faqs.md +``` + +If the chapter needs sub-pages, use a nested section with its own `index.md` (this also earns +a breadcrumb parent): + +```yaml + - Using AI Agents: + - userguides/gautschi/using_ai_agents/index.md + - Getting Started: userguides/gautschi/using_ai_agents/getting-started.md + - Frequently Asked Questions: userguides/gautschi/faqs.md +``` + +Also add a matching bullet to the Gautschi hub list in +`docs/userguides/gautschi/index.md` (the hub ends at the "Frequently Asked Questions" +bullet) so the hub page and the nav stay in sync — the hub currently lists each chapter as a +markdown bullet (`docs/userguides/gautschi/index.md`, the `- [**...**](...)` list). + +> Caveat — invariant §10 (per-cluster parallelism, `.agents/factory/invariants.md`; AGENTS.md +> "Invariants" §10): the cluster chapter set is deliberately identical across clusters. A +> Gautschi-only "Using AI Agents" chapter intentionally breaks that symmetry. It is what the +> task asks for; just flag it in REVIEW so it is a conscious exception, not an accident. + +Naming: use lowercase `snake_case` (`using_ai_agents.md`), per +`.agents/factory/style-guide.md:149`. + +--- + +## 2. Hub / landing archetype (grid cards, front-matter, section headers) + +### `docs/lifesciences/index.md` (a clean section-hub example) + +Front-matter (`docs/lifesciences/index.md:1-5`): + +```yaml +--- +title: Life Sciences +tags: + - Life Sciences +--- +``` + +Then a single `#` H1, a one-paragraph intro, and one `grid cards` block. The card grid +(`docs/lifesciences/index.md:11-45`) is the canonical pattern: + +```markdown +
+ +- :material-rocket-launch:{ .lg .middle } __Start Here__ + + --- + + New to HPC for bioinformatics? Begin with the orientation guide. + + [:octicons-arrow-right-24: HPC Orientation for Biologists](guides/hpc-orientation.md) + +- :material-file-document:{ .lg .middle } __Guides__ + + --- + + Step-by-step guides for common bioinformatics tasks: Nextflow, R, SRA downloads, QC, and more. + + [:octicons-arrow-right-24: Running Bioinformatics on RCAC](guides/running-bioinformatics.md) + +
+``` + +Anatomy of one card: `- :material-ICON:{ .lg .middle } __Bold title__` → blank line → +` ---` (horizontal rule = card header divider) → blank line → description prose → blank +line → an arrow link `[:octicons-arrow-right-24: label](relative/path.md)`. Indentation +inside a card is 4 spaces. Links are **relative** page-to-page (invariant §5). + +### `docs/index.md` (the site home page — richer variant) + +Front-matter (`docs/index.md:1-9`) uses `hide:` and a wrapping div: + +```yaml +--- +hide: + - footer + - toc +# - navigation +meta: + - name: google-site-verification + content: HQsOhuVVa11DY0qIrxXPW6GUR9jk4EnFfQgUzCgKNbw +--- +``` + +Body is wrapped in `
` … `
` +(`docs/index.md:11` and `:291`). It uses **section headers with an inline icon**, e.g. +`docs/index.md:43`, `:99`, `:215`, `:253`: + +```markdown +## :material-rocket-launch: New to RCAC? +## :material-server-network: HPC User Guides +## :material-bookshelf: RCAC Resources +## :material-lifebuoy: Need Help? +``` + +Each header is followed by its own `
` block. Cards can carry +a **bold spec line** (`docs/index.md:109`, `:129`) and **multiple arrow links separated by +` `** (`docs/index.md:55`): + +```markdown + **128 cores/node | 256GB-1TB RAM | A100/H100 GPUs** + + [:octicons-arrow-right-24: Purdue account](https://www.rcac.purdue.edu/account/request)    [:octicons-arrow-right-24: ACCESS account](userguides/anvil/access/) +``` + +### `.md-button` / quick-launcher patterns + +- `.md-button` is documented in `.agents/factory/style-guide.md:129` (`[text](url){ .md-button }`) + and `:68` (`[Back to index](index.md){ .md-button }`) — used for back/prev-next nav on + lessons, not seen on the two hub pages above. +- The Gautschi hub uses a bespoke **quick-launcher** widget instead of grid cards + (`docs/userguides/gautschi/index.md`): a `
` containing + `quick-launcher-card` blocks with `` — styled from + `docs/stylesheets/extra.css`. This is the cluster-hub convention (index.md + a bullet list + of chapters), distinct from the domain-hub grid-cards convention. + +### Copyable skeleton for a new "Agentic AI" section hub (`docs/agentic-ai/index.md`) + +```markdown +--- +title: Agentic AI +tags: + - Agentic AI +--- + +# Agentic AI + +One or two sentences describing agentic AI on RCAC systems and who this section is for. + +
+ +- :material-robot:{ .lg .middle } __Start Here__ + + --- + + New to running AI agents on RCAC? Begin with the overview. + + [:octicons-arrow-right-24: Overview](overview.md) + +- :material-cog:{ .lg .middle } __Using AI Agents__ + + --- + + Configure and run agentic AI tools on RCAC clusters. + + [:octicons-arrow-right-24: Using AI Agents on RCAC](using-ai-agents.md) + +
+``` + +Add each card's target page under `docs/agentic-ai/` and register it in `nav:` (§1). If you +want the home-page `## :material-...:` section-header style, that is only needed on +`docs/index.md`; a section hub typically leads with its `#` H1 + one grid. + +--- + +## 3. `navigation.indexes` and the hub front-matter contract + +`theme.features` (`mkdocs.yml:17-28`) enables, among others: + +```yaml + features: + - navigation.indexes # mkdocs.yml:18 + - navigation.sections # mkdocs.yml:21 + - navigation.top + - navigation.footer + - toc.follow +``` + +`navigation.indexes` is what lets a section's `index.md` become the clickable landing page +for that section header (rather than a separate "Index" child). That is why every section in +`nav:` lists a **bare `index.md` string as its first child** (e.g. `mkdocs.yml:99` +`- userguides/anvil/index.md`; `mkdocs.yml:136` `- userguides/gautschi/index.md`; +`mkdocs.yml:333` `- lifesciences/index.md`). To make `Agentic AI` (or a nested `Using AI +Agents`) behave as a landing hub, its **first child must be the bare `…/index.md` string**. + +The style guide confirms the convention: "Every section directory has an `index.md` hub +(`navigation.indexes`)" — `.agents/factory/style-guide.md:151`. + +Front-matter contract, cross-checked against `.agents/factory/style-guide.md`: + +- **Hub / landing page** (archetype E, `.agents/factory/style-guide.md:70-73`): "Front-matter + uses `hide: [footer]` (and often `toc`); minimal prose; grid-card or bulleted navigation." + The site home does exactly this (`docs/index.md:1-9`). A lighter domain hub may instead use + just `title` + `tags` (`docs/lifesciences/index.md:1-5`) — both are in use; pick per how + much chrome you want to hide. +- **Normal cluster user-guide page** (archetype A, `.agents/factory/style-guide.md:13-26`): + `tags:` (cluster name) + `authors:` (RCAC handle) + `search: boost: 2`, plus `resource:` + only if the body uses `{{ resource }}`. The Gautschi index/overview confirm this exact set + (`docs/userguides/gautschi/index.md:1-8`, `docs/userguides/gautschi/overview.md:1-8`). + +So a **Gautschi "Using AI Agents"** page (living under `docs/userguides/gautschi/`) should +carry the archetype-A front-matter (`tags: [Gautschi]`, `authors:`, `search: boost: 2`), +**not** the hub front-matter — unless it becomes a nested section with its own index.md, in +which case that index.md is the hub. + +--- + +## 4. Breadcrumbs — trigger, output, whether the drafter must regenerate + +### Generator header (`tools/generate_breadcrumbs.py:1-13`) + +``` +Generate breadcrumbs.json from mkdocs.yml nav. + - Only include parent titles if that parent section contains an actual index.md + - Do NOT emit an entry for the root "/" (no breadcrumb for Home page) + - Writes output to docs/assets/data/breadcrumbs.json so MkDocs will serve it +``` + +Key mechanics for planning: +- **Output path:** `docs/assets/data/breadcrumbs.json` + (`tools/generate_breadcrumbs.py:62`, `OUTPUT_JSON`). It is a **generated file** (invariant + §3 — do not hand-edit; regenerate). +- **Parent inclusion rule:** a nav section contributes its title to a child's breadcrumb chain + **only if that section contains a direct `index.md` string child** + (`tools/generate_breadcrumbs.py:110-113`). This is the concrete reason the `Agentic AI` + section (and any nested `Using AI Agents` section) must include a bare `…/index.md` first + child if you want it to appear as a breadcrumb ancestor. +- **Title overrides:** `BREADCRUMB_TITLE_OVERRIDES` (`tools/generate_breadcrumbs.py:39-55`) + maps e.g. `Gautschi` → `Gautschi User Guide`. So a Gautschi child page's breadcrumb will + read `Home / Gautschi User Guide / Using AI Agents`. `Agentic AI` has **no** override, so it + appears verbatim; add an entry there only if a different breadcrumb label is desired. + +### CI trigger (`.github/workflows/update_breadcrumbs.yml`) + +```yaml +on: + push: + branches: [ main, dev ] # lines 4-7 + paths: + - 'mkdocs.yml' # lines 8-9 +``` + +The workflow runs only when **`mkdocs.yml` changes** on `main` or `dev`. It runs +`python tools/generate_breadcrumbs.py` (`.yml:50`) and **bot-commits** the regenerated +`docs/assets/data/breadcrumbs.json` as `purduercac-docs-bot` (`.yml:43-46, 52-69`). + +### Does the drafter need to run it manually? + +**Optional but recommended.** CI will regenerate and bot-commit it after the nav change lands +on `main`/`dev`. However: +- For local `mkdocs serve` / `mkdocs build --strict` verification of the new pages' + breadcrumbs, run it manually: `python tools/generate_breadcrumbs.py` (from repo root — it + resolves `mkdocs.yml` relative to its own location, `tools/generate_breadcrumbs.py:60-61`). +- If you commit the regenerated JSON in the same PR, expect the bot to produce a no-op (or a + redundant) follow-up commit; committing it yourself keeps the PR self-consistent. Either + way it is a generated artifact — never hand-edit it. + +--- + +## 5. `mkdocs.yml` — `docs_dir`, `exclude_docs`, `not_in_nav`, macros/snippets config + +- **`docs_dir`:** not set anywhere in `mkdocs.yml` → defaults to `docs/`. All nav paths and + content live under `docs/`. +- **`exclude_docs`** (`mkdocs.yml:4-5`): + + ```yaml + exclude_docs: | + snippets/ + ``` + + Only `docs/snippets/` is excluded from the build (they are `--8<--` includes, not pages). +- **`not_in_nav`:** **not present** in `mkdocs.yml`. There is no allow-list of intentionally + un-navigated pages, so any page under `docs/` that is not in `nav:` (and not excluded) is a + `--strict` orphan warning. → **Every new Agentic AI / Using AI Agents page must be added to + `nav:`** (invariant §4). +- **`macros` plugin** (`mkdocs.yml:94`): declared as a bare `- macros` with **no config + block** (no `module_name`, no `include_dir`). The macros module is `main.py` at repo root + (the mkdocs-macros default `main`). Extend macros there, don't fork prose (invariant §7). +- **Snippets:** there is **no `snippets` plugin**; snippet includes come from the markdown + extension `- pymdownx.snippets` (`mkdocs.yml:416`), also declared with **no config block** + (default base path = project root, so includes are written `--8<-- "docs/snippets/…"`). + +**Implication for "published verbatim config files":** if the Agentic AI docs need to show +literal config files verbatim (e.g. an MCP/agent config), the clean options are (a) keep them +as pages under `docs/agentic-ai/…` with fenced code blocks, or (b) store a canonical copy +under `docs/snippets/` and pull it in with `--8<--` (snippets are build-excluded, so they +won't orphan). A file placed under `docs/` that is neither in `nav:` nor under `snippets/` +will trip the `--strict` orphan check. Also mind Jinja escaping: literal `{{`/`{%` in prose +must be wrapped in `{% raw %}` because the `macros` plugin processes every page +(invariant §7; `.agents/factory/style-guide.md:102-108`). + +--- + +## 6. `--strict` baseline (known pre-existing warnings) + +Source: `.agents/factory/strict-baseline.txt`. Verify gate command +(`.agents/factory/strict-baseline.txt:13`): + +``` +.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py +``` + +Exit 0 = no *new* warnings vs baseline (PASS); exit 1 = a regression your change introduced. +CI itself builds with `--verbose`, **not** `--strict`, so these ship silently today +(`.agents/factory/strict-baseline.txt:5-9`). The **7 known pre-existing warnings** +(`.agents/factory/strict-baseline.txt:25-31`), summarized: + +1. `software/ngc_catalog.md` — bad image link `../../assets/images/nvidia_logo.png` (should be + `../assets/...`). **Generated file — fix at `tools/`, not by hand.** +2. `software/rocm_catalog.md` — bad image link `../../assets/images/rocm_logo.png`. + **Generated — fix at `tools/`.** +3. `userguides/gilbreth/run_jobs/learning/customml.md` — broken `index.md` link (missing + `learning/index.md`). +4. `userguides/gilbreth/run_jobs/learning/ml_batch.md` — same missing `learning/index.md`. +5. `userguides/negishi/run_jobs/biocontainers.md` — wrong `../` depth on a biocontainers image. +6. `userguides/negishi/run_jobs/windows/launcher.md` — wrong `../` depth on a windows image. +7. `userguides/scholar/run_jobs/r_example.md` — broken cross-cluster link to a Gautschi + `example_r_profile_setup.md` (doubled `userguides/`). + +None touch `docs/index.md`, `docs/lifesciences/`, `docs/agentic-ai/`, or the Gautschi files +being edited — so the drafter starts from a clean slate for the new work. The bar: introduce +**zero** new `--strict` warnings. The #1 way to add one is getting relative-link `../` depth +wrong (`.agents/factory/style-guide.md:138-140`), so count directory levels carefully on the +new pages and hub links, and add every new page to `nav:` to avoid orphan warnings. +Last baseline capture: 2026-07-14, mkdocs 1.6.1 / mkdocs-material 9.6.16 +(`.agents/factory/strict-baseline.txt:23`). diff --git a/spec/agentic-ai-docs/research/02-macros-and-jinja.md b/spec/agentic-ai-docs/research/02-macros-and-jinja.md new file mode 100644 index 00000000..c3c58522 --- /dev/null +++ b/spec/agentic-ai-docs/research/02-macros-and-jinja.md @@ -0,0 +1,321 @@ +# Research 02 — Macros, Jinja, and publishing verbatim config safely + +Scope: how the `mkdocs-macros` Jinja pass interacts with published content, the SAFEST way to +publish verbatim harness config / `agents.d` context files (the #1 technical risk), the reusable +macros we should call instead of hardcoding, and the policy-page archetype to mirror. + +All claims below are backed by file citations and by an isolated, empirical mkdocs build in +`/tmp/macrotest` reproducing the exact stack (`mkdocs 1.6.1`, `mkdocs-macros-plugin 1.3.9`, +`pymdown-extensions 10.16.1`) — versions from `requirements.txt:16-28`. + +--- + +## 0. TL;DR — the recommended verbatim pattern + +**Keep each harness config / context file as a REAL file on disk (exact bytes) under +`docs/snippets/agentic-ai/…`, and embed it in a docs page inside a fenced code block via +`--8<--`.** Proven immune to Jinja regardless of file content, single source of truth, syntax +highlighted, and the surrounding page can still use `{{ macros() }}` freely. + +```markdown +```json title="settings.json" +--8<-- "docs/snippets/agentic-ai/claude/settings.json" +``` +``` + +(That outer fence is just for display here; on the page it is a single ```` ```json ```` block +whose only body line is the `--8<--` directive.) + +Why it is safe: **the macros/Jinja pass runs on a page's OWN source BEFORE Markdown parsing; +`pymdownx.snippets` splices `--8<--` files in DURING Markdown parsing, AFTER Jinja. Included +content is therefore never Jinja-evaluated** (proven in §3). Do NOT paste config bodies directly +into a page unless you wrap them in `{% raw %}…{% endraw %}` or set `render_macros: false` in the +page front matter. + +--- + +## 1. `main.py` macros to REUSE (signatures + what each EMITS) + +`main.py` registers macros via `@env.macro` inside `define_env(env)`. Two flavors: **inline +f-strings** (built from `resource`) and **snippet-file readers** (read a `docs/snippets/*.md` +file and string-substitute `{CLUSTER}`→`resource.title()`, `{cluster}`→`resource.lower()`). + +The invocation idiom (used on every cluster page and documented in `docs/templates.md:70-89`): + +```markdown +{% set resource = "gautschi" %} +{{ module_system(resource) }} +``` + +`{% set resource = "anvil" %}` is real Jinja executed by the macros plugin (see +`docs/userguides/anvil/policies.md:10`). `resource.title()` → `Gautschi`, `resource.lower()` → +`gautschi` (naming convention noted at `main.py:1-4`). + +| Macro (signature) | Kind | Emits | Citation | +|---|---|---|---| +| `resource_use(resource)` | snippet reader (special) | The standard **"Acceptable Purdue IT Research Resource Use"** block: queuing-system policy, links to Purdue IT Policy V.4.1 & Remote Access V.1.6, User Credential Standards, and the activity-logging notice. For `anvil` it deletes the credential-standards lines (`lines[4:6]`). | `main.py:742-748`; source text `docs/snippets/resourceuse.md` | +| `module_system(resource)` | inline f-string | Full **Lmod / `module`** walkthrough for `{resource.title()}`: `module spider/avail/load/unload/purge/show`, hierarchical modules, auto default compiler/MPI, `$RCAC_APPNAME_ROOT/_VERSION`. | `main.py:540-632` | +| `slurm_general_overview(resource)` | inline f-string | The **"## Overview / Slurm Basics"** section: what Slurm is, "SSH lands on a login node" note, "don't run jobs on login nodes" warning, the queue concept — for `{resource.title()}`. | `main.py:246-266` | +| `slurm_general_jobs(resource)` | inline f-string | The large Slurm job tutorial: **Job Submission Script** (sample `#SBATCH`), env-var table, **Submitting a Job** (`sbatch`), multi-node, **Interactive jobs** (`sinteractive`), redirecting output, holding jobs — for `{resource.title()}`. | `main.py:268-430` | +| `ssh_keys_snippet(resource)` | inline f-string | **"### SSH Keys general overview"** + Mac/Linux `ssh-keygen`/`ssh-copy-id` steps + Windows/PuTTY key-generation steps, all parameterized to `{resource}.rcac.purdue.edu`. | `main.py:27-129` | +| `storage_quota(resource)` | snippet reader | The **storage-quota** section (`{CLUSTER}`/`{cluster}` substituted). | `main.py:688-692`; `docs/snippets/storage_quota.md` | +| `scratch_space(resource)` | snippet reader | The **scratch-space** section. | `main.py:694-698`; `docs/snippets/scratch_space.md` | +| `scratch_purge(resource)` | snippet reader (special) | The **scratch-purge policy**; per-cluster edits: `anvil`/`bell` → 60→30 days; `anvil` also strips the purge-warning-email + `purgelist` section and rewrites Depot/Fortress → "project space". | `main.py:750-795`; `docs/snippets/scratchpurge.md` | +| `environment_variables(resource)` | snippet reader | The **environment-variables** section (only `{cluster}`→lower substituted). | `main.py:682-686`; `docs/snippets/environment_variables.md` | +| `accounts_md_snippet(resource)` | inline f-string | The full **"# Accounts on {resource.title()}"** page body: obtaining an account, R4P note, and login methods (ThinLinc web / Gateway-OnDemand / SSH). | `main.py:488-532` | + +Other macros exist (SSH X11, ThinLinc, scp/sftp/globus/htar/hsi, apptainer/biocontainers/rocm, +compile, recover_*) — same two patterns; read `main.py` before hardcoding anything (per CLAUDE.md). + +**Relevance to agentic-AI docs:** for HPC background sections (modules, Slurm, SSH, storage) prefer +these macros over rewriting prose, e.g. `{{ module_system(resource) }}`, `{{ slurm_general_overview(resource) }}`. + +--- + +## 2. `mkdocs.yml` — plugin & extension config that governs the risk + +**`macros` plugin** is registered with **NO options** (`mkdocs.yml:94`): + +```yaml +plugins: + - search + - blog: … + - tags: … + - git-revision-date-localized: … + - rss: … + - macros # <-- line 94, bare, so ALL defaults apply +``` + +Confirmed `mkdocs-macros-plugin 1.3.9` defaults (read from `MacrosPlugin.config_scheme`): + +| Option | Default | Consequence | +|---|---|---| +| `module_name` | `'main'` | `main.py` at repo root is auto-loaded (no override needed). | +| `render_by_default` | `True` | **Every page is Jinja-rendered.** Opt out per page with `render_macros: false` in front matter. | +| `on_undefined` | `'keep'` | A **bare** undefined variable `{{ FOO }}` is kept literally, no error. | +| `on_error_fail` | `False` | A Jinja **error** does not fail a plain build — it logs and (in non-strict) deploys a broken page. | +| `j2_variable_start_string` / `j2_block_start_string` | `''` (Jinja defaults `{{` / `{%`) | Triggers are the standard `{{`, `{%`, `{#`. | + +**`markdown_extensions`** (`mkdocs.yml:404-425`), relevant parts: + +```yaml +markdown_extensions: + - toc: { permalink: true } + - tables + - attr_list + - md_in_html + - admonition + - pymdownx.details + - pymdownx.blocks.caption + - pymdownx.emoji: { … } + - pymdownx.snippets # <-- line 416, bare, ALL defaults apply + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + linenums: true + - pymdownx.inlinehilite + - pymdownx.superfences # <-- line 423, bare + - pymdownx.tabbed: { alternate_style: true } +``` + +**`pymdownx.snippets` is bare (`mkdocs.yml:416`)**, so its defaults apply (read from source): + +| Option | Default | Consequence | +|---|---|---| +| `base_path` | `['.']` | Include paths resolve from the dir mkdocs runs from = **repo root**. So `--8<-- "docs/snippets/…"` is correct (matches all existing includes). Prod confirms it: `Dockerfile.geddes-prod` does `WORKDIR /docs` → `COPY . .` → `mkdocs build`, i.e. runs from the repo-root copy (`Dockerfile.geddes-prod:4,9,13`). | +| `restrict_base_path` | `True` | Cannot include files **outside** the repo tree → canonical files must live inside the repo. | +| `check_paths` | **`False`** | **Footgun:** a mistyped/missing include path silently yields an **empty** code block — no error, even under `--strict`. Mitigate by eyeballing the render (verify gate) or, optionally, enabling `check_paths: true`. | +| `url_download` | `False` | No remote includes; files must be local. | + +`pymdownx.highlight` gives `linenums: true` globally, so every fenced block gets line numbers — +fine for showing config. `pymdownx.superfences` enables the fenced blocks / nesting we rely on. + +Note `exclude_docs: | snippets/` (`mkdocs.yml:4-5`) — anything under `docs/snippets/` is **not +built as a standalone page**, so putting canonical files there does NOT create nav orphans +(satisfies invariant #4) while remaining `--8<--`-includable. + +--- + +## 3. The escaping solution — processing order, proven empirically + +### Processing order (the key fact) +1. **mkdocs-macros runs Jinja on a page's OWN markdown source** (pre-page phase), with `main.py` + macros + `mkdocs.yml` `extra:` vars + front-matter `{% set %}` vars in scope. +2. **Then Markdown conversion runs**; `pymdownx.snippets` (`--8<--`) splices file contents in as + raw text during the Markdown preprocessor phase — **after** Jinja. + +⇒ **Content pulled in by `--8<--` is never seen by Jinja.** Literal `{{`/`{%`/`{#` in an included +file passes straight through. + +### Empirical proof (isolated `/tmp/macrotest`, exact stack, `mkdocs build --strict`, exit 0) +- **Macros run on page source:** `{{ hello("A") }}` in a page → rendered `MACRO_EMITTED[A]`. +- **`--8<--` includes are NOT Jinja-processed:** an included file containing + `"echo {{ THIS_WOULD_BREAK }}"` and `{% if x %}…{% endif %}` rendered **verbatim** (found in + HTML, no error). Had Jinja seen it, the function/undefined-tag would have errored. +- **A raw `.json`/`.toml` file embedded INSIDE a fenced block works:** a real `settings.json` + containing `{{ this_is_not_jinja }}`, `${{ github.token }}`, and `{% raw %}…{% endraw %}`, + included via ```` ```json … --8<-- "…/settings.json" … ``` ````, rendered verbatim with syntax + highlighting, `--strict` exit 0 — **and** a `{{ hello("still-works") }}` macro on the same page + still evaluated. This is the recommended pattern. +- **`{% raw %}…{% endraw %}` around a fenced block in a page's own source works:** a block with + `"{{ user }}"` and `"{% for f in files %}{{ f }}{% endfor %}"` rendered verbatim. +- **`render_macros: false` front matter disables Jinja for the whole page:** a page with + `{{ BOOM }}` and `{% if x %}` in a code block rendered verbatim, `--strict` exit 0. + +### The risk, quantified (why this matters) +- A **bare undefined variable** `{{ FOO }}` → kept (because `on_undefined: keep`), `--strict` exit 0. +- **BUT** a function call / attribute / filter / unknown tag — e.g. `{{ nope_fn("z") }}`, + `{{ a.b }}`, `{% unknowntag %}` — raises `UndefinedError`/`TemplateSyntaxError`. Measured: + under `mkdocs build --strict` this **aborts (exit 1, "Aborted with 1 warnings in strict mode")**; + under plain `mkdocs build` (what CI/prod uses, per CLAUDE.md) it logs an ERROR but **exits 0 and + deploys a garbled page**. Only the factory's `--strict` verify gate catches it. +- Harness config/context files WILL contain such triggers: Warp workflow YAML uses `{{arg}}` + placeholders; GitHub-Actions / Go-template / Jinja examples use `${{ … }}` and `{{ .Var }}`; + hooks and context markdown can contain `{%`. JSON single braces (`{ "a": 1 }`) are NOT triggers + (Jinja needs doubled `{{`/`{%`), and a lone `}}` is harmless — but you cannot safely hand-audit + every file. Make the mechanism immune instead. + +### Existing precedent in the repo +- `docs/templates.md:70-83` documents the intended escape: to *show* macro syntax literally it + wraps `{% set resource = "gautschi" %}` and `{{ ssh_keys_snippet(resource) }}` in + `{% raw %} … {% endraw %}`. +- Live proof that snippets bypass Jinja: `docs/snippets/examples/apps/mathematica.md:66` contains + a **lone `{% endraw %}`** (no opening `{% raw %}`). It is `--8<--`-included with **no** raw + wrapper by `docs/userguides/hammer/run_jobs/examples/apps/mathematica.md:11` and the bell + equivalent — and the site builds. A lone `{% endraw %}` would be a fatal `TemplateSyntaxError` + if snippets were Jinja-processed; it is not, confirming the ordering. (It does render as stray + literal text on those pages — a pre-existing cosmetic bug, not ours to fix here.) + +### Ranking of the three safe options +1. **`--8<--` include of a real file inside a fenced block (RECOMMENDED)** — single source of + truth (the on-disk file is the exact, copyable, operationally-usable config), zero escaping, + syntax-highlighted, page keeps macro access. Only caveat: `check_paths:false` means a bad path + fails silently → verify by rendering. +2. **`{% raw %}…{% endraw %}` around an inline fenced block** — good when the content is short and + you want it literally in the page file, but it **duplicates** the config (two copies to keep in + sync) — violates single-source-of-truth; use only for tiny snippets. +3. **`render_macros: false` front matter** — bulletproof but all-or-nothing: the whole page loses + `{{ macro() }}` access. Use only for a page that is *entirely* verbatim config and needs no macros. + +--- + +## 4. Policy-page archetype (`docs/userguides/anvil/policies.md`) + `resourceuse.md` + +**Gautschi has NO `policies.md`** — confirmed: `find docs -name policies.md` returns only +`docs/userguides/anvil/policies.md`. The policy archetype currently exists for Anvil alone, so +there is no per-cluster policy parallelism to maintain (invariant #10 doesn't bind here). + +**Structure to mirror** (`docs/userguides/anvil/policies.md`): +```markdown +--- +tags: + - Anvil # cluster/topic tag(s) +authors: + - dane # from docs/blog/.authors.yml +--- + +# Anvil Policies + +{% set resource = "anvil" %} + +## Software Installation Request Policy +… +## Helpful Tips +… +## Acceptable Purdue IT Research Resource Use +{{ resource_use(resource) }} + +## Scratch File Purging +{{ scratch_purge(resource) }} +``` +Notes: this page carries only `tags` + `authors` (no `resource:`/`search.boost:` — lighter than a +cluster user-guide article). It opens with `{% set resource = … %}` then mixes hand-written +sections with macro-emitted standard blocks. + +**`docs/snippets/resourceuse.md`** (rendered by `resource_use`) is the reusable acceptable-use +text: 4 short paragraphs — (1) use the queuing system / specify cores accurately; (2) comply with +**Purdue IT Acceptable Use Policy V.4.1** and **Remote Access Policy V.1.6** (linked); (3) refer +to **User Credential Standards**; (4) activity is logged/archived/analyzed (modules, apps, perf +counters, disk, job scripts). Line 1 is an HTML comment noting the `resource_use` macro mutates it +per resource (drops the credential-standards line for Anvil). + +**For our agentic-AI acceptable-use page:** mirror this archetype — front matter (`tags` + +`authors`), one H1, hand-written policy/etiquette sections, and reuse `{{ resource_use(resource) }}` +if we want the canonical Purdue AUP block verbatim. Because agentic-AI policy is likely +cross-cluster, either pick a representative `resource` for the AUP block or link Purdue Policy +V.4.1 / V.1.6 directly rather than `{% set %}`-ing a single cluster. + +--- + +## 5. Concrete recommendation (copyable) + +### (a) Where canonical files live +Store the exact harness config + `agents.d` context files as **real files** under a new +`docs/snippets/agentic-ai/` tree (excluded from page build by `exclude_docs: snippets/`, still +`--8<--`-includable, resolves in prod). Suggested layout: + +``` +docs/snippets/agentic-ai/ + claude/settings.json + codex/config.toml + gemini/settings.json + opencode/opencode.json + warp/warp.yaml + agents.d/unix.md + agents.d/filesystems.md + agents.d/lmod.md + agents.d/slurm.md + agents.d/policies.md +``` +These files hold the byte-exact config/context (single source of truth). If they must also match +files deployed to `/etc/agents.d`, keep them here and treat this as canonical (or symlink/copy in +tooling) — but the files themselves stay inside the repo tree (`restrict_base_path: true`). + +### (b) How to render them (builds clean under `--strict`, single source of truth) +On the published page (which lives under `docs/…` and IS in `nav:` per invariant #4), embed each +file inside a fenced code block. No `{% raw %}` needed — the include is post-Jinja: + +````markdown +## Claude Code — `settings.json` + +Drop this at `~/.claude/settings.json`: + +```json title="~/.claude/settings.json" +--8<-- "docs/snippets/agentic-ai/claude/settings.json" +``` + +## `/etc/agents.d/slurm.md` + +```markdown title="/etc/agents.d/slurm.md" +--8<-- "docs/snippets/agentic-ai/agents.d/slurm.md" +``` +```` + +The surrounding page can still freely use macros in its prose, e.g. +`{% set resource = "gautschi" %}` … `{{ slurm_general_overview(resource) }}`. + +Choose the ```` ``` ```` fence to match each file: `json` (Claude/Gemini/opencode), `toml` +(Codex), `yaml` (Warp), `markdown` (agents.d/*). `title="…"` (an `attr_list`/superfences feature, +already enabled) labels the block with the on-disk path. + +### Verification (per CLAUDE.md verify gate) +```bash +mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py # no NEW warnings +mkdocs serve # eyeball: config renders verbatim, no empty blocks (check_paths:false is silent) +``` +Confirm every new published page is added to `mkdocs.yml` `nav:` (snippet files are correctly NOT +in nav). Because `check_paths` is off, a typo'd `--8<--` path fails silently — the visual render +check is the safety net. + +--- + +## Appendix — files cited +- `main.py` (macros; lines cited per macro above) +- `mkdocs.yml:4-5` (exclude_docs), `:94` (macros bare), `:404-425` (extensions), `:416/423` (snippets/superfences) +- `docs/templates.md:32-89` (snippet + macro + `{% raw %}` precedent) +- `docs/userguides/anvil/policies.md` (policy archetype) +- `docs/snippets/resourceuse.md` (reusable AUP text) +- `docs/snippets/examples/apps/mathematica.md:66` + `docs/userguides/hammer/run_jobs/examples/apps/mathematica.md:11` (lone `{% endraw %}` in an unwrapped include — live proof snippets bypass Jinja) +- `Dockerfile.geddes-prod:4,9,13` (prod builds from repo root → snippet base_path resolves) +- Empirical harness: `/tmp/macrotest` (mkdocs 1.6.1 / macros 1.3.9 / pymdownx 10.16.1) diff --git a/spec/agentic-ai-docs/research/03-gautschi-facts.md b/spec/agentic-ai-docs/research/03-gautschi-facts.md new file mode 100644 index 00000000..92096cbb --- /dev/null +++ b/spec/agentic-ai-docs/research/03-gautschi-facts.md @@ -0,0 +1,323 @@ +# Gautschi Cluster Facts Sheet (citable) + +Purpose: exact, citable Gautschi facts for authoring the shared-context files `slurm.md`, +`filesystems.md`, `lmod.md`, `unix.md`, `policies.md`. HPC accuracy is safety-critical — +every fact below carries a `path:line` citation. Anything not verifiable in the source tree +is marked `[UNCONFIRMED]`. + +All paths are relative to repo root +`/Users/geoffrey/Software/github.com/purduercac/RCAC-Docs`. + +IMPORTANT META-NOTE ON WHERE TEXT LIVES: many Gautschi storage/account pages are thin +wrappers that pull real text from `main.py` macros or `docs/snippets/*.md`. Citations below +point at the actual source of the text (the macro/snippet), not just the wrapper page. + +--- + +## 1. SYSTEM + +- **OS:** Rocky Linux 9. "Gautschi nodes run Rocky Linux 9 and use Slurm ... as the batch + scheduler." — `docs/userguides/gautschi/overview.md:40`. (Note: one apptainer example page + shows `CentOS Linux release 7.3.1611` inside a *container* image — that is container OS, not + the host — `docs/userguides/gautschi/run_jobs/apptainer_example.md:45`.) +- **Scheduler:** Slurm — `overview.md:40`, `run_jobs/index.md:14`. +- **CPU compute node (sub-cluster A):** Two AMD EPYC 9654 96-core "Genoa" CPUs @ 2.4 GHz = + **192 cores/node, 384 GB memory**; 400 Gbps NDR InfiniBand; 338 nodes; retires 2030. + — `overview.md:12`, `overview.md:22`, `overview.md:34`. +- **High-memory node (sub-cluster B):** Two AMD EPYC 9654 96-core "Genoa" @ 2.4 GHz = 192 + cores, **1.5 TB memory**; 6 nodes. — `overview.md:35`, `run_jobs/queues.md:98`. +- **Small-GPU node (sub-cluster G):** Two AMD EPYC 9554 64-core "Genoa" @ 3.1 GHz = 128 + cores, 384 GB memory, **Two NVIDIA L40S GPUs (48 GB each)**; 6 nodes. + — `overview.md:36`, `run_jobs/queues.md:131`. +- **GPU/AI node (sub-cluster H):** Two Intel Xeon Platinum 8480+ 56-core CPUs @ 3.8 GHz = 112 + cores, **1 TB CPU memory**, **Eight NVIDIA H100 GPUs (80 GB each)**, 8-way NVLinked; 20 + nodes. — `overview.md:37`, `run_jobs/queues.md:75`. (`overview.md:22` says "1032 GB", + `overview.md:12` says "1031 GB", table `overview.md:37` says "1TB" — minor internal + inconsistency; treat as ~1 TB. `[UNCONFIRMED exact GB]`.) +- **Interconnect:** 400 Gbps NDR InfiniBand on all compute nodes — `overview.md:12`, + `overview.md:22`. +- **Front-end / login nodes:** **8 front-end nodes**, each Two AMD EPYC 9654 96-core "Genoa" + @ 2.4 GHz = 192 cores, **768 GB memory** — `overview.md:24-28`. (Observed login node + hostname `login02` in an scontrol example — `run_jobs/monitoring_job.md:44`.) +- **Login hostname:** `gautschi.rcac.purdue.edu`. SSH form: + `ssh username@gautschi.rcac.purdue.edu` — `main.py:520`, `main.py:528`, `main.py:52` + (`ssh-copy-id ... username@{resource}.rcac.purdue.edu`, resource=gautschi via + `accounts.md:12`). +- **Compute node naming:** `gautschi-a001.rcac.purdue.edu` style; short names `a166`, + `a[014-015]` — `run_jobs/simple_job.md:45`, `run_jobs/monitoring_job.md:46`, + `run_jobs/multiple_node.md:28`, `run_jobs/mpi_jobs.md:69`. +- **Other access endpoints:** ThinLinc remote desktop `desktop.gautschi.rcac.purdue.edu` + (`main.py:181`, `main.py:190-191`, `index.md:19`); Open OnDemand "Gateway" + `gateway.gautschi.rcac.purdue.edu` (`main.py:509`, `index.md:15`). +- **Authentication:** SSH public keys OR Purdue MFA (two-factor). "Gautschi supports either + Purdue's MFA two-factor authentication or SSH keys." — `main.py:515`. Password login: + "enter your Purdue career account password. Your MFA app will then receive a notification to + approve the login." — `main.py:531`; Gateway: "Career account username and Purdue Login MFA + client" — `docs/userguides/gautschi/gateway.md:20`. SSH-key setup is a 3-step + keygen/copy/test flow — `main.py:28-70`. **`[UNCONFIRMED]`: the term "BoilerKey" does not + appear anywhere in the Gautschi docs or `main.py`; the docs say "Purdue Login MFA" / "MFA".** +- **Account eligibility:** must be in a research group that purchased access; external + collaborators need a valid Purdue career account (R4P if none) — `main.py:492-497`. + +## 2. SLURM + +### Account / allocation model +- Every Gautschi job request has **four parts**: resources, **account** (`-A/--account=`), + **QOS** (`-q/--qos=`), and **partition** (`-p/--partition=`) — `run_jobs/queues.md:14-19`. +- **You MUST explicitly specify BOTH the account and the partition.** "you must explicitly + define the account that you want to submit to using the '-A/--account=' option" + (`queues.md:27`); "instead of only needing to specify the account name in the job script, + the desired partition must also be specified." (`queues.md:47`). +- Accounts = queues, named after the purchasing partner; behave like bank accounts of + purchased cores / GPU-hours. Use **`slist`** to list your accounts and their balances — + `queues.md:25`, `run_jobs/mpi_jobs.md:127`, `run_jobs/gpu_usage_monitoring.md:106`. +- "standby"/"highmem"/"gpu" are **no longer accounts** on Gautschi (they were on older + clusters) — they are now QOS/partitions — `queues.md:27`, `queues.md:41`. + +### QOS model +Two QOS classes: **Partition QOS** (auto-applied by the partition) and **Job QOS** (chosen +with `-q/--qos=`) — `queues.md:31-36`. Job QOSes available to every user/account: +- **`normal`** (default if no `-q` given): high priority, no extra limits — `queues.md:40`. +- **`standby`** (opt-in `-q standby`/`--qos=standby`): access to idle resources, **low + priority, max 4 hours**, usage **NOT** charged against the account; replaces the old + `-A standby` style — `queues.md:41`. +- **`preemptible`** (AI partition only, `-q preemptible`): low priority, charged **0.25 + GPU-hour per GPU per hour** (4x cheaper), max 2 weeks, **can be cancelled/preempted** to make + room for `normal` jobs — "imperative to use checkpointing" — `queues.md:87-92`. + +### Partitions (authoritative list from queues.md) +| Partition | Nodes | Node type | Cores/node | Memory model | GPUs | QOSes | Max walltime | Submit | +|---|---|---|---|---|---|---|---|---| +| **`cpu`** | 336 Gautschi-A | AMD EPYC 9654, 192c | 192 | ~2 GB/core requested | — | normal, standby | normal: **2 weeks**; standby: 4 h | `-p cpu` | +| **`ai`** | 20 Gautschi-H | Intel Xeon 8480+, 112c | 112 | ~9 GB/core requested | 8× H100 80 GB/node (160 total) | normal, preemptible | **2 weeks** | `-p ai` | +| **`highmem`** | 6 Gautschi-B | AMD EPYC 9654, 192c | 192 | ~8 GB/core; 1.5 TB/node | — | normal | **24 h** | `-p highmem` | +| **`profiling`** | 2 Gautschi-A | AMD EPYC 9654 (perf counters) | 192 | whole-node only | — | normal | **24 h** | `-p profiling` | +| **`smallgpu`** | 6 Gautschi-G | AMD EPYC 9554, 128c | 128 | ~3 GB/core | 2× L40S 48 GB/node | normal | **24 h** | `-p smallgpu` | + +Citations: cpu `queues.md:49-71`; ai `queues.md:73-94`; highmem `queues.md:96-110`; profiling +`queues.md:112-127`; smallgpu `queues.md:129-142`. + +Partition-specific limits/notes: +- **cpu:** memory ~2 GB per requested core; `normal` withdraws cores from account until job + ends; GPU-only groups may use `cpu` only via `standby` — `queues.md:51-69`. +- **ai:** memory ~9 GB per requested core; `normal` charges **1 GPU-hour per GPU per hour** + (tracked by the minute) — `queues.md:75-85`. +- **highmem:** ~8 GB/core; **no expectation of start time**; max **2 running / 8 submitted** + jobs; **your job must use more than 48 of the 192 cores** — `queues.md:102-108`. +- **profiling:** hardware perf counters (e.g. AMD MicroProf); **whole-node only**; max 1 + running job; monitored to prevent use as plain compute — `queues.md:114-125`. +- **smallgpu:** ~3 GB/core; request cores proportional to GPUs used; up to **2 GPUs**; max 8 + submitted jobs — `queues.md:131-140`. + +### Default limits +- Access sold in units of 48 cores (quarter of a CPU node) or a 5-GPU-year package — + `overview.md:14`. +- **`[UNCONFIRMED]`: no explicit default walltime or default node/core count is stated on any + Gautschi page.** (The generic `main.py` macro `slurm_general_jobs` states "time: 30 minutes, + nodes: 1" at `main.py:335-336`, but that macro is NOT invoked by any Gautschi page — see + §Modules note — so do not attribute it to Gautschi.) + +### Example real `#SBATCH` directives (gold-standard submit script) +From `run_jobs/creating_the_submission_script.md:15-34` (cpu partition): +```bash +#!/bin/bash +# FILENAME: myjobsubmissionfile +#SBATCH --account=myLabAccount +#SBATCH --partition=cpu +#SBATCH --qos=normal +#SBATCH --nodes=1 +#SBATCH --ntasks=1 +#SBATCH --time=1-00:00:00 +module load matlab +cd $SLURM_SUBMIT_DIR +matlab -nodisplay -singleCompThread -r myscript +``` +Combined `-A ... -p ...` on one directive line — `run_jobs/directives.md:26-28`. Full +command-line form: `sbatch -A myLabAccount -p cpu -q normal -N1 -n1 -t 1-00:00:00 file` +— `run_jobs/submit_script.md:28`. + +### Slurm / cluster commands referenced in Gautschi docs (verified present) +- **`sbatch`** — submit batch job — `submit_script.md:16`, `directives.md:16`. +- **`squeue -u `** — view your jobs — `monitoring_job.md:15,21`. +- **`scontrol show job `** — job detail — `monitoring_job.md:31`. +- **`scontrol hold job ` / `scontrol release job `** — hold/release — + `docs/snippets/holding_job.md` (included by `run_jobs/holding_job.md:11`). +- **`scancel `** — cancel job — `docs/snippets/cancelling_job.md:6` (included by + `run_jobs/cancelling_job.md:11`). +- **`sinteractive -A -p -N.. -n..`** — interactive job — `interactive_jobs.md:15-21`. +- **`srun` / `mpiexec`** — launch MPI ranks; `srun --mpi=pmi2` for Intel MPI — + `mpi_jobs.md:35,44,47`, `monitoring_resources.md:53,62`. +- **`slist`** — list accounts / balances (incl. GPU-hour balance) — `queues.md:25`, + `mpi_jobs.md:127`, `gpu_usage_monitoring.md:106`, `gateway/jobs.md:17`, + `gaussian_example.md:110`. +- **`sfeatures`** — node/GPU feature detail — `gpu_jobs.md:92`. +- **`myquota`** — home/scratch quota — `docs/snippets/storage_quota.md:8`. +- **`findscratch`** — print scratch path — `docs/snippets/scratch_space.md:15`. +- **`purgelist`** — files scheduled for scratch purge — `docs/snippets/scratchpurge.md:21-24`. +- **`monitor`** (module) — resource telemetry — `monitoring_resources.md:21-25`. +- **`[UNCONFIRMED]`: `sinfo`, `sacct`, `jobinfo`, `mybalance`, `showpartitions`, `wait_time` + do NOT appear anywhere under `docs/userguides/gautschi/`** (grep-verified). `mybalance`, + `showpartitions`, `wait_time`, and the `wholenode` partition appear only in the generic + `main.py` macro `slurm_general_jobs` (`main.py:301-391`), which Gautschi does not use — do + not present them as Gautschi commands. + +### KNOWN DOC ERRORS to NOT propagate (copy-paste leftovers from Gilbreth) +- `run_jobs/gpu_jobs.md:41,49-51` and `run_jobs/mpi_jobs.md:32` use **`--partition=a10`**, + which is **not a Gautschi partition** (valid GPU partitions are `ai` and `smallgpu`). + Grep count: 7 uses of `--partition=a10` under gautschi. **Do not use `a10` in generated docs.** +- `run_jobs/mpi_jobs.md:94` uses **`#SBATCH -A standby`** — invalid on Gautschi (standby is a + QOS `-q standby`, not an account). — contradicts `queues.md:41`. + +## 3. MODULES (Lmod) + +- Gautschi's `software.md` demonstrates module use directly with **`module avail`**, + **`module spider `**, **`module load /`** — `software.md:23-56`. + (`module spider` is an Lmod-specific command, so the module system is **Lmod**; note the + word "Lmod" itself is only in the `module_system` macro, which Gautschi does **not** invoke — + see below. `[UNCONFIRMED that a Gautschi page literally names "Lmod"]`.) +- **Default compiler + MPI (recommended):** **GCC 14.1.0** and **OpenMPI** — + `overview.md:42-45`. `[UNCONFIRMED: exact OpenMPI version not stated on any Gautschi page.]` +- Example load lines: `module load hypershell/2.7.0` (`software.md:55`), `module load cuda` + (`gpu_jobs.md:26`), `module load matlab` (`creating_the_submission_script.md:27`), + `module load monitor` (`monitoring_resources.md:24`). +- Core Applications sample (`module avail`) includes `amduprof/5.1.701`, + `anaconda/2024.10-py312`, `anaconda/2025.06-py313 (D)`, `hypershell/*`, `ngc/default`, + `oclfpga/2024.1.0`, `openblas/0.3.27` — `software.md:25-29`. +- **Do NOT attribute the generic Lmod write-up to Gautschi:** the `module_system(resource)` + macro (`main.py:540-632`, describes hierarchy, `module unload/purge/show/help`, + `$RCAC_APPNAME_ROOT`, auto compiler/MPI/CUDA loading) is invoked by anvil/bell/gilbreth/ + hammer/negishi software pages **but not by Gautschi** (grep-verified: `module_system` usage + is in `docs/userguides/{anvil,bell,gilbreth,hammer,negishi}` only). Those extra module + commands are RCAC-standard and almost certainly apply, but are `[UNCONFIRMED for Gautschi + specifically]` from the Gautschi page set. General module commands (`module unload`, + `module purge`, `module list`, `module show`) do appear in Gautschi job-script examples/prose + via `main.py:360-363` only if a Gautschi page calls that macro — it does not; however + `module purge`/`module list` appear as generic script comments. Verified in-Gautschi: + `module avail`, `module spider`, `module load` (software.md), `module avail openmpi` + (`mpi_jobs.md:18`). + +## 4. FILESYSTEMS + +Text lives in `main.py` macros / `docs/snippets/`; wrapper pages just include them. + +- **Home directory:** path **`/home/myusername`**, env var **`$HOME`**. On GPFS. Long-term, + medium-performance, **not** for heavy-I/O jobs. — `docs/snippets/home_directory.md:8-13,29-34` + (included via `storage/home_directory.md:11`). Snapshots: **nightly for 7 days, weekly for 4 + weeks, monthly for 3 months** — `home_directory.md:27`. +- **Home quota:** the `myquota` example shows a **25.0 GB** home limit + (`docs/snippets/storage_quota.md:14-15`), but this is illustrative example output in a shared + snippet, and home quota "cannot be increased beyond current level" + (`storage_quota.md:54`). **`[UNCONFIRMED: 25 GB is an example, not a Gautschi-stated + official quota.]`** +- **Scratch:** path **`/scratch/gautschi/myusername`** (`{cluster}` = `gautschi`), env var + **`$RCAC_SCRATCH`** (NOT `$CLUSTER_SCRATCH`). Find it with **`findscratch`**. High- + performance parallel filesystem; not backed up; not recoverable. — `docs/snippets/ + scratch_space.md:12-27` (via `storage/scratch_space.md:11`), `docs/snippets/ + environment_variables.md:12,28,37`. **CORRECTION to task hypothesis: the variable is + `$RCAC_SCRATCH`, and there is no `$CLUSTER_SCRATCH` in these docs.** +- **Scratch purge policy:** **files not accessed or content-modified in 60 days are purged** + (Gautschi uses the default 60-day threshold; only Anvil/Bell are rewritten to 30 days by + `main.py:755-757`). — `docs/snippets/scratch_space.md:10`, `docs/snippets/scratchpurge.md:10`, + `scratchpurge.md:67`. Purge is by last access + content-modification time; metadata changes + don't protect a file; no warning emails; use `purgelist` to see candidates — + `scratchpurge.md:8-19,21-33`. +- **Scratch quota:** `myquota` example shows scratch **100.0 TB / 2,000k files** + (`storage_quota.md:15`) — illustrative example output, not a Gautschi-official figure. + **`[UNCONFIRMED exact Gautschi scratch quota.]`** Quota increases available on request + (`storage_quota.md:58`). +- **/tmp:** node-local short-term storage on each front-end/compute node; data lives only + while the program runs; not backed up; not recoverable — `docs/snippets/tmp_directory.md:4-6` + (via `storage/tmp_directory.md:11`). +- **Data Depot:** long-term storage for active research data (purchasable); listed as an + archival target for staging scratch data — `docs/snippets/scratchpurge.md:6,47,51`; + purchasable via `storage_quota.md:54`. `[UNCONFIRMED: no Gautschi page states the Depot mount + path or per-space quota; Depot path is typically `/depot/` on RCAC but not cited here.]` +- **Fortress (long-term / archive):** HPSS archival system, **>10 PB** capacity; use `hsi`/ + `htar`. — `docs/snippets/long_term_storage.md:4-9` (via `storage/long_term_storage.md:11`), + `scratch_space.md:4`. +- **Quota-check command:** **`myquota`** (reports home + scratch, size and file-count) — + `docs/snippets/storage_quota.md:8-16`. +- **Storage environment variables:** `HOME` = `/home/myusername`; `PWD`; `RCAC_SCRATCH` = + `/scratch/gautschi/myusername` — `docs/snippets/environment_variables.md:10-12`. +- **Slurm/runtime env vars documented for Gautschi:** `SLURM_SUBMIT_DIR` (used in gold + submit script, `creating_the_submission_script.md:30`), `SLURM_JOB_NODELIST` + (`multiple_node.md:20`), `CUDA_VISIBLE_DEVICES` (`gpu_jobs.md:30`). The fuller table + (`SLURM_SUBMIT_DIR`, `SLURM_JOBID`, `SLURM_JOB_NAME`, `SLURM_JOB_NODELIST`, + `SLURM_SUBMIT_HOST`, `SLURM_JOB_PARTITION`) exists in the generic macro `main.py:310-317` + but that macro is not used by Gautschi — the individual vars above ARE used in Gautschi pages. + +## 5. POLICIES + +- **Login-node "do not run compute" wording (quote it):** + > "Do NOT run large, long, multi-threaded, parallel, or CPU-intensive jobs on a front-end + > login host. All users share the front-end hosts, and running anything but the smallest + > test job will negatively impact everyone's ability to use Gautschi. Always use SLURM to + > submit your work as a job." + — `docs/userguides/gautschi/run_jobs/index.md:16-17` (an `!!! important` admonition). +- **Front-end use policy (external link):** "The normal acceptable front-end use policy ... + applies" → — + `docs/userguides/gautschi/gateway/cluster_tools.md:18`. +- **Scratch purge policy (external link):** + — `docs/snippets/scratch_space.md:10`. +- **Acceptable Use:** scratch snippet states using scratch as long-term storage or + circumventing purging "is considered a violation of Acceptable Research Resource Use" + — `docs/snippets/scratchpurge.md:71-73` (anchors to + `#acceptable-purdue-it-research-resource-use`). `[UNCONFIRMED: no direct link to a Purdue IT + AUP URL is present on the Gautschi pages read; the reference is an in-page anchor.]` +- **Data-sensitivity guidance:** `[UNCONFIRMED]` — no explicit data-classification/sensitivity + (e.g. HIPAA/export-control/regulated-data) statement found in the Gautschi page set read. +- **Network-drive access requires campus network/VPN** (CIFS "Z: drive") — + `docs/userguides/gautschi/faqs.md:29`. +- **Data-retention / access removal:** losing Gautschi access removes mailing-list + subscription; copy data out first — `faqs.md:18`. +- **Cluster lifetime:** hardware "service through 2030" / "Retires in 2030" — `overview.md:12`, + `overview.md:28,34-37`. + +## 6. UNIX / GENERAL + +- **Default shell:** `[UNCONFIRMED]` — no Gautschi page states the default login shell. + Submission-script examples use `#!/bin/bash` (e.g. `simple_job.md:18`, + `creating_the_submission_script.md:16`) and some generic examples `#!/bin/sh -l` + (`main.py:277`). The env-var snippet documents both bash (`export`) and tcsh/csh (`setenv`) + syntax — `docs/snippets/environment_variables.md:41-51`, implying csh/tcsh users exist. +- **Home is CWD at login:** "Your home directory becomes your current working directory, by + default, when you log in." — `docs/snippets/home_directory.md:4`. +- **Gautschi home is isolated:** the Gautschi home directory is exclusive to Gautschi front- + ends/compute nodes; not shared with or synced to other RCAC clusters — `faqs.md:32`. +- **Scratch is per-cluster:** only `/scratch/gautschi` is mounted on Gautschi; no other + cluster's scratch is available — `docs/snippets/scratch_space.md:26-27`. +- **Env-var best practice:** use `$HOME`, `$RCAC_SCRATCH`, etc. instead of hard-coded paths + (paths may change) — `docs/snippets/environment_variables.md:4`, + `docs/snippets/scratch_space.md:19`. +- **Module gotcha:** the generic guidance "do not put `module load` in your bash profile (can + break ThinLinc init)" lives in the `module_system` macro (`main.py:587`) which Gautschi does + not invoke — `[UNCONFIRMED for Gautschi]` but standard RCAC advice. +- **GUI apps:** launch via ThinLinc (Applications → Cluster Software) — `faqs.md:38-41`. +- **X11 forwarding:** supported via `ssh -Y`; `$DISPLAY` must be left as SSH sets it — + `main.py:159,162`. +- **App-specific gotchas documented:** Firefox lock ("Firefox is already running") and Jupyter + "database is locked" recovery — `faqs.md:76-82` (snippets `firefox_lock.md`, `jupyter_lock.md`). + +--- + +## Summary of `[UNCONFIRMED]` items +1. Exact GPU-node CPU memory (1 TB vs 1031 vs 1032 GB — internal inconsistency). +2. "BoilerKey" — term never used; docs say "Purdue Login MFA" / SSH keys. +3. Default Slurm walltime / default nodes-cores for Gautschi (generic macro value not + attributable to Gautschi). +4. `sinfo`, `sacct`, `jobinfo`, `mybalance`, `showpartitions`, `wait_time`, `wholenode` — not + in Gautschi docs. +5. Literal "Lmod" naming and full module command set (`unload`/`purge`/`show`/`help`) not on a + Gautschi page (module_system macro unused by Gautschi). +6. Exact OpenMPI version (only "OpenMPI" + "GCC 14.1.0" stated). +7. Official home quota (25 GB) and scratch quota (100 TB / 2000k files) — shown only as + illustrative `myquota` example output in a shared snippet. +8. Data Depot mount path / quota; Purdue IT AUP direct URL; data-sensitivity/classification + policy; default login shell. + +## Documentation-error warnings (do NOT propagate into generated shared context) +- `--partition=a10` in `gpu_jobs.md` and `mpi_jobs.md` — invalid Gautschi partition (use `ai` + or `smallgpu`). +- `#SBATCH -A standby` in `mpi_jobs.md:94` — `standby` is a QOS (`-q standby`), not an account. + + diff --git a/spec/agentic-ai-docs/research/04-mcp-servers.md b/spec/agentic-ai-docs/research/04-mcp-servers.md new file mode 100644 index 00000000..ae655666 --- /dev/null +++ b/spec/agentic-ai-docs/research/04-mcp-servers.md @@ -0,0 +1,354 @@ +# Research: The three public RCAC MCP servers + +Source of truth: the GitHub repos themselves (README, SECURITY.md, ROADMAP.md, +`pyproject.toml`, and selected source files), fetched via `gh` on 2026-07-14. +Anything not stated by the source is marked `[UNVERIFIED]`. Commands/config in +fenced blocks are copied verbatim from the repos. + +Common facts across all three: + +- All three are **Python** projects built with `fastmcp` (FastMCP `>=2.0.0,<3`) + and `cmdkit`, packaged with `hatchling`, version `0.1.0`, source under + `src//`, console entry point ` = ":main"`. +- All three are **installed/run with `uvx` straight from Git** (no PyPI release, + no npm, no required Docker for the desktop path). `uvx git+https://…` clones, + builds, and runs in one step. +- **License: MIT** — declared in each README and in SPDX source headers + (`SPDX-License-Identifier: MIT`). Note: `rcac-mcp` and `rcac-docs-mcp` have **no + standalone `LICENSE` file** at repo root (GitHub's license API returns 404); + `globus-mcp` is detected as MIT by GitHub. +- None are archived. Only `globus-mcp` has a tagged release (`v0.1.0`). + +--- + +## 1. `rcac-mcp` — PurdueRCAC/rcac-mcp + +Repo: +Description: "Prototype MCP server implementation enabling agentic workflows to +connect with RCAC systems." Default branch `main`; also a `wip` branch. +Last pushed 2026-02-12. + +### 1. Purpose +The RCAC MCP Server "enables agentic development with HPC clusters and storage +services." It gives an AI agent/harness a set of tools to operate on an RCAC HPC +cluster — run shell commands, move/read/write files, inspect storage quota and +paths, and drive Slurm (submit/monitor/cancel jobs) — plus a local full-text +documentation search. In the default desktop mode it runs **locally as a stdio +subprocess** and executes everything **on the cluster over the user's existing +SSH**, so no new service or credential is introduced. + +### 2. Language / package / install +- Language: Python (`requires-python >=3.14`), package name `rcac-mcp`, entry + point `rcac-mcp`. +- Deps: `fastmcp`, `cmdkit`, `pyjwt`, `fabric` (SSH), `pyyaml`, `jinja2`. +- Install/run (desktop, via `uvx` from Git — no separate install step needed): + the MCP client launches `uvx git+https://github.com/purduercac/rcac-mcp`. +- License: MIT (README + SPDX headers; no root `LICENSE` file). + +### 3. How to connect it to a harness +**stdio subprocess** (default) for Claude Desktop / Cursor / Warp. Exact +`mcpServers` block from the README: + +```json +{ + "mcpServers": { + "rcac": { + "command": "uvx", + "args": ["git+https://github.com/purduercac/rcac-mcp", "--ssh-host", "cluster.rcac.purdue.edu"] + } + } +} +``` + +"This runs the server locally in `stdio` mode, executing commands on the cluster +over SSH using your existing `~/.ssh/config` and keys." You may instead set +`RCAC_SSH_HOST` in the environment and omit `--ssh-host`. + +Optional **hosted HTTP** mode (for a shared, authenticated instance): + +```bash +rcac-mcp -t http --ssh-host cluster.rcac.purdue.edu +``` + +With JWT auth + delegate execution, and client-token generation: + +```bash +export JWT_SECRET="your-secret-key-at-least-32-characters" +rcac-mcp -t http -a jwt -e delegate + +# Generate a token for clients +rcac-mcp --generate-token --lifetime 86400 +``` + +Docker Compose + TLS is documented for production-like deploys (`docker compose +up`, served at `https://mcp.rcac.dev:8443`). +`[UNVERIFIED]` the README does not give a `claude mcp add …` or `codex mcp …` +one-liner — only the `mcpServers` JSON block and the HTTP CLI invocations above. + +### 4. Tools exposed (verbatim from README "Available Tools") +Shell & Filesystem: +- `run_command(command, cwd, timeout)`, `list_directory(path, show_hidden)`, + `read_file(path, encoding, max_size)`, `write_file(path, content, append, + create_dirs)`, `upload_file(local_path, remote_path)`, + `download_file(remote_path, local_path)` + +RCAC Cluster: +- `myquota()`, `storage_paths()`, `jobinfo(job_id)`, `jobcmd(job_id)`, + `jobenv(job_id)`, `jobscript(job_id)`, `showpartitions()`, + `average_wait(partition, account)` + +Slurm: +- `sbatch(...)`, `squeue(...)`, `scancel(...)`, `sacct(...)`, `sinfo(...)`, + `scontrol_show_job(job_id)`, `scontrol_show_node(node)`, `slist()` (accounts & + usage, RCAC-specific), `sfeatures()` (node hardware features, RCAC-specific) + +Documentation: +- `doc_search(query, category)` — full-text search over RCAC docs +- `doc_load(path)` — load a full documentation page + +Build the local docs index first (FTS5, from a local RCAC-Docs clone): + +```bash +rcac-mcp --index-docs --docs-path /path/to/RCAC-Docs +``` + +DB defaults to `~/.config/rcac-mcp/docs.db` (override with `--docs-output` or +`RCAC_DOCS_DB`); re-running is incremental. + +### 5. Architecture — local-first, confirmed +SECURITY.md defines three execution modes: + +| Mode | Transport | Auth | Use | +|------|-----------|------|-----| +| `ssh` | stdio | none (single-user) | local MCP client → remote HPC cluster (default) | +| `local` | http/sse | none | dev/testing only, runs on the server's own `$SHELL` | +| `delegate` | http/sse | JWT/OIDC | multi-user prod; server runs as root, `sudo -u ` per request | + +Local-first model **confirmed**: in the default `ssh`/stdio mode the server runs +on the user's machine, uses the user's existing `~/.ssh/config` + keys, adds **no +MCP-layer auth and no new credential**, and all commands run as the SSH user on +the cluster ("Security boundary is the SSH connection itself"). Delegate mode +isolates concurrent users with `contextvars.ContextVar` and maps token identity +to a local Unix user via `RCAC_USER_MAP` (`/etc/rcac-mcp/users.map`), wrapping +commands as `sudo -n -H -u `. + +**`/etc/agents.d` context injection — confirmed.** `src/rcac_mcp/resources.py` +implements a `cluster_context` MCP resource that, over the same SSH executor, +runs `find /etc/agents.d -maxdepth 1 -name "*.md" -type f | sort`, `cat`s each +file, concatenates them (with `` headers), and caches the +result per hostname. `server.py` exposes it as the resource +`rcac://context` ("Cluster-specific context loaded from /etc/agents.d/*.md"). +So the cluster can drop admin-authored markdown into `/etc/agents.d/` and the +server surfaces it to the agent as read-only context. + +**Plugin / `cluster-mcp[slurm,lmod,…]` direction — `[UNVERIFIED]`.** No public +artifact in this repo (README, ROADMAP.md, source on `main` or `wip`) mentions +`cluster-mcp`, a plugin/extras architecture, or an "HPC-only" refocus. An +org-wide `gh search code --owner PurdueRCAC "cluster-mcp"` returned **zero +hits**. As of the last push (2026-02-12) the repo still bundles the docs-search +tools (`doc_search`/`doc_load`), i.e. it has **not** yet been split into +HPC-only + a plugin catalog. Treat the plugin/`cluster-mcp` framing as roadmap +intent not yet reflected in the public code. (Note the docs-search capability was +subsequently extracted into the separate `rcac-docs-mcp` repo, pushed +2026-06-26 — see §3 below.) + +### 6. Status / caveats +Repo description and README both call it a **prototype**. Version `0.1.0`, **no +tagged release**. ROADMAP.md is actually the completed plan for the docs-search +feature (status `complete`, `local-to-mcp-process`), not a product roadmap. +`local` mode is explicitly "dev/testing only, NEVER expose to a network without +auth." A live `wip` branch exists. + +--- + +## 2. `globus-mcp` — PurdueRCAC/globus-mcp + +Repo: . Last pushed 2026-04-12; +tagged release `v0.1.0` "Auth, Consent, and Globus Compute." + +### 1. Purpose +"Globus MCP Server gives AI agents federated data transfer and remote code +execution across research storage systems at institutions worldwide." It wraps +the **Globus CLI** (data transfer) and the **Globus Compute SDK** (remote Python +execution on HPC endpoints), so an agent can search endpoints, browse remote +filesystems, run async transfers, and submit Python functions to Compute +endpoints. + +### 2. Language / package / install +- Language: Python (`requires-python >=3.11,<3.14`), package name `globus-mcp`, + entry point `globus-mcp`. +- Deps: `fastmcp`, `cmdkit`, `globus-compute-sdk>=4.0.0,<5` (Globus CLI wrapped + at runtime). +- Optional prerequisite: **Globus Connect Personal** for transfers to/from the + local machine. +- Install/run: `uvx` from Git (handled automatically by the MCP client). +- License: MIT. + +### 3. How to connect it to a harness +**stdio subprocess** for Claude Desktop / Cursor / Warp. Exact `mcpServers` block +from the README: + +```json +{ + "mcpServers": { + "globus": { + "command": "uvx", + "args": ["git+https://github.com/purduercac/globus-mcp"] + } + } +} +``` + +"The `uvx` invocation handles installation automatically. On first use, the +server's `globus_login()` and `compute_login()` tools will walk users through +authentication via the browser." Auth is **interactive Globus OAuth**, done +through tool calls at runtime (not config). `[UNVERIFIED]` no `claude mcp add`, +`codex mcp`, or hosted-HTTP endpoint is documented — this server is +stdio/local-only in the README, plus `globus-mcp` for dev. + +### 4. Tools exposed (verbatim from README "Available Tools") +Identity & Auth: `whoami()`, `globus_login()`, `session_consent(scopes)` +Endpoints: `endpoint_search(query)`, `endpoint_show(endpoint_id)`, +`endpoint_local_id()` +Filesystem: `ls(endpoint_id, path)`, `stat(endpoint_id, path)`, +`mkdir(endpoint_id, path)`, `rename(endpoint_id, source_path, dest_path)`, +`rm(endpoint_id, path)` (sync), `delete(endpoint_id, path)` (async) +Transfers: `transfer(...)`, `transfer_batch(...)`, `task_list()`, +`task_show(task_id)`, `task_wait(task_id)`, `task_cancel(task_id)`, +`task_event_list(task_id)` +Compute (remote code execution): `compute_login()`, `compute_endpoint_list()`, +`compute_endpoint_status(endpoint_id)`, `compute_submit(...)`, +`compute_batch_submit(...)`, `compute_status(task_ids)`, `compute_result(task_id)` + +### 5. Architecture — local-first, confirmed +Runs as a **local stdio subprocess** of the harness. It uses the **user's own +Globus identity**: auth is obtained interactively via browser OAuth through +`globus_login()`/`compute_login()`, and endpoint-specific access is unlocked via +`session_consent(scopes)` (the server returns a structured `ConsentRequired` +error carrying the needed scopes, the agent grants consent, then retries). For +Compute with a `requirements` list, the server auto-provisions a cached `uv` +virtualenv on the remote endpoint. No new RCAC-hosted service or credential store +is introduced — it rides on the user's Globus auth and local Globus tooling. + +### 6. Status / caveats +Prominent README banner: **"⚠️ Beta Software … has not reached a stable v1.0.0 +release. APIs, tool signatures, and behavior may change without notice. Use with +caution in production workflows."** Version `0.1.0`; the one tagged release is +`v0.1.0`. + +--- + +## 3. `rcac-docs-mcp` — PurdueRCAC/rcac-docs-mcp + +Repo: . Last pushed 2026-06-26. +Description: "MCP service for RCAC Documentation site with full-text search and +load tools." + +### 1. Purpose +"A single-purpose FastMCP server that exposes Purdue RCAC's documentation to AI +agents via full-text search." It indexes the RCAC-Docs content (user guides, +software catalog, datasets, blog posts, workshops) into an FTS5/BM25 SQLite index +and exposes exactly two tools so agents ground their advice in current +authoritative docs rather than general knowledge. It runs **unauthenticated** and +is **hosted at `docs.rcac.purdue.edu/mcp`**. + +### 2. Language / package / install +- Language: Python (`requires-python >=3.14`), package name `rcac-docs-mcp`, + entry point `rcac-docs-mcp`. +- Deps: `fastmcp`, `cmdkit`, `pyyaml`, `jinja2`. +- Install/run (local): `uvx` from Git. +- License: MIT (README + no root `LICENSE` file, so GitHub license API 404s). + +### 3. How to connect it to a harness +Two supported paths. + +**(a) Hosted remote HTTP endpoint (recommended for most users).** A shared, +no-auth instance is hosted at **`docs.rcac.purdue.edu/mcp`**. Per the README: +"Point an HTTP-capable MCP client at that URL — no token or credentials are +required." `[UNVERIFIED]` the README gives the URL but **no** literal client +config JSON or `claude mcp add --transport http …` line for the hosted endpoint; +use your harness's standard "add remote/HTTP MCP server" flow pointed at +`https://docs.rcac.purdue.edu/mcp`. + +**(b) Local stdio subprocess.** Exact `mcpServers` block from the README: + +```json +{ + "mcpServers": { + "rcac-docs": { + "command": "uvx", + "args": ["git+https://github.com/PurdueRCAC/rcac-docs-mcp"] + } + } +} +``` + +The local server needs a search index before it can answer; build it first (see +§5). Transports and serve commands (verbatim): + +```bash +rcac-docs-mcp # serve over stdio (local clients) +rcac-docs-mcp -t http -H 0.0.0.0 # serve over HTTP (hosted) +``` + +(`rcac-docs-mcp -t http` alone serves streamable HTTP on `localhost:8000` for +dev.) + +### 4. Tools exposed (verbatim — "exactly two tools") +- `doc_search(query, category=None)` — FTS5 / BM25 full-text search over indexed + RCAC documentation. Returns up to 20 BM25-ranked results with path, title, + heading, and a matching snippet. Supports `OR`, quoted phrases, prefix + wildcards (`contai*`); `category` is a path-prefix filter (`userguides`, + `software`, `datasets`, `blog`, `workshops`). +- `doc_load(path)` — return the full rendered markdown of one document by its + relative path (as shown in `doc_search` results). + +### 5. Architecture — hosted HTTP + local-first, confirmed +Runs unauthenticated over two transports: **`stdio`** (default, local clients) +and **`http`** (streamable HTTP, for the hosted deployment). Unlike `rcac-mcp` +and `globus-mcp`, this server **does not SSH anywhere and needs no user +credentials** — it only reads a local SQLite index, so the hosted instance can be +fully public/no-auth. + +A "**site**" is one directory holding both a clone of RCAC-Docs (`repo/`) and the +index (`index.db`); resolved from `--site` → `RCAC_DOCS_SITE` → default +`~/.local/share/rcac-docs-mcp`. Env vars: `RCAC_DOCS_SITE`, `RCAC_DOCS_URL` +(default `https://github.com/PurdueRCAC/RCAC-Docs`), `MCP_BASE_URL` (public URL +for absolute icon links). Build the index (two-step operator flow): + +```bash +rcac-docs-mcp --update-site # clone or git-pull /repo +rcac-docs-mcp --index # build/refresh /index.db from /repo +``` + +`--update-site` does `git pull --rebase --autostash origin main`; `--index` is +incremental (SHA-256 hashing, stale-doc pruning). The indexing pipeline walks +`docs/`, parses YAML frontmatter, resolves `--8<--` snippet includes, renders +Jinja2 macros/templates from the docs repo's `main.py` + `mkdocs.yml extra:`, +strips the `` blog marker, chunks on `##` (H2) boundaries, and +upserts into SQLite; search uses an FTS5 virtual table with BM25 + `snippet()`. +This is the same doc-search capability that lives inside `rcac-mcp`, factored out +into its own hostable service. + +### 6. Status / caveats +Version `0.1.0`, no tagged release; single-purpose and intentionally minimal +("exactly two tools"). No explicit "beta" banner, but pre-1.0. When run locally +without a built index, the tools return a helpful message explaining how to build +one. Many integration tests depend on a git-submodule fixture +(`tests/fixtures/RCAC-Docs`) that skips cleanly when uninitialized. + +--- + +## Org scan — other agentic/MCP-related PurdueRCAC repos +From `gh repo list PurdueRCAC --limit 200`: + +- **`agentbase`** (private, updated 2026-05-19) — "Core framework for agentic + content search over internal knowledge bases." The most relevant adjacent + project; private, so contents `[UNVERIFIED]`. Likely the internal-KB analogue + of the public docs-search MCP work. +- **`genai-studio`** (public) — "Reference configurations for Purdue GenAI + Studio." AI-adjacent but not MCP. +- **`CANARI`** (public) — "CANARI framework developed at RCAC" — unclear scope, + possibly AI-related; `[UNVERIFIED]`. +- No other repos reference MCP or `cluster-mcp`. The three servers above are the + only public MCP servers in the org. diff --git a/spec/agentic-ai-docs/research/05-harness-configs.md b/spec/agentic-ai-docs/research/05-harness-configs.md new file mode 100644 index 00000000..d0f5e74d --- /dev/null +++ b/spec/agentic-ai-docs/research/05-harness-configs.md @@ -0,0 +1,415 @@ +# Agentic Coding Harness Configuration on HPC — Per-Harness Reference + +Research date: 2026-07-14. External/current-docs research. Config keys verified against the +cited official docs; anything not confirmed there is marked `[UNVERIFIED]`. + +Scope: for each of the five harnesses — config file paths + format, permissions/sandbox +(including a DENY of destructive ops and login-node lockdown), MCP wiring (stdio + remote), +context-file convention (and AGENTS.md support), and install/run on a Linux login node. + +--- + +## Comparison table + +| harness | config path | format | permission mechanism | MCP config | context-file name | +|---|---|---|---|---|---| +| **Claude Code** | `~/.claude/settings.json` (user), `.claude/settings.json` + `.claude/settings.local.json` (project), `/etc/claude-code/managed-settings.json` (managed, Linux) | JSON | `permissions.allow` / `permissions.deny` / `permissions.ask` arrays; `defaultMode`; OS sandbox (`sandbox.*`) | `claude mcp add …` / `.mcp.json` (project) / `~/.claude.json` (user/local); stdio + http/sse/ws | `CLAUDE.md` (does **not** read AGENTS.md natively) | +| **Codex CLI** | `~/.codex/config.toml` (user), `.codex/config.toml` (project), profile files `~/.codex/.config.toml` | TOML | `approval_policy` + `sandbox_mode` (read-only / workspace-write / danger-full-access); OS sandbox | `[mcp_servers.NAME]` in config.toml, or `codex mcp add`; stdio + streamable HTTP | `AGENTS.md` (native standard) | +| **Gemini CLI** | `/etc/gemini-cli/settings.json` (system), `~/.gemini/settings.json` (user), `.gemini/settings.json` (project) | JSON | `tools.core` (allow) / `tools.exclude` (deny) with `run_shell_command()`; `approvalMode`; sandbox (`tools.sandbox`) | `mcpServers` in settings.json; stdio + SSE (`url`) + streamable HTTP (`httpUrl`) | `GEMINI.md`; configurable via `context.fileName` (can include `AGENTS.md`) | +| **opencode** | `~/.config/opencode/opencode.json` (global), `opencode.json[c]` (project), `/etc/opencode/` (system) | JSON / JSONC | `permission` object; per-tool + per-bash-pattern map `{ "rm *": "deny" }` (allow/ask/deny) | `mcp` object; `type:"local"` (stdio) + `type:"remote"` (http) | `AGENTS.md` (native; `CLAUDE.md` fallback) + `instructions` array | +| **Warp** | Desktop-app UI: Settings > Agents > Profiles / MCP servers; rules in Warp Drive (cloud) | UI + JSON (MCP) | Agent Profiles: autonomy (Agent Decides / Always Ask / Always Allow) + regex allowlist/**denylist** | Settings > Agents > MCP servers, JSON (`mcpServers`); stdio (`command`) + remote (`url`) | `AGENTS.md` (ALL-CAPS required; `WARP.md` legacy) + Global Rules in Warp Drive | + +**Biggest HPC caveat up front:** Warp is a *locally-installed desktop terminal app* (macOS/ +Linux/Windows) — you cannot install it on a headless login node. You run Warp on your +workstation and SSH into the cluster. The other four are true CLIs that run headless over SSH. + +--- + +## 1. Claude Code (Anthropic CLI) + +Docs: · +· · · + + +### Config files (JSON) +| Scope | Path | +|---|---| +| User | `~/.claude/settings.json` | +| Project (shared, committed) | `.claude/settings.json` | +| Project local (gitignored) | `.claude/settings.local.json` | +| Managed (Linux/WSL) | `/etc/claude-code/managed-settings.json` (+ drop-in dir `/etc/claude-code/managed-settings.d/*.json`) | +| Managed CLAUDE.md (Linux) | `/etc/claude-code/CLAUDE.md` | + +Precedence (high→low): managed → CLI args → local → project → user. **Permission rules merge +across scopes; a deny at any scope wins.** Schema: `"$schema": "https://json.schemastore.org/claude-code-settings.json"`. + +### Permissions / sandbox +`permissions.allow` / `permissions.deny` / `permissions.ask` arrays. Evaluation order is +**deny → ask → allow** (first match wins; a broad deny cannot carry allowlist exceptions). +Rule syntax: `Tool` or `Tool(specifier)`. +- Bash: `Bash(rm -rf:*)` or `Bash(rm -rf *)` (the `:*` suffix == trailing ` *`). Claude Code + is shell-operator-aware, so `Bash(safe *)` does **not** authorize `safe && evil`; each + subcommand must match. A bare `Bash` deny removes the tool entirely. +- Files: `Read(...)` / `Edit(...)` use gitignore semantics; `//abs`, `~/home`, `/settings-relative`, `path`. +- Web: `WebFetch(domain:example.com)`. MCP: `mcp__server__tool`. + +`defaultMode` (in settings) selects a permission mode: `default` (prompt), `acceptEdits`, +`plan` (read-only exploration), `auto`, `dontAsk` (auto-deny unless pre-allowed), +`bypassPermissions`. Lock down bypass with `permissions.disableBypassPermissionsMode: "disable"` +and `permissions.disableAutoMode: "disable"` (best in managed settings). `rm -rf /` / `rm -rf ~` +prompt even in bypass mode as a circuit breaker. + +OS-level **sandbox** (Bash-only, defense-in-depth): `sandbox.enabled`, `sandbox.network` +(`allowedDomains` / `deniedDomains`), `sandbox.filesystem` (`allowRead` / `denyRead`), +`autoAllowBashIfSandboxed` (default true). On Linux it uses namespace-based isolation +(bubblewrap-style) — may be restricted on hardened login nodes; deny rules still apply without it. + +**Minimal login-node config** (`~/.claude/settings.json` or a managed file): +```json +{ + "$schema": "https://json.schemastore.org/claude-code-settings.json", + "permissions": { + "defaultMode": "default", + "deny": [ + "Bash(rm -rf:*)", + "Bash(sudo:*)", + "Bash(srun:*)", "Bash(sbatch:*)", + "Read(./.env)", "Read(~/.ssh/**)" + ], + "ask": ["Bash(git push:*)"], + "additionalDirectories": ["/scratch/$USER"] + }, + "disableBypassPermissionsMode": "disable" +} +``` +(Note: `defaultMode` also exists as a top-level key; it is documented under `permissions` in the +managed examples. `disableBypassPermissionsMode` sits under `permissions` — shown flat here per docs.) + +### MCP +`claude mcp add` (scopes: `local` default → `~/.claude.json`; `project` → `.mcp.json`; `user`). +```bash +# local stdio server +claude mcp add --transport stdio myserver -- npx -y some-mcp-server +# remote streamable HTTP +claude mcp add --transport http notion https://mcp.notion.com/mcp \ + --header "Authorization: Bearer TOKEN" +``` +`.mcp.json` (project, committed) block: +```json +{ + "mcpServers": { + "local-tool": { "command": "/path/to/server", "args": [], "env": {} }, + "remote-api": { "type": "http", "url": "https://mcp.example.com/mcp", + "headers": { "Authorization": "Bearer ${API_KEY}" } } + } +} +``` +An entry with `url` but no `type` is an error (defaults to stdio). `type: "streamable-http"` is +an accepted alias for `http`. Env expansion `${VAR:-default}` supported. SSE (`--transport sse`) +is deprecated. `claude mcp login --no-browser` works over SSH (prints URL to paste). + +### Context files +`CLAUDE.md` load order (broad→specific): managed `/etc/claude-code/CLAUDE.md` → user +`~/.claude/CLAUDE.md` → project `./CLAUDE.md` or `./.claude/CLAUDE.md` → `./CLAUDE.local.md`; +plus ancestor dirs walked up from cwd. Import more files with `@path/to/file` (relative/absolute, +max 4 hops). `.claude/rules/*.md` (optionally `paths:` frontmatter-scoped). +**Claude Code does NOT read `AGENTS.md` natively** — bridge it with `@AGENTS.md` at the top of +`CLAUDE.md`, or `ln -s AGENTS.md CLAUDE.md`. To wire in `/etc/agents.d/*.md`, deploy the managed +`/etc/claude-code/CLAUDE.md` and import each: `@/etc/agents.d/rcac.md`. Load CLAUDE.md from +`--add-dir` dirs with `CLAUDE_CODE_ADDITIONAL_DIRECTORIES_CLAUDE_MD=1`. + +### Install & run +`npm install -g @anthropic-ai/claude-code`, or native installer +`curl -fsSL https://claude.ai/install.sh | bash`. Runs headless over SSH; non-interactive with +`claude -p "…"`. Fully supported on Linux login nodes. + +--- + +## 2. OpenAI Codex CLI + +Docs: · + · + · (redirects +to learn.chatgpt.com). AGENTS.md: . + +### Config file (TOML) +- User: `~/.codex/config.toml` (override home with `CODEX_HOME`). +- Project: `.codex/config.toml`. +- Profiles: separate files `~/.codex/.config.toml`, selected with `codex --profile `. + (As of Codex 0.134.0+ the legacy inline `[profiles.name]` block is removed — use profile files.) + +### Permissions / sandbox +Two orthogonal knobs (no per-command allow/deny list like the others — control is via sandbox + +approvals): +```toml +approval_policy = "on-request" # untrusted | on-request | never (granular table also supported) +sandbox_mode = "workspace-write" # read-only | workspace-write | danger-full-access + +[sandbox_workspace_write] +writable_roots = ["/scratch/USER"] +network_access = false # network OFF by default in workspace-write +exclude_tmpdir_env_var = false +exclude_slash_tmp = false +``` +`model = "gpt-5.6"`. CLI flags: `--sandbox `, `--ask-for-approval `. +Enforcement: macOS = Seatbelt; **Linux/WSL2 = requires `bubblewrap` (`bwrap`) on PATH** — a key +HPC gotcha, since bwrap/user-namespaces are often absent or disabled on login nodes. Windows in +PowerShell uses a native sandbox. `[UNVERIFIED]` whether an older `on-failure` approval value is +still accepted (current reference lists `untrusted`/`on-request`/`never`). + +**Login-node lockdown** (`~/.codex/config.toml`): read-only, never auto-run: +```toml +approval_policy = "untrusted" +sandbox_mode = "read-only" +``` +If `bwrap` is unavailable, keep `sandbox_mode="read-only"` + `approval_policy="untrusted"` so +nothing runs without an explicit approval; do **not** use `danger-full-access` on shared nodes. + +### MCP +`[mcp_servers.NAME]` blocks in `config.toml`, or `codex mcp add` / `codex mcp list`. +```toml +# local stdio +[mcp_servers.example_stdio] +command = "python -m example_server" +args = ["--flag", "value"] +env = { API_KEY = "secret" } +enabled = true +startup_timeout_sec = 10 +tool_timeout_sec = 60 + +# remote streamable HTTP +[mcp_servers.example_http] +url = "https://api.example.com/mcp" +bearer_token_env_var = "MCP_TOKEN" +http_headers = { Authorization = "Bearer token" } +enabled = true +``` + +### Context files +`AGENTS.md` (native — Codex is a first-class member of the agents.md standard). Discovered and +merged from `~/.codex/AGENTS.md` (user) → repo root → current working directory. Config keys: +`project_doc_max_bytes` (per-file cap), `project_doc_fallback_filenames` (extra names to try), +and `experimental_instructions_file` (point at an arbitrary instructions file — a route to wire +in `/etc/agents.d/*.md`). `project_doc_fallback_filenames` can also add cluster-standard names. + +### Install & run +`npm install -g @openai/codex` · `brew install --cask codex` · shell installer +`curl -fsSL https://chatgpt.com/codex/install.sh | sh`. Prebuilt binaries for macOS +(arm64/x86_64) and Linux (x86_64/arm64). Headless/non-interactive: `codex exec "…"`. Runs on +Linux login nodes (Ubuntu 20.04+/Debian 10+; Windows only via WSL2). + +--- + +## 3. Gemini CLI (Google) + +Docs: · +`docs/tools/mcp-server.md` · `docs/tools/shell.md` · `docs/cli/gemini-md.md` · +`docs/cli/enterprise.md` (all under the repo). + +### Config files (JSON) — precedence low→high +1. System defaults `system-defaults.json` +2. User `~/.gemini/settings.json` +3. Workspace `/.gemini/settings.json` +4. **System overrides** (final say): Linux `/etc/gemini-cli/settings.json`, macOS + `/Library/Application Support/GeminiCli/settings.json`, Windows + `C:\ProgramData\gemini-cli\settings.json`. Override path via `GEMINI_CLI_SYSTEM_SETTINGS_PATH`. + +Newer builds use a nested structure (`tools.*`, `security.*`, `context.*`, `mcpServers`, `mcp`); +older flat keys (`coreTools`, `excludeTools`, `contextFileName`) still appear in the wild. + +### Permissions / sandbox +Command control is via the shell tool allow/deny lists with **prefix matching** on +`run_shell_command()`: +```json +{ + "tools": { + "core": ["run_shell_command(git)", "run_shell_command(npm)"], + "exclude": ["run_shell_command(rm)", "run_shell_command(sudo)", "run_shell_command(curl)"] + } +} +``` +`tools.exclude` beats `tools.core`; chained commands (`&&`, `||`, `;`) are split and each part +validated — any disallowed part blocks the whole line. `approvalMode` governs prompting; +`security.disableYoloMode` prevents auto-approve-everything mode. Sandbox: `tools.sandbox` +(bool/`"docker"`/`"podman"`/`"sandbox-exec"`), `tools.sandboxAllowedPaths`, +`tools.sandboxNetworkAccess`, `security.toolSandboxing`. **HPC gotcha:** Gemini's `--sandbox` +relies on Docker/Podman (or macOS Seatbelt) — container runtimes are usually unavailable on HPC +login nodes, so rely on `tools.exclude` + default `approvalMode` instead of the sandbox. + +Minimal login-node config (`~/.gemini/settings.json`): +```json +{ + "tools": { "exclude": ["run_shell_command(rm)", "run_shell_command(sudo)"] }, + "security": { "disableYoloMode": true } +} +``` + +### MCP +`mcpServers` in settings.json. stdio uses `command`; remote SSE uses `url`; streamable HTTP uses +`httpUrl`. Per-server tool filtering with `includeTools`/`excludeTools`; `trust: true` skips +confirmations. Global gate: `mcp.allowed` / `mcp.excluded`. +```json +{ + "mcpServers": { + "pythonTools": { "command": "python", "args": ["-m", "my_mcp_server"], + "cwd": "./mcp", "env": { "DB": "$DB_CONNECTION_STRING" }, "timeout": 15000 }, + "httpServer": { "httpUrl": "http://localhost:3000/mcp", + "headers": { "Authorization": "Bearer TOKEN" } }, + "sseServer": { "url": "https://api.example.com/sse", + "headers": { "Authorization": "Bearer TOKEN" } } + } +} +``` + +### Context files +`GEMINI.md` — global `~/.gemini/GEMINI.md`, plus project + ancestor dirs (and just-in-time when a +tool touches a subtree). **Configurable filename**, which is how Gemini adopts the shared +standard: `{"context": {"fileName": ["AGENTS.md", "CONTEXT.md", "GEMINI.md"]}}` (older key: +`contextFileName`). `context.loadMemoryFromIncludeDirectories` and `context.discoveryMaxDirs` +tune discovery. To wire in `/etc/agents.d/*.md`, either symlink into the search path or add its +filename to `context.fileName` and place a copy at a discovered location `[UNVERIFIED]` whether an +absolute path list is accepted directly. + +### Install & run +`npm install -g @google/gemini-cli` (or `npx https://github.com/google-gemini/gemini-cli`). +Headless: `gemini -p "…"` / `--yolo` (avoid on shared nodes). Runs on Linux login nodes over SSH. + +--- + +## 4. opencode + +Docs: · · + · · + +### Config files (JSON / JSONC) +- Global: `~/.config/opencode/opencode.json` (or `.jsonc`); TUI: `~/.config/opencode/tui.json`. +- Project: `opencode.json` / `opencode.jsonc` in repo root. +- System-managed: Linux `/etc/opencode/`, macOS `/Library/Application Support/opencode/`. +- Overrides: `OPENCODE_CONFIG` (file), `OPENCODE_CONFIG_DIR` (dir). Schema: `"$schema": "https://opencode.ai/config.json"`. + +### Permissions +`permission` key: a string applied to all, or an object keyed by tool +(`read, edit, glob, grep, bash, task, skill, lsp, question, webfetch, websearch, +external_directory, doom_loop`). Values `allow` / `ask` / `deny`. `bash` accepts a **pattern map** +(glob `*`, `?`); **last matching pattern wins**, so put the catch-all first: +```json +{ + "permission": { + "bash": { + "*": "ask", + "ls *": "allow", "cat *": "allow", "git status": "allow", + "git push *": "ask", + "rm -rf *": "deny", "sudo *": "deny", "srun *": "deny", "sbatch *": "deny" + }, + "edit": "ask", + "webfetch": "ask" + } +} +``` +Most tools default to `allow`; `doom_loop` and `external_directory` default to `ask`; reading +`.env*` is denied by default. No OS sandbox — permission rules are the enforcement layer. + +### MCP +`mcp` object. Local (stdio) uses `type:"local"` + `command` **array**; remote uses `type:"remote"` ++ `url`. +```json +{ + "mcp": { + "local-tool": { "type": "local", "command": ["npx", "-y", "my-mcp"], + "environment": { "MY_ENV": "val" }, "enabled": true }, + "remote-api": { "type": "remote", "url": "https://mcp.example.com", + "headers": { "Authorization": "Bearer KEY" }, "enabled": true } + } +} +``` +Local extras: `cwd`, `timeout` (ms, default 5000). Remote extras: `oauth`, `timeout`. + +### Context files +Reads `AGENTS.md` (project root) and `~/.config/opencode/AGENTS.md` (global); falls back to +`CLAUDE.md` / `~/.claude/CLAUDE.md`. **Native agents.md support.** Additional files via the +`instructions` array (local globs + remote URLs), which is the clean way to wire in shared docs: +```json +{ "instructions": ["docs/standards.md", "packages/*/AGENTS.md", "/etc/agents.d/rcac.md"] } +``` + +### Install & run +`curl -fsSL https://opencode.ai/install | bash` · `npm install -g opencode-ai` · +`brew install anomalyco/tap/opencode` (also AUR/pacman). It is a terminal TUI, but the process +runs on the host you launch it on, so it works over SSH on a login node; non-interactive +`opencode run "…"` and a headless `opencode serve` are available. `[UNVERIFIED]` in docs that SSH +is explicitly blessed, but it is a standard terminal program (needs a real terminal emulator on +your side). + +--- + +## 5. Warp + +Docs: · + · + · + + +### Config model +Warp is a **desktop terminal application** (macOS 10.14+, Windows 10 1903+, Linux glibc≥2.31), +configured through its GUI (Settings) and cloud "Warp Drive" — not a dotfile-first CLI. There is +no `~/.warp/config.*` you edit for agent behavior; MCP is the one place you paste JSON. + +### Permissions (Agent Profiles) +Settings > Agents > Profiles. Per-permission-type autonomy: **Agent Decides** / **Always Ask** / +**Always Allow**, applied to: apply diffs, read files, create plans, execute commands, full +terminal use, ask questions. Command control uses **regex**: +- **Allowlist** — auto-run without confirmation, e.g. `ls(\s.*)?`, `grep(\s.*)?`. +- **Denylist** — always require approval; default denylist already blocks `rm(\s.*)?`, + `curl(\s.*)?`, `wget(\s.*)?`, `eval(\s.*)?`. +- **Denylist beats both the allowlist and "Agent decides."** +Gotcha: **Run-until-completion** (`Cmd/Ctrl+Shift+I`) *ignores the denylist entirely* — dangerous +on a login node. Profiles can be team-managed for shared policy. + +### MCP +Settings > Agents > MCP servers (also Warp Drive / Command Palette "Open MCP Servers"). JSON under +an `mcpServers` key; local CLI/stdio uses `command`+`args`(+`env`,`working_directory`); remote +uses `url`(+`headers`). +```json +{ + "filesystem": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "/path"] }, + "externalDocs":{ "url": "http://localhost:4000/mcp/stream", "headers": { "my-header": "val" } } +} +``` + +### Rules / context +Global Rules live in Warp Drive (Personal > Rules). Project Rules are `AGENTS.md` files in the repo +(root or subdirectories); `WARP.md` is the legacy name. **The filename must be ALL CAPS** +(`AGENTS.md`, not `agents.md`). Precedence: subdirectory AGENTS.md → root AGENTS.md → Global Rules. +Native agents.md support, so a cluster-provided `AGENTS.md` is read automatically. + +### Install & run — the HPC caveat +Install locally: direct download, `brew install --cask warp` (macOS), WinGet (Windows), or Linux +packages (`.deb`/`.rpm`/`.AppImage`/pacman). **You cannot run Warp headless on a login node** — it +is a GUI app that needs a display and (on first launch) internet + account login. HPC usage: run +Warp on your workstation, then SSH into the RCAC cluster; Warp advertises "Feature support over +SSH," so Agent Mode operates within the remote session while the app stays local. This makes the +per-cluster settings-file story different from the four CLIs: publish an `AGENTS.md` in the repo +(read automatically) and document the recommended Profile denylist; there is no server-side Warp +config file to ship. + +--- + +## HPC deployment notes (cross-cutting) + +- **Login-node safety:** deny `rm -rf`, `sudo`, and (arguably) scheduler-launch commands + (`srun`/`sbatch`) so agents don't spawn jobs unattended. Claude Code and opencode express this + as per-command deny rules; Gemini as `tools.exclude`; Codex has no per-command list — use + `sandbox_mode="read-only"` + `approval_policy="untrusted"`; Warp ships a default regex denylist. +- **OS sandboxes are the weak spot on HPC:** Codex needs `bubblewrap` on Linux; Gemini needs + Docker/Podman; Claude Code uses Linux namespaces — all of which are frequently disabled on + shared login nodes. Treat the permission/approval layer (not the sandbox) as the primary control + there, and point writable roots at `/scratch/$USER`. +- **Shared `AGENTS.md`:** Codex, opencode, and Warp read `AGENTS.md` natively; Gemini reads it if + added to `context.fileName`; Claude Code needs `@AGENTS.md` import or a symlink. A single repo + `AGENTS.md` therefore covers 4/5 with a one-line Claude bridge. +- **Wiring `/etc/agents.d/*.md`:** Claude Code → managed `/etc/claude-code/CLAUDE.md` with + `@/etc/agents.d/*.md` imports; opencode → `instructions` array (accepts absolute paths/globs); + Codex → `experimental_instructions_file` / `project_doc_fallback_filenames`; Gemini → + `context.fileName`; Warp → repo `AGENTS.md` only (no arbitrary include path). + + diff --git a/spec/agentic-ai-docs/research/06-context-and-conventions.md b/spec/agentic-ai-docs/research/06-context-and-conventions.md new file mode 100644 index 00000000..7e9c1f4f --- /dev/null +++ b/spec/agentic-ai-docs/research/06-context-and-conventions.md @@ -0,0 +1,309 @@ +--- +title: "Context files, harness conventions, and peer prior art" +slug: agentic-ai-docs +research_task: "06 — /etc/agents.d intent, AGENTS.md convention mapping, peer-center prior art" +date: 2026-07-14 +sources: + - glentner/pearc26-hello-computer (outline/03-approach.md, outline/04-discussion.md, rules/*, tips/*) + - Gautschi user guide (overview, storage, software, run_jobs/queues) + main.py macros / snippets + - NERSC, TACC, ALCF, NCSA docs (web) + - agents.md open standard +--- + +# Research 06 — Shared context, harness conventions, and prior art + +## 1. Paper intent for `/etc/agents.d` + per-file content outlines + +### 1.1 What the authors intend `/etc/agents.d` to be + +From `outline/03-approach.md` ("System-Wide Configurations"), the `/etc/agents.d` +directory hierarchy is RCAC's mechanism to **inject HPC-specific context into the +well-known rules-file locations agentic tools already look for**, so "the agent absorbs +the cluster's policies before the user asks their first question." The paper names two +kinds of content that belong there: + +1. **Facts / "which command for what"** — quota checks (`myquota`), batch submission + (`sbatch`), loading software via environment modules, which filesystems serve which + purpose. +2. **Prohibitions** — "don't run computationally intensive work on login nodes, don't + store sensitive data in world-readable locations, don't submit jobs without time + limits." + +So each file is a **cluster-context brief**, not a tutorial: declarative facts + explicit +prohibitions, written *to the agent*. It is the local-knowledge layer that fixes the +paper's central failure mode — the agent that confidently emits "a Slurm script that +would work perfectly… on a cluster we don't operate" (`04-discussion.md` §4.1/§4.4). + +### 1.2 Framing / voice to carry into the files (from `04-discussion.md`) + +- **Proactive engagement, not prohibition / "mostly harmless"** — "We don't pretend they + don't exist or discourage their use" (03); confinement already exists (cgroups, health + checks, root-squash, quotas) so agents "accelerate the pace of potential errors, which + means existing hardening must be strengthened, not invented" (§4.5). +- **Context engineering ("Tea, Earl Grey, Hot")** — structure information so the agent + produces reliable output *from the outset*; specificity beats generality. `/etc/agents.d` + is that engineered specificity supplied by the center rather than the user. +- **Verify, don't forbid / augmented not outsourced ("I Know Kung Fu", "The Answer is 42")** + — "use AI to accelerate your learning, not to bypass it… ask *why*, not just *what*"; + "Verification isn't optional; it's the core competency." Files should tell the agent to + prefer center-verified facts and flag uncertainty over inventing flags. +- **Don't cross the streams (§4.5)** — the concrete blast-radius risks the prohibitions + target: `rm -rf` of a project dir, allocation exhaustion in hours, secrets logged into a + persistent context window. + +### 1.3 What the authors' own `rules/` + `tips/` teach about good rules content + +These are the authors' *battle-tested* agent files; the patterns are the template for tone +and structure: + +- **`rules/file_deletion.md`** — a hard prohibition + the safe alternative (use `del`/ + `/bin/rm` deliberately, never bare `rm`), with a rationale and a scoped exception. Pattern: + *state the danger, give the safe path, name the exception.* +- **`rules/session_logs.md` / `planning_docs.md` / `structural_docs.md`** — "capture what + was requested + done + why"; keep index/structural docs in sync. Maps to RCAC's + reproducibility interest (§4.5 "capturing agent interactions that contribute to + scientific outcomes"). +- **`rules/wip_commits.md`** — branch discipline, commit-often, "force-push OK here but + **never on `main`**." Pattern: *explicit allow + explicit hard prohibition in the same + breath.* +- **`tips/agent-text-editing-pitfalls.md`** — agents silently truncate trailing content in + diffs → "review diffs, request minimal edits, verify after." Reinforces *verify output*. +- **`tips/long-horizon-tasks.md`** — structured memory files (YAML front-matter, session + prompt templates) beat one long context. This is the same "context engineering" the paper + sells; RCAC's `/etc/agents.d/*.md` are exactly these persistent "memory anchors." +- **`tips/warp-conversation-history.md`** — recovering verbatim inputs; the caveat "don't + rely on it, log promptly" mirrors "verification is the whole game." + +Common shape to reuse in every RCAC file: **short declarative facts → the correct +command/path → an explicit "do not …" list → a rationale line.** Each file self-contained +and skimmable, front-matter optional. + +### 1.4 Proposed content outlines (section headings + one-line intent) + +Cluster-context oriented (facts + prohibitions), **not tutorials**. Grounded in verified +Gautschi facts: Rocky Linux 9, Lmod, 8 AMD EPYC "Genoa" front-ends; partitions +`cpu`/`ai`/`highmem` (+ `profiling`/`smallgpu`); QOS `normal`/`standby` (cpu) & +`normal`/`preemptible` (ai); `-A` (see `slist`), `-p`, `-q` all required; home 25 GB w/ +daily snapshots, `$RCAC_SCRATCH`=`/scratch/gautschi/$USER` (huge, **not** backed up, purged +~60 days), `/tmp` node-local, Depot + Fortress (hsi/htar) for long-term; `myquota`, +`findscratch`. + +#### `unix.md` — the login-node OS/shell environment +- **Operating system & shell** — Rocky Linux 9, `bash` default, standard GNU coreutils; + what an agent can assume is present. +- **Front-end nodes are shared** — 8 multi-tenant login nodes; intended for editing, + compiling, job submission, light pre/post-processing only. +- **Do NOT run real compute on login nodes** *(prohibition)* — no heavy/parallel/ + long-running work on front-ends; move it to Slurm (CPU/memory limits enforced). +- **No elevated privileges** *(prohibition)* — no `sudo`/root; can't install system + packages; use environment modules and user space instead. +- **Access & session model** — SSH-key access, ThinLinc for GUI; the agent inherits the + user's existing SSH access, nothing more. +- **Process hygiene** *(prohibition)* — no unbounded background processes / runaway loops; + clean up temp processes; watch for agent retry storms. + +#### `filesystems.md` — where data lives and what is safe +- **Home directory** — path + 25 GB quota + daily snapshots (recoverable); for code/config, + not large data. +- **Scratch (`$RCAC_SCRATCH`)** — `/scratch/gautschi/$USER`, large high-performance parallel + FS for job I/O; **not backed up, purged ~60 days** — never treat as durable *(prohibition)*. +- **`/tmp` / node-local** — ephemeral, per-node, cleared after the job. +- **Long-term storage (Depot, Fortress)** — group/archive space; move keepers off scratch via + `hsi`/`htar`. +- **Check before you write** — use `myquota`, `findscratch`, `du -h --max-depth=1`; respect + quotas (file-count and size). +- **Prohibitions** — no big data in home; no secrets/sensitive data in world-readable paths; + no bulk/`rm -rf` deletes without user confirmation. + +#### `lmod.md` — the software environment +- **Software comes from modules, not the package manager** *(prohibition on apt/yum/system + installs)* — RCAC uses Lmod. +- **Discovering software** — `module avail`, `module spider ` for versions/prereqs. +- **Loading & managing** — `module load/list/purge`; the `(D)` default marker; pin versions + and load the same modules inside job scripts. +- **Recommended toolchain** — GCC 14.1.0 + OpenMPI; hierarchical modules. +- **Python/Conda** — use the `anaconda` modules and user environments; don't pollute base. +- **Prohibitions / pitfalls** — never invent a module name or version; verify with + `module spider` before claiming a package exists. + +#### `slurm.md` — the scheduler +- **Scheduler basics** — Slurm; batch (`sbatch`) vs interactive (`salloc`/`sinteractive`); + `squeue`/`scancel`/`sacct`; `jobinfo`. +- **Every Gautschi job needs four parts** *(the key gotcha)* — resources + account (`-A`, + discover via `slist`) + QOS (`-q`) + **partition (`-p`, mandatory)**. +- **Partitions** — `cpu`, `ai`, `highmem` (+ `profiling`, `smallgpu`): what each is for and + node type. +- **QOS & charging** — `normal`/`standby` on `cpu`; `normal`/`preemptible` on `ai`; time + limits per QOS; checkpoint `preemptible` jobs. +- **Always set a time limit and realistic resources** *(prohibition on no-`--time` jobs)* — + memory is allocated proportional to cores. +- **Prohibitions** — don't emit PBS/LSF/other-scheduler flags; don't invent partitions/QOS; + don't run compute on login nodes; confirm before bulk/`scancel` of others' jobs; don't + exhaust the allocation. + +#### `policies.md` — rules, data handling, and cautions +- **Acceptable use** — bound by Purdue IT / RCAC AUP; link the policy of record. +- **Data handling & sensitivity** *(prohibition)* — no regulated/sensitive/export-controlled + data without prior approval. +- **Secrets & credentials** *(prohibition)* — never store API keys/tokens/private keys in + world-readable or shared paths; never paste them into prompts/agent context. +- **Resource stewardship** — recap: no heavy compute on login nodes; Slurm jobs carry time + limits; steward the allocation. +- **Destructive operations require confirmation** *(prohibition)* — `rm -rf`, mass moves, + bulk `scancel`; the agent must confirm with the user. +- **Account integrity & getting help** — no account/credential sharing (the agent acts *as + you*); verify AI output before running; report bad AI commands to `rcac-help@purdue.edu`. + +--- + +## 2. The context-file convention and how one canonical set maps onto every harness + +### 2.1 The `AGENTS.md` open standard (the anchor to cite) + +`AGENTS.md` () is an **open, tool-agnostic format** — "a README for +agents": plain Markdown with flexible headings, placed at the **repo root** (or system/home +root for a global config), with nested files allowed and "the closest `AGENTS.md` to the +edited file wins." Its philosophy is *separation of concerns* — `README.md` is for humans, +`AGENTS.md` is the detailed context agents need. The site reports **60k+ projects** and +**~25–30+ tools** consuming it, explicitly including **OpenAI Codex, Gemini CLI, Cursor, +Aider, VS Code, GitHub Copilot, Zed, Warp, JetBrains Junie, opencode/Windsurf**, etc. Its +migration FAQ is the load-bearing quote for our strategy: **"Rename existing files to +`AGENTS.md` and create symbolic links for backward compatibility."** + +### 2.2 The well-known per-harness filenames (the fan-out targets) + +Each harness historically reads its *own* context filename and its own settings/permissions +format. For our five: + +| Harness | Context file it reads | Settings / permissions file | +|---|---|---| +| **Claude Code** | `CLAUDE.md` (and `@path` imports); also reads `AGENTS.md` in recent versions | `.claude/settings.json` (`permissions.allow`/`deny`/`ask`) | +| **OpenAI Codex** | `AGENTS.md` natively | `~/.codex/config.toml` | +| **Gemini CLI** | `GEMINI.md` (context filename is configurable → can point at `AGENTS.md`) | `.gemini/settings.json` | +| **opencode** | `AGENTS.md` natively | `opencode.json` | +| **Warp** | `WARP.md` / Warp Rules (Warp also added `AGENTS.md` support) | Warp Rules / drive config | + +The key fact: **`AGENTS.md` is already the common denominator** — Codex and opencode read it +natively, Gemini can be pointed at it, Warp supports it, and Claude Code reads `CLAUDE.md` +which the standard tells you to *symlink to `AGENTS.md`* (this repo does exactly that: +`CLAUDE.md -> AGENTS.md`, `.claude -> .agents`). + +### 2.3 Recommended mapping strategy (one canonical source → every harness) + +RCAC's canonical source of truth is **five separate topic files** in `/etc/agents.d/` +(`unix.md`, `filesystems.md`, `lmod.md`, `slurm.md`, `policies.md`). Surface them two ways +depending on deployment mode (per GOAL R8/R12): + +1. **On-cluster mode (harness runs on a Gautschi login node).** Concatenate the five files, + in a stable order, into a single canonical **`AGENTS.md`** placed at a system/global + context location, then **symlink the other well-known names to it**: + `CLAUDE.md → AGENTS.md`, `GEMINI.md → AGENTS.md`, `WARP.md → AGENTS.md`. This is the + standard's own prescribed pattern and exactly the trick used in the paper repo and this + RCAC-Docs repo. Concatenation (not five separate files) is preferred because most + harnesses load a *single* root context file; the `/etc/agents.d/*.md` split is a + *maintenance/source* convenience, collapsed at deploy time. Puppet performs the copy + + concat + symlink onto the hosts (out of scope for the docs feature, but the mechanism to + describe). +2. **Local mode (harness on the user's laptop, targeting the cluster over MCP + SSH).** The + HPC MCP server (`rcac-mcp`) **reads `/etc/agents.d/` over SSH and injects the markdown as + agent context** at connect time (GOAL R8) — no local file placement needed. The path is + configurable. + +**Permissions** are *not* portable across harnesses — each settings format is different — so +R11's per-harness settings files must be authored natively (`.claude/settings.json`, +`config.toml`, `.gemini/settings.json`, `opencode.json`, Warp Rules), each (a) wiring in the +shared context (via `AGENTS.md`/symlink or an import directive) and (b) encoding the same +starting-point deny/allow policy (deny destructive ops + login-node heavy compute; require +Slurm time limits; no secrets in world-readable paths; confirm destructive actions). The +*context* is DRY via `AGENTS.md`; the *permissions* are parallel translations of one policy. + +**Net recommendation:** treat `/etc/agents.d/*.md` as the DRY source, concatenate to a single +canonical `AGENTS.md`, symlink every other harness's context filename to it (cite the +agents.md FAQ), inject the same files over SSH via MCP for local mode, and hand-translate the +one permission policy into each harness's native settings file. + +--- + +## 3. Prior-art survey — what peer HPC centers publish about agentic AI + +Verdict up front: **the space is early but no longer empty.** Two centers (NERSC, TACC) now +publish explicit agentic-AI guidance; the tone at both is *permit-with-guardrails + +verify*, matching RCAC's stance almost exactly. Others (ALCF, NCSA) had nothing agent-facing +visible at survey time. RCAC would be an early mover but not a first mover — and can credibly +cite NERSC/TACC as convergent practice. + +### 3.1 NERSC — a full, dedicated page (strongest prior art) +**URL:** + +A complete, RCAC-shaped guide titled around **AI Coding Tools / coding agents**. Tone is +**cautiously permissive**: *"The goal is not to let a model operate unsupervised. The goal is +to use these tools to reduce repetitive work while keeping humans in control."* Notable +because it independently arrives at nearly every RCAC pillar: + +- **Login-node rule** — small interactive debugging OK; no long-running/resource-intensive + work on login nodes. +- **Sandboxing / write scope** — recommends **workspace-write mode**, confining agent writes + to `$HOME`/`$SCRATCH`; sensitive `$CFS` data read-only or copied out. +- **Secrets** — *"Do not paste credentials, tokens, or private keys into prompts."* +- **Verification mandate** — *"Users are responsible for validating generated code, commands, + configurations, and scientific results"*; a whole section **"Slurm and Module Advice Needs + Verification"** and **"Verification Is the Whole Game."** +- **AUP boundary** — agents "must not be given prohibited data, must not bypass access + controls," must stay in approved project scope. +- Rich section set incl. **Context Engineering**, **Sandboxing and Approval Boundaries**, + **Common AI Mistakes on HPC Systems**, **Prompt Template for NERSC Tasks**, and *"If You + Only Remember Four Things."* References Claude Code by name. (No MCP-server section + observed — an opening for RCAC to differentiate.) + +This page is the closest analog to RCAC's planned content and a strong structural model / +tone reference; RCAC's differentiators are the **published `/etc/agents.d` context**, +**per-harness settings**, and **MCP servers** — which NERSC does not appear to cover. + +### 3.2 TACC — a concise policy block in the Good Conduct Guide +**URL:** ("AI Tools on TACC Resources") + +Short, blunt, policy-flavored — folded into the login-node conduct rules. Verbatim rules: +- *"All AI-assisted workloads must be executed on compute nodes only."* +- *"YOU are responsible for all your processes initiated on TACC resources, including those + launched via AI tools or agents."* +- *"All Service Units (SUs) consumed by AI-related workloads will be charged against your + allocation."* +- Suggested workflow: grab a compute node via **`idev`**, then point the AI tool at the + allocated node. Warning: violations → *"degraded access… or administrative action."* + +Tone is firmer/more enforcement-y than NERSC, but the substance is identical to RCAC's +(no login-node compute; user is accountable; allocation is charged). Good source for the +acceptable-use/etiquette page (R2) — note TACC pushes *all* AI work to compute nodes, a +stricter line than RCAC's "light work on login nodes OK." + +### 3.3 ALCF — nothing agent-facing found +**URL:** (returned HTTP 403 to the fetcher). ALCF publishes +extensive *AI-for-science* (training/inference) material but no agentic coding-assistant / +AI-agent acceptable-use guidance was locatable. Treat as "no prior art at survey time." + +### 3.4 NCSA — nothing agent-facing found +**URLs:** and the Delta guide +. No mention of AI coding agents, +LLM assistants, or MCP servers surfaced. Treat as "no prior art at survey time." + +### 3.5 MCP-for-HPC prior art +WebSearch was blocked in this environment (org policy on the search feature), so a broad +sweep for third-party "MCP + Slurm" projects couldn't be completed here. From the material +available: **no peer center documents an MCP server for cluster/Slurm operations** — NERSC, +TACC, ALCF, NCSA pages contain no MCP references. This makes RCAC's `rcac-mcp` / +`rcac-docs-mcp` / `globus-mcp` documentation a genuine **differentiator / white space** and +supports the paper's "local-first MCP" framing as novel published practice. (Recommend a +follow-up unrestricted web search to confirm no community "slurm-mcp" prior art before +claiming first-mover status in the docs.) + +### 3.6 Takeaways for the RCAC docs +- **Not first, but early.** Cite NERSC + TACC as convergent peer practice; it de-risks the + stance and lends authority ("centers including NERSC and TACC…"). +- **Tone converges on RCAC's:** permit + guardrails + *verification is the whole game* + + login-node discipline + secrets prohibition + user-accountability. RCAC can adopt this + register confidently. +- **RCAC's white space:** published `/etc/agents.d` context files, per-harness settings, and + documented MCP servers — none of the four peers publish these. Lead with them. +- **One caution to mirror TACC/NERSC precisely:** the login-node line. TACC = compute nodes + only; NERSC = light interactive OK. RCAC should state its own line explicitly (GOAL R5: + light on front-ends, real work via Slurm) rather than leave it implied. From 020e4b4fb1dcfab7e5df40a7cb69964a13cdd5d6 Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Wed, 15 Jul 2026 09:38:35 -0400 Subject: [PATCH 03/32] [feature] Refine agentic-ai-docs GOAL/plan per review feedback --- spec/agentic-ai-docs/GOAL.md | 43 ++++++++++-- spec/agentic-ai-docs/PLAN.md | 40 ++++++----- spec/agentic-ai-docs/TECH.md | 78 +++++++++++++--------- spec/agentic-ai-docs/research/00-digest.md | 49 +++++++++----- 4 files changed, 141 insertions(+), 69 deletions(-) diff --git a/spec/agentic-ai-docs/GOAL.md b/spec/agentic-ai-docs/GOAL.md index e8b1ac54..a8f8e300 100644 --- a/spec/agentic-ai-docs/GOAL.md +++ b/spec/agentic-ai-docs/GOAL.md @@ -93,8 +93,9 @@ Reader-facing, observable outcomes. Stable IDs survive squash-merge. its public GitHub repository: the HPC operations server (`rcac-mcp`, generalizing toward a plugin architecture, e.g. `cluster-mcp[slurm,lmod,…]`); the docs-search server (`rcac-docs-mcp`, hosted at `docs.rcac.purdue.edu/mcp`); and `globus-mcp` (data transfers). It - SHALL explain *why* MCP matters (context that knows our clusters) and present the tooling as - evolving/prototype. + SHALL explain *why* MCP matters (context that knows our clusters), present all three as **working + prototypes under active development**, and explicitly **call out the planned rename / + re-architecture of `rcac-mcp`** toward the HPC-only plugin model. - **R8** — The MCP page SHALL explain that the HPC MCP server reads the host's `/etc/agents.d` markdown context (a configurable location) over SSH and injects it into the agent's context. - **R9** — Any install/connection commands or tool names shown for the MCP servers SHALL be @@ -104,12 +105,16 @@ Reader-facing, observable outcomes. Stable IDs survive squash-merge. - **R10** — The section SHALL publish the actual Gautschi shared-context files verbatim (`unix.md`, `filesystems.md`, `lmod.md`, `slurm.md`, `policies.md`); their content SHALL be - HPC-accurate for Gautschi (Rocky Linux 9; Lmod; partitions `cpu`/`ai`/`highmem`; QOS - `normal`/`standby`/`preemptible`; correct filesystem paths and quotas). + HPC-accurate for Gautschi (Rocky Linux 9; Lmod; partitions `cpu`/`ai`/`highmem`/`smallgpu`/ + `profiling`; QOS `normal`/`standby`/`preemptible`; correct filesystem paths and technologies — + `/home` ZFS, `/depot` GPFS, `/scratch` Lustre — deferring volatile quota numbers to + `myquota`/`slist` rather than hardcoding them). - **R11** — The section SHALL publish per-harness settings/permission files for all five harnesses (Claude Code, Codex, Gemini CLI, opencode, Warp), each in that harness's native settings - format, wiring in the shared context and encoding a starting-point set of enforced permissions - (e.g. deny destructive operations and login-node heavy compute). + format, wiring in the shared context and encoding a starting-point permission policy that both + (a) **denies** destructive/dangerous operations (e.g. `rm -rf`, `sudo`) and login-node heavy + compute, and (b) **allow-lists common read-only sanity commands** (`myquota`, `slist`, + `sfeatures`, `module list`, `module avail`, …) so the agent runs them eagerly without prompting. - **R12** — A page SHALL explain how `/etc/agents.d` context and the harness settings fit together (context injected; permissions enforced), state that these files are the canonical source of truth copied into cluster config management, and tell readers how to contribute feedback and @@ -181,6 +186,32 @@ Resolved with the human during shaping on 2026-07-14. "mostly harmless"; verify don't forbid; augmented not outsourced; "don't cross the streams" caution. Professional, instructional, second person (house style). +### Resolved during planning review (2026-07-15) + +Refinements from the human after reviewing `PLAN.md`/`TECH.md`: + +- **MCP framing (confirmed).** All three servers (`rcac-mcp`, `globus-mcp`, `rcac-docs-mcp`) are + **working prototypes under active development**. Docs SHALL explicitly **call out the planned + rename / re-architecture of `rcac-mcp`** toward the HPC-only plugin model (`cluster-mcp[slurm, + lmod,…]`) — as a stated direction, not a shipped fact. (Refines R7.) +- **Warp is first-class and recommended.** Warp is the **recommended harness for most users**; the + docs SHALL tell the story of using it *well* — run locally on the user's workstation, targeting the + cluster over SSH (Warp is a desktop app and does not run on login nodes). Give it prominent, + first-class treatment in the local (MCP + SSH) guidance, not a footnote. +- **Container model is Apptainer, not Docker.** These clusters use **Apptainer**. RCAC's Apptainer + configuration **auto bind-mounts `/home`, `/depot`, and `/scratch`** into containers for + convenience, so containers offer **limited protection** — the most likely failure mode is an agent + *editing files* on those (still-writable) mounts; users can disable the bind-mounts by invoking + Apptainer explicitly. Best-practices/caution and the context files SHALL reflect this (do not imply + Docker or strong container sandboxing on the nodes). +- **Filesystem technologies (authoritative, human-corrected).** `/home` is **ZFS**, `/depot` is + **GPFS**, `/scratch` is **Lustre** — this corrects the generic snippet that implied `/home` on + GPFS. `filesystems.md` SHALL state these correctly. (Refines R10.) +- **Eager read-only sanity checks + allow-list.** Guidance and context SHALL instruct agents to + **eagerly run read-only sanity commands** — `myquota`, `slist`, `sfeatures`, `module list`, + `module avail`, and similar — and the per-harness settings SHALL **allow-list** them so the agent + does not stop to ask. (Refines R11.) + *No unresolved `[NEEDS CLARIFICATION]` markers remain; the GOAL is ready for `docs-plan`.* ## Related materials diff --git a/spec/agentic-ai-docs/PLAN.md b/spec/agentic-ai-docs/PLAN.md index 91d2e431..3831fe04 100644 --- a/spec/agentic-ai-docs/PLAN.md +++ b/spec/agentic-ai-docs/PLAN.md @@ -42,13 +42,13 @@ context/settings artifacts — see *Reuse* below. | `index.md` | *Agentic AI at RCAC* — stance (proactive engagement, not prohibition; verify, don't forbid), grid-card nav to subpages | R1 | | `acceptable_use.md` | *Acceptable Use & Etiquette* — rules for agents on RCAC systems; reuse `{{ resource_use(resource) }}`; link Purdue IT AUP | R2 | | `best_practices.md` | *Best Practices & Limitations* — agent *for research* vs *for operations*; context engineering; verification ("augmented, not outsourced"); caution/blast-radius | R3 | -| `mcp_servers.md` | *RCAC MCP Servers* — why MCP; the three servers with verified connect commands; `/etc/agents.d` injection | R7, R8, R9, R16 | +| `mcp_servers.md` | *RCAC MCP Servers* — why MCP; the three **actively-developed prototypes** with verified connect commands; `/etc/agents.d` injection; **call out the `rcac-mcp` rename/re-architecture** | R7, R8, R9, R16 | | `running_agents/index.md` | *Running Agents* — the two deployment modes explained; card links | R4 | | `running_agents/on_cluster.md` | *On the Cluster (Login Nodes)* — five harnesses on login nodes; login-node constraints; Warp = local-only note | R4, R5, R16 | -| `running_agents/local.md` | *Local, Targeting the Cluster (MCP + SSH)* — local-first architecture; connect each harness to the MCP servers | R4, R6 | +| `running_agents/local.md` | *Local, Targeting the Cluster (MCP + SSH)* — local-first architecture; connect each harness to the MCP servers; **feature Warp prominently (RCAC's recommended harness)** | R4, R6 | | `shared_context/index.md` | *Shared Context & Settings* — how `/etc/agents.d` + settings fit; canonical source-of-truth → Puppet + MCP injection; **how to contribute feedback** | R12 | | `shared_context/context_files.md` | *Context Files (`/etc/agents.d`)* — the five Gautschi context files, published verbatim | R10, R16 | -| `shared_context/settings.md` | *Harness Settings & Permissions* — the five per-harness settings, published verbatim | R11 | +| `shared_context/settings.md` | *Harness Settings & Permissions* — the five per-harness settings, published verbatim; **deny destructive ops + allow-list read-only sanity commands** | R11 | **New Gautschi chapter:** `docs/userguides/gautschi/using_ai_agents.md` (R13, R16). @@ -153,8 +153,8 @@ meaningful alt text. Code blocks carry a `title=` label. This is content-level a | R7 | `mcp_servers.md` — three servers, focus + repo links, "why MCP" | | R8 | `mcp_servers.md` — HPC server reads `/etc/agents.d` over SSH, injects as context | | R9 | `mcp_servers.md` — install/connect commands verified against the live repos (research 04) | -| R10 | `docs/snippets/agentic-ai/agents.d/*.md` (Gautschi-accurate) + `shared_context/context_files.md` | -| R11 | `docs/snippets/agentic-ai/{claude,codex,gemini,opencode,warp}/…` + `shared_context/settings.md` | +| R10 | `docs/snippets/agentic-ai/agents.d/*.md` (Gautschi-accurate; `/home` ZFS · `/depot` GPFS · `/scratch` Lustre; quotas via `myquota`) + `shared_context/context_files.md` | +| R11 | `docs/snippets/agentic-ai/{claude,codex,gemini,opencode,warp}/…` (deny destructive ops + **allow-list read-only sanity commands**) + `shared_context/settings.md` | | R12 | `shared_context/index.md` — how it fits, source-of-truth → Puppet + MCP, feedback channels | | R13 | `docs/userguides/gautschi/using_ai_agents.md` + Gautschi nav entry + cross-links | | R14 | Every phase adds its nav line; every `verify:` runs `mkdocs build --strict` | @@ -221,24 +221,32 @@ after this design. Touched sections and how the design honors each: ## 5. Risks & open questions -- **⚠ `cluster-mcp` / HPC-only refocus is [UNVERIFIED] publicly** (research 04): the public +- **`cluster-mcp` / HPC-only refocus is [UNVERIFIED] publicly** (research 04): the public `rcac-mcp` is a prototype that still bundles docs tools; the plugin architecture - (`cluster-mcp[slurm,lmod,…]`) has no public trace. **Mitigation:** document `rcac-mcp` as the - current prototype with its real tools; present the plugin/refocus direction as *planned/evolving*, - clearly labeled. **Human check before publish** that this framing is acceptable. + (`cluster-mcp[slurm,lmod,…]`) has no public trace. **Human confirmed the framing (2026-07-15):** + treat all three servers as **actively-developed working prototypes** and **explicitly call out the + planned rename/re-architecture of `rcac-mcp`** — as a stated direction, not shipped fact. No longer + a blocker; keep the "planned/evolving" labeling. - **Gautschi `[UNCONFIRMED]` facts** — exact home/scratch quotas (illustrative only), Depot mount path, OpenMPI version, default shell, availability of `sinfo`/`sacct`/`jobinfo`, a Purdue AUP direct URL. **Mitigation:** context files tell the agent to run `myquota`/`findscratch`/`slist` instead of hardcoding; state only verified facts; use the AUP block from `resource_use` / link - Purdue IT Policy V.4.1 (from `resourceuse.md`). + Purdue IT Policy V.4.1 (from `resourceuse.md`). **Filesystem tech now confirmed** (human): `/home` + ZFS · `/depot` GPFS · `/scratch` Lustre. - **Warp is a local desktop GUI, not a headless CLI** — cannot be installed on a login node - (research 05). **Mitigation:** on-cluster coverage for Warp = "run on your workstation, SSH in"; - no server-side Warp settings file — represent Warp's "settings" as `AGENTS.md` + a documented - Agent-Profile denylist, and note the Run-until-completion denylist bypass. + (research 05) — **but it is RCAC's *recommended* harness for most users** (human, 2026-07-15). + **Mitigation:** give Warp **first-class, prominent** treatment in the **local (MCP + SSH)** guidance + — tell the story of using it well ("run on your workstation, SSH in"); represent its "settings" as + `AGENTS.md` + a documented Agent-Profile denylist, and note the Run-until-completion denylist bypass. - **Enforced-permissions policy is a v0 starting point** (GOAL clarification) — label it as such and - invite feedback. OS sandboxes (bwrap/Docker) are often disabled on shared nodes → lean on the - permission/approval layer; the real config-management enforcement points are Claude - `/etc/claude-code/managed-settings.json` and Gemini `/etc/gemini-cli/settings.json`. + invite feedback. It must both **deny** destructive ops (`rm -rf`, `sudo`) **and allow-list common + read-only sanity commands** (`myquota`, `slist`, `sfeatures`, `module list`, `module avail`, …) so + agents run them eagerly without prompting (human, 2026-07-15). **Container model is Apptainer, not + Docker:** RCAC's Apptainer config **auto bind-mounts `/home`, `/depot`, `/scratch`** → limited + protection (editing files is the real risk; bind-mounts disable-able on explicit invocation) — the + caution/best-practices content must reflect this, not imply Docker sandboxing. The real + config-management enforcement points are Claude `/etc/claude-code/managed-settings.json` and Gemini + `/etc/gemini-cli/settings.json`. - **`check_paths:false`** — a mistyped `--8<--` path fails **silently** (empty block, no `--strict` error). **Mitigation:** each context/settings phase greps the built `site/` for a sentinel token. - **Feedback channel** for context corrections — default to the RCAC-Docs GitHub issues + diff --git a/spec/agentic-ai-docs/TECH.md b/spec/agentic-ai-docs/TECH.md index 9537d9c5..14bb9719 100644 --- a/spec/agentic-ai-docs/TECH.md +++ b/spec/agentic-ai-docs/TECH.md @@ -181,9 +181,13 @@ engineering, verification, and caution/blast-radius. - [ ] Create `docs/agentic-ai/best_practices.md`: research-vs-operations distinction; context engineering ("Tea, Earl Grey, hot" — specificity); verification ("augmented, not outsourced"; - ask *why* not just *what*; the expertise paradox); caution/blast-radius (destructive commands, - allocation exhaustion, credential/secret leakage; existing HPC confinement contains blast - radius). Frame from the paper's Discussion; professional/instructional voice. + ask *why* not just *what*; the expertise paradox); **let agents run read-only sanity checks + eagerly** (`myquota`, `slist`, `sfeatures`, `module list`, `module avail`) to ground themselves + before acting; caution/blast-radius (destructive commands, allocation exhaustion, + credential/secret leakage; existing HPC confinement contains blast radius). **Container note: + RCAC uses Apptainer (not Docker), and its config auto bind-mounts `/home`, `/depot`, `/scratch` + — so containers give limited protection and the likeliest failure is an agent *editing files*; + don't imply Docker-style sandboxing.** Frame from the paper's Discussion; professional voice. - [ ] Add nav line; back-link to hub. - **Verify:** `… strict_check.py && grep -q 'best_practices.md' mkdocs.yml` - **Touches:** `docs/agentic-ai/best_practices.md`, `mkdocs.yml`. @@ -193,17 +197,17 @@ engineering, verification, and caution/blast-radius. **Goal:** document the three RCAC MCP servers accurately (why MCP; current focus; verified connect commands; the `/etc/agents.d` injection), presenting the tooling as prototype/evolving. -- [ ] Create `docs/agentic-ai/mcp_servers.md`: "why MCP" (context that knows our clusters); a - per-server section for **rcac-mcp** (HPC ops, prototype, stdio via `uvx`, runs over the user's - existing SSH, tool list, `rcac://context`), **globus-mcp** (transfers, beta, user's Globus - OAuth), **rcac-docs-mcp** (hosted HTTP at `docs.rcac.purdue.edu/mcp`, `doc_search`/`doc_load`). - Link each public repo. Use the **verified** connect blocks from research 04 (inline fenced - JSON/TOML — short, no Jinja triggers). +- [ ] Create `docs/agentic-ai/mcp_servers.md`: "why MCP" (context that knows our clusters); present + **all three as actively-developed working prototypes**; a per-server section for **rcac-mcp** + (HPC ops, stdio via `uvx`, runs over the user's existing SSH, tool list, `rcac://context`), + **globus-mcp** (transfers, beta, user's Globus OAuth), **rcac-docs-mcp** (hosted HTTP at + `docs.rcac.purdue.edu/mcp`, `doc_search`/`doc_load`). Link each public repo. Use the + **verified** connect blocks from research 04 (inline fenced JSON/TOML — short, no Jinja triggers). - [ ] State that the HPC server reads the host's `/etc/agents.d` markdown over SSH and injects it as context (R8). -- [ ] Frame the `cluster-mcp[slurm,lmod,…]` plugin/HPC-refocus direction as **planned/evolving**, - clearly labeled — NOT as shipped fact (research 04 marks it `[UNVERIFIED]`). ⚠ Surface to the - human at review/publish. +- [ ] **Explicitly call out the planned rename / re-architecture of `rcac-mcp`** toward the HPC-only + plugin model (`cluster-mcp[slurm,lmod,…]`) — framed as a **stated direction / planned&evolving**, + NOT shipped fact (human-confirmed framing 2026-07-15; research 04 marks it `[UNVERIFIED]`). - [ ] Add nav line; back-link to hub. - **Verify:** `… strict_check.py && grep -q 'mcp_servers.md' mkdocs.yml` - **Touches:** `docs/agentic-ai/mcp_servers.md`, `mkdocs.yml`. @@ -219,8 +223,11 @@ covering all five harnesses with the login-node constraints. - [ ] Create `docs/agentic-ai/running_agents/on_cluster.md`: install/run each of Claude Code, Codex, Gemini CLI, opencode on a login node (Linux); the login-node compute constraint (mirror the Gautschi Running-Jobs rule); point writable work at `$RCAC_SCRATCH`; note OS sandboxes are - often unavailable on shared nodes. **Warp:** state honestly it is a local desktop GUI and - **cannot** run on a login node — run it on your workstation and SSH in (see local page). + often unavailable on shared nodes (RCAC uses **Apptainer**, which auto bind-mounts + `/home`/`/depot`/`/scratch` → limited protection). **Warp:** state honestly it is a local + desktop GUI and **cannot** run on a login node — but it is **RCAC's recommended harness for most + users**; run it on your workstation and SSH in (point to the local page for the recommended + workflow). - [ ] Add the `Running Agents` nav sub-section (index + On the Cluster); back-links to hub. - **Verify:** `… strict_check.py && grep -q 'running_agents/on_cluster.md' mkdocs.yml` - **Touches:** `docs/agentic-ai/running_agents/{index,on_cluster}.md`, `mkdocs.yml`. @@ -234,7 +241,10 @@ existing SSH via the MCP servers. credentials, no hosted infra — "if you can SSH to the cluster, your agent can too"); how to register `rcac-mcp` (and globus-mcp / rcac-docs-mcp) in each harness (reference research 05 MCP-config forms); cite `docs/lifesciences/guides/vscode.md` as prior art for external-tool → - cluster tunneling. Back-link to hub; forward links to `mcp_servers.md` deferred to P10. + cluster tunneling. **Feature Warp first-class and up front — it is RCAC's recommended harness + for most users: tell the story of using it *well* locally (Agent Mode on the workstation, + targeting Gautschi over SSH, with `rcac-mcp` connected).** Back-link to hub; forward links to + `mcp_servers.md` deferred to P10. - [ ] Add the `Local (MCP + SSH)` nav line under Running Agents. - **Verify:** `… strict_check.py && grep -q 'running_agents/local.md' mkdocs.yml` - **Touches:** `docs/agentic-ai/running_agents/local.md`, `mkdocs.yml`. @@ -247,11 +257,14 @@ publish them verbatim; the shared-context hub explains the model + how to give f - [ ] Author canonical files under `docs/snippets/agentic-ai/agents.d/`: `unix.md`, `filesystems.md`, `lmod.md`, `slurm.md`, `policies.md` — each *fact → correct command/path → "do not…" → rationale* (outlines in research 06), grounded in verified Gautschi facts (research 03). Also - write `agents.d/AGENTS.md` (the concatenated canonical context harnesses read). **Accuracy - gate:** partitions cpu/ai/highmem/smallgpu/profiling; QOS normal/standby/preemptible; `-A` - (via `slist`) + `-q` + `-p` mandatory; `$RCAC_SCRATCH`, 60-day purge, `myquota`/`findscratch`; - GCC 14.1.0 + OpenMPI; **no `--partition=a10`, no `-A standby`**; defer quota numbers to - `myquota`. + write `agents.d/AGENTS.md` (the concatenated canonical context harnesses read). Instruct the + agent to **eagerly run read-only sanity checks** (`myquota`, `slist`, `sfeatures`, + `module list`, `module avail`) before acting. **Accuracy gate:** partitions + cpu/ai/highmem/smallgpu/profiling; QOS normal/standby/preemptible; `-A` (via `slist`) + `-q` + + `-p` mandatory; filesystems **`/home` ZFS · `/depot` GPFS · `/scratch` (`$RCAC_SCRATCH`) Lustre, + 60-day purge**, `myquota`/`findscratch`; GCC 14.1.0 + OpenMPI; **Apptainer (not Docker), config + auto bind-mounts `/home`/`/depot`/`/scratch` → limited protection**; **no `--partition=a10`, + no `-A standby`**; defer quota numbers to `myquota`. - [ ] Create `docs/agentic-ai/shared_context/index.md` (subdir hub): what `/etc/agents.d` is, that these files are the **canonical single source of truth** copied into cluster config management (Puppet) and injected by the MCP server over SSH, and **how to contribute feedback/corrections** @@ -268,16 +281,21 @@ publish them verbatim; the shared-context hub explains the model + how to give f **Goal:** author and publish verbatim the per-harness settings/permission files for all five harnesses, wiring in the shared context and encoding a starting-point enforced-permission policy. -- [ ] Author canonical files under `docs/snippets/agentic-ai/`: `claude/settings.json` - (`permissions.deny` for `rm -rf`/`sudo`; note managed `/etc/claude-code/managed-settings.json` - as the enforcement point; `mcpServers` for rcac-mcp; CLAUDE.md→AGENTS.md bridge), - `codex/config.toml` (`approval_policy`/`sandbox_mode`; `[mcp_servers.rcac]`; AGENTS.md native), - `gemini/settings.json` (`tools.exclude` `run_shell_command(rm)`/`(sudo)`; note system - `/etc/gemini-cli/settings.json` as the enforcement point; `mcpServers`), `opencode/opencode.json` - (`permission` bash-pattern map, catch-all first; `mcp` local), `warp/AGENTS.md` (Warp has no - server-side settings file — represent it as the repo AGENTS.md + a documented Agent-Profile - denylist; note Run-until-completion bypasses the denylist). Use research 05 for exact schema; - mark any `[UNVERIFIED]` key honestly. +- [ ] Author canonical files under `docs/snippets/agentic-ai/`. Every file must both **allow-list + read-only sanity commands** (`myquota`, `slist`, `sfeatures`, `module list`, `module avail`) so + they run without prompting, **and deny** destructive/dangerous ops (`rm -rf`, `sudo`): + `claude/settings.json` (`permissions.allow` for the sanity commands + `permissions.deny` for + `rm -rf`/`sudo`; note managed `/etc/claude-code/managed-settings.json` as the enforcement point; + `mcpServers` for rcac-mcp; CLAUDE.md→AGENTS.md bridge), `codex/config.toml` + (`approval_policy`/`sandbox_mode`; `[mcp_servers.rcac]`; AGENTS.md native — note Codex has no + per-command allowlist, so rely on approval/sandbox mode), `gemini/settings.json` (`tools.core` + allow-list `run_shell_command(myquota)`/`(slist)`/`(module)`… + `tools.exclude` + `run_shell_command(rm)`/`(sudo)`; note system `/etc/gemini-cli/settings.json` as the enforcement + point; `mcpServers`), `opencode/opencode.json` (`permission` bash-pattern map — allow the sanity + commands, deny `rm -rf *`/`sudo *`, **catch-all first**; `mcp` local), `warp/AGENTS.md` (Warp has + no server-side settings file — represent it as the repo AGENTS.md + a documented Agent-Profile + allow/deny list; note Run-until-completion bypasses the denylist). Use research 05 for exact + schema; mark any `[UNVERIFIED]` key honestly. - [ ] Create `docs/agentic-ai/shared_context/settings.md`: publish each settings file verbatim via `--8<--` fenced blocks (json/toml/markdown); explain each harness's permission model briefly, that this is a **v0 starting point** inviting feedback, and where config-management enforcement diff --git a/spec/agentic-ai-docs/research/00-digest.md b/spec/agentic-ai-docs/research/00-digest.md index 12d1be67..30ccdb13 100644 --- a/spec/agentic-ai-docs/research/00-digest.md +++ b/spec/agentic-ai-docs/research/00-digest.md @@ -99,11 +99,13 @@ clearly labeled — not as shipped fact. Confirm with the human before publish.* `standby` (idle, free, 4h max), `preemptible` (ai only, 0.25 GPU-hr, killable). List accounts with **`slist`**. Verified commands: `sbatch`, `squeue -u`, `scontrol show/hold/release`, `scancel`, `sinteractive`, `srun`, `slist`, `sfeatures`, `myquota`, `findscratch`, `purgelist`. -- **Filesystems:** home `/home/$USER` (`$HOME`, GPFS, snapshots); scratch `/scratch/gautschi/$USER` - via **`$RCAC_SCRATCH`** (NOT `$CLUSTER_SCRATCH`), **60-day** purge, not backed up; Depot/Fortress - for long-term; check with **`myquota`** / `findscratch`. **Quotas (25 GB home / 100 TB scratch) - are illustrative `myquota` output, not official — context files should tell the agent to RUN - `myquota`, not hardcode numbers.** +- **Filesystems (tech corrected by the human, authoritative 2026-07-15):** home `/home/$USER` + (`$HOME`) is **ZFS** (snapshotted); **`/depot` is GPFS**; scratch `/scratch/gautschi/$USER` via + **`$RCAC_SCRATCH`** (NOT `$CLUSTER_SCRATCH`) is **Lustre**, **60-day** purge, not backed up; + Fortress (HPSS) for archive; check with **`myquota`** / `findscratch`. *(This corrects research 03, + which read `/home` as GPFS from a generic snippet.)* **Quotas (25 GB home / 100 TB scratch) are + illustrative `myquota` output, not official — context files must tell the agent to RUN `myquota`, + not hardcode numbers.** - **Modules:** environment modules (Lmod — implied by `module spider`); `module avail/spider/load`; recommended GCC 14.1.0 + OpenMPI (exact OpenMPI version unstated). - **Login-node rule (quote):** "Do NOT run large, long, multi-threaded, parallel, or CPU-intensive @@ -125,13 +127,20 @@ clearly labeled — not as shipped fact. Confirm with the human before publish.* | **opencode** | `~/.config/opencode/opencode.json`; project `opencode.json`; `/etc/opencode/` | JSON | `permission` per-bash-pattern map (**last-match-wins → catch-all first**); no OS sandbox | `mcp` `type:"local"`/`"remote"` | `AGENTS.md` (native) + `instructions` | | **Warp** | **Desktop-app UI** (Settings > Agents); MCP JSON; Warp Drive rules | UI/JSON | Agent-Profile autonomy + regex allow/deny (Run-until-completion **ignores denylist**) | Settings > Agents > MCP (stdio+remote) | `AGENTS.md` (ALL-CAPS; `WARP.md` legacy) | -**Cross-cutting HPC realities:** OS sandboxes (bwrap/Docker/namespaces) are frequently **disabled on -shared login nodes** → treat the **permission/approval layer as primary**, point writable roots at -`/scratch/$USER`, deny `rm -rf`/`sudo`. The **enforcement points** for config management are the -*system/managed* files: Claude `/etc/claude-code/managed-settings.json`, Gemini -`/etc/gemini-cli/settings.json`. **Warp is the outlier — a local desktop GUI, cannot be installed on -a login node**; on-cluster coverage for Warp = "run it on your workstation and SSH in" (document -honestly, don't fabricate a login-node path). +**Cross-cutting HPC realities:** the harnesses' own OS sandboxes (Codex `bubblewrap`, Gemini +Docker/Podman, Linux namespaces) are frequently **unavailable on shared nodes** → treat the +**permission/approval layer as primary**, point writable roots at `$RCAC_SCRATCH`, deny +`rm -rf`/`sudo`. **Container model on these clusters is Apptainer, not Docker** (human, 2026-07-15): +RCAC's Apptainer config **auto bind-mounts `/home`, `/depot`, `/scratch`** into containers for +convenience, so containers give **limited protection** — the likeliest failure mode is an agent +*editing files* on those still-writable mounts (bind-mounts can be disabled if the user invokes +Apptainer explicitly). The **enforcement points** for config management are the *system/managed* +files: Claude `/etc/claude-code/managed-settings.json`, Gemini `/etc/gemini-cli/settings.json`. +**Read-only sanity commands should be *allow-listed*** so the agent runs `myquota`, `slist`, +`sfeatures`, `module list`, `module avail`, … eagerly without prompting (human, 2026-07-15). +**Warp is a local desktop GUI, cannot be installed on a login node — but it is RCAC's *recommended* +harness for most users**, so give it first-class treatment in the **local (MCP + SSH)** guidance: +"run it on your workstation and SSH in" (don't fabricate a login-node path). ## 6. Context-file design + mapping + prior art @@ -166,9 +175,15 @@ honestly, don't fabricate a login-node path). subdir) + one Gautschi chapter; canonical artifacts under `docs/snippets/agentic-ai/`. 2. Publish verbatim via `--8<--` fenced includes; verify by grepping built `site/` (check_paths footgun). -3. Author context files from verified Gautschi facts; instruct agents to run `myquota`/`slist` - rather than hardcoding volatile numbers; never emit the two doc-errors. -4. Present MCP `rcac-mcp` as prototype; the `cluster-mcp` plugin direction is planned/evolving and - needs human confirmation before publish. -5. Warp is local-only; be honest about the on-cluster gap. Enforcement lives in the system/managed +3. Author context files from verified Gautschi facts (incl. `/home` ZFS · `/depot` GPFS · + `/scratch` Lustre); instruct agents to run `myquota`/`slist`/`sfeatures`/`module list` rather than + hardcoding volatile numbers; never emit the two doc-errors. +4. Present all three MCP servers as **actively-developed working prototypes** (human-confirmed + framing); **explicitly call out the planned rename/re-architecture of `rcac-mcp`** toward the + plugin model — stated direction, not shipped fact. +5. **Warp is first-class and RCAC's recommended harness for most users** — tell the story of using it + *well* locally over SSH (it can't run on a login node). Enforcement lives in the system/managed settings files (Claude `/etc/claude-code/`, Gemini `/etc/gemini-cli/`). +6. Settings **allow-list read-only sanity commands** (run eagerly, no prompt) and **deny** + destructive ops; caution content reflects the **Apptainer** model (auto bind-mounts /home,/depot, + /scratch → limited protection; editing files is the real risk), not Docker sandboxing. From af033cb3b960927965bc6ed060fd17865726ff4d Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Wed, 15 Jul 2026 09:58:35 -0400 Subject: [PATCH 04/32] [harness] Fail the build-integrity gate on ERRORs, not just new warnings --- .agents/factory/bin/strict_check.py | 44 +++++++++++++++++++++++++++-- .agents/factory/invariants.md | 18 ++++++++---- .agents/factory/templates/PLAN.md | 5 +++- .agents/factory/templates/TECH.md | 5 ++++ AGENTS.md | 4 +-- 5 files changed, 65 insertions(+), 11 deletions(-) diff --git a/.agents/factory/bin/strict_check.py b/.agents/factory/bin/strict_check.py index 2b2ffb71..2e0589b2 100755 --- a/.agents/factory/bin/strict_check.py +++ b/.agents/factory/bin/strict_check.py @@ -16,9 +16,16 @@ Options: --baseline PATH baseline file (default: .agents/factory/strict-baseline.txt) +A `mkdocs build --strict` can also fail *hard* — a macro/Jinja `UndefinedError`, a config +error, a plugin crash — which surfaces as an `ERROR` line and/or a Python traceback, NOT a +`WARNING`. Those are never acceptable debt, so this gate fails on any `ERROR`/traceback +regardless of the baseline. (We deliberately do NOT rely on the pipeline exit code / `pipefail`: +under `--strict` mkdocs aborts non-zero on *any* warning, including the tolerated baseline ones, +so this script — not the exit code — must be the arbiter.) + Exit codes: - 0 no new warnings beyond the baseline (PASS) - 1 new warning(s) attributable to this change (STOP) + 0 no ERRORs and no new warnings beyond the baseline (PASS) + 1 a build ERROR/traceback, or new warning(s) attributable to this change (STOP) 2 usage / I/O error (e.g. missing baseline file) """ from __future__ import annotations @@ -30,10 +37,12 @@ from pathlib import Path # Public interface -__all__ = ["main", "extract_warnings", "load_baseline"] +__all__ = ["main", "extract_warnings", "extract_errors", "load_baseline"] # A mkdocs strict warning line looks like: "WARNING - ". _WARNING_RE = re.compile(r"^WARNING\s*-\s*(?P.*\S)\s*$") +# A hard failure looks like "ERROR - " (macros/Jinja error, config error, …). +_ERROR_RE = re.compile(r"^ERROR\s*-\s*(?P.*\S)\s*$") def extract_warnings(text: str) -> set[str]: @@ -46,6 +55,22 @@ def extract_warnings(text: str) -> set[str]: return out +def extract_errors(text: str) -> set[str]: + """Return hard-failure signals (ERROR lines + a crash traceback) in a build log. + + These indicate the build did not succeed (e.g. an unescaped ``{{ }}`` in published + config raising a macros ``UndefinedError``). They are never baseline debt. + """ + out: set[str] = set() + for line in text.splitlines(): + m = _ERROR_RE.match(line) + if m: + out.add(m.group("msg").strip()) + if "Traceback (most recent call last):" in text: + out.add("Python traceback in build output (the build crashed)") + return out + + def load_baseline(path: Path) -> set[str]: """Return the set of baseline warning messages (ignoring '#' and blanks).""" out: set[str] = set() @@ -78,9 +103,22 @@ def main(argv: list[str]) -> int: return 2 found = extract_warnings(text) + errors = sorted(extract_errors(text)) new = sorted(found - baseline) fixed = sorted(baseline - found) + if errors: + print(f"FAIL: {len(errors)} build ERROR(s) — the build did not succeed cleanly:") + for msg in errors: + print(f" ! {msg}") + print( + "\nERRORs are never baseline debt. Fix the build — a common cause here is an " + "unescaped '{{' / '{%' in published content hitting the macros/Jinja pass " + "(publish verbatim config via a '--8<--' include instead), a bad config, or a " + "plugin crash." + ) + return 1 + if new: print(f"FAIL: {len(new)} new mkdocs --strict warning(s) introduced by this change:") for msg in new: diff --git a/.agents/factory/invariants.md b/.agents/factory/invariants.md index 9c3bd982..53c56cca 100644 --- a/.agents/factory/invariants.md +++ b/.agents/factory/invariants.md @@ -93,11 +93,19 @@ the build breaks. ## §8 — Build integrity (hammerable: false) -`mkdocs build --strict` must introduce **no new warnings** versus `strict-baseline.txt`. -CI builds with `--verbose`, not `--strict`, so broken links and nav orphans would -otherwise ship silently. Gate with: -`.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py`. -This is a correctness gate, not a nicety — do not scope-hammer it. +`mkdocs build --strict` must introduce **no new warnings** versus `strict-baseline.txt`, +**and must not fail** (no build `ERROR`s or tracebacks). CI builds with `--verbose`, not +`--strict`, so broken links and nav orphans would otherwise ship silently. Gate with: +`.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py` +— the checker fails on new warnings **and** on any `ERROR`/traceback (e.g. an unescaped +`{{`/`{%` in published content hitting the macros pass). Do **not** add `pipefail`: under +`--strict` mkdocs aborts non-zero on *any* warning, including the tolerated baseline ones, so +`strict_check.py` — not the exit code — is the arbiter. + +**`--strict` does not catch everything.** A mistyped `pymdownx.snippets` include +(`--8<-- "…"`) fails **silently** (`check_paths: false` → an empty block, no warning). When a +page embeds a verbatim file, also **eyeball the render** or grep the built `site/` for a known +token. This is a correctness gate, not a nicety — do not scope-hammer it. ## §9 — Content-level accessibility, WCAG 2.1 (hammerable: false) diff --git a/.agents/factory/templates/PLAN.md b/.agents/factory/templates/PLAN.md index b523ff1a..e07ff184 100644 --- a/.agents/factory/templates/PLAN.md +++ b/.agents/factory/templates/PLAN.md @@ -69,8 +69,11 @@ name / Slurm flag / module?) or "how does the existing site handle X". How we will *prove* the pages are correct — this seeds each phase's `verify:` in `TECH.md`: - **Build integrity:** `.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py` - (no new warnings vs baseline). + (no new warnings **and no build ERRORs/tracebacks** vs baseline). - **Nav:** the new page(s) appear in `mkdocs.yml` and resolve (no "not in nav" warning). +- **Silent-failure guard:** a mistyped `--8<--` include renders an **empty** block with no + `--strict` warning (`check_paths: false`). For any page that embeds a verbatim file, add a + `grep -rq site//` to the phase `verify:` and eyeball the render. - **Render:** `mkdocs serve` and view the page(s) — layout, tabs, admonitions, macros expand. - **Front-matter:** matches the archetype (blog `` + allowed category + known author, etc.). - **Accessibility:** images have alt text; headings descend logically. diff --git a/.agents/factory/templates/TECH.md b/.agents/factory/templates/TECH.md index e7d0e374..6c71490a 100644 --- a/.agents/factory/templates/TECH.md +++ b/.agents/factory/templates/TECH.md @@ -65,6 +65,11 @@ content+state commit. Run from the repo root with the project env active. writing). A phase stuck `uphill` across drafts is a raised hand → escalate to the human. - `verify`: the exact command that proves the phase — prefer the `--strict` gate plus a nav/ front-matter check; add a `mkdocs serve` render check in the phase steps when layout matters. + The gate (`strict_check.py`) fails on new warnings **and** build ERRORs/tracebacks. `--strict` + does **not** catch a mistyped `--8<--` include (silent empty block), so for pages that embed a + verbatim file, chain a `grep -rq site//`. **Quote the whole `verify:` string** + (it contains pipes/colons); keep single-quotes *inside* it. Example gate + include check: + `.venv/bin/mkdocs build --strict 2>&1 | python3 .agents/factory/bin/strict_check.py && grep -q '.md' mkdocs.yml && grep -rq '' site//` ## Conventions (apply to every phase) diff --git a/AGENTS.md b/AGENTS.md index 857ac888..0b7845fa 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -121,8 +121,8 @@ review). The headlines: 5. **Links relative, assets absolute.** 6. **Front-matter per archetype** (blog `categories` from the allowed set; ``). 7. **Macros/snippets + Jinja escaping.** -8. **Build integrity** — `mkdocs build --strict` introduces no new warnings vs the baseline - (`hammerable: false`). +8. **Build integrity** — `mkdocs build --strict` introduces no new warnings **and no build + errors** vs the baseline (`hammerable: false`). 9. **Content-level WCAG 2.1 accessibility** (`hammerable: false`). 10. **Per-cluster parallelism** — keep the shared chapter set/order. 11. **HPC technical accuracy** — commands, flags, partitions, paths, specs must be correct From 8289056d65530d8db8277a9149bd316418c3ad5b Mon Sep 17 00:00:00 2001 From: Geoffrey Lentner Date: Wed, 15 Jul 2026 10:44:50 -0400 Subject: [PATCH 05/32] [harness] Standardize on a uv-synced .venv for all factory tooling --- .agents/factory/bin/_fsm.py | 18 ++++++++----- .agents/factory/templates/TECH.md | 8 +++--- .agents/settings.json | 12 ++++++--- .agents/skills/docs-draft/SKILL.md | 30 +++++++++++++-------- .agents/skills/docs-plan/SKILL.md | 13 +++++++-- AGENTS.md | 42 ++++++++++++++++++------------ 6 files changed, 79 insertions(+), 44 deletions(-) diff --git a/.agents/factory/bin/_fsm.py b/.agents/factory/bin/_fsm.py index 1a10faca..e000bf38 100644 --- a/.agents/factory/bin/_fsm.py +++ b/.agents/factory/bin/_fsm.py @@ -8,11 +8,14 @@ ``.agents/factory/methodology.md``). Requires PyYAML, which is a declared project dependency (``requirements.txt``) -and lives in the project virtual environment — NOT the system interpreter. Run -these scripts with the project env active, e.g. from the repo root:: +and lives in the uv-synced project virtualenv (``.venv``) — NOT the system +interpreter. Run these scripts via the venv Python from the repo root:: - source .venv/bin/activate # or: conda activate rcac-docs - python3 .agents/factory/bin/