diff --git a/.claude/agents/beamer-translator.md b/.claude/agents/beamer-translator.md index 775a39527..91616e829 100644 --- a/.claude/agents/beamer-translator.md +++ b/.claude/agents/beamer-translator.md @@ -2,7 +2,8 @@ name: beamer-translator description: Specialist agent for translating Beamer LaTeX slides to Quarto RevealJS. Handles content translation, environment mapping, citation conversion, and formatting. Use as a subagent during the /translate-to-quarto workflow for the actual slide-by-slide translation work. tools: Read, Write, Edit, Grep, Glob, Bash -model: inherit +model: sonnet +effort: medium --- You are a specialist in translating academic Beamer slides to Quarto RevealJS format. diff --git a/.claude/agents/claim-verifier.md b/.claude/agents/claim-verifier.md index e76ea2103..f27c3d98c 100644 --- a/.claude/agents/claim-verifier.md +++ b/.claude/agents/claim-verifier.md @@ -2,7 +2,8 @@ name: claim-verifier description: Fresh-context verifier for factual claims made by other agents or skills. Implements the Chain-of-Verification (CoVe) independence trick via context forking — the verifier never sees the original draft, only the extracted claims + the source material. Use when a skill has produced a draft that contains citations, numerical facts, named entities, or literature references that need hallucination-checking before returning to the user. tools: Read, Grep, Glob, WebFetch, WebSearch, Bash -model: inherit +model: opus +effort: high --- @@ -39,6 +40,11 @@ claims: text: "The method requires conditional parallel trends." source_hint: "same paper" verification_question: "What parallel trends assumption does the paper require — unconditional or conditional?" + author_alternative: "" # OPTIONAL. If the author has pre-recorded a + # concrete named alternative that accounts for + # an expected numeric/directional gap, put it + # here; a contradiction then resolves to + # EXPLAINED instead of HIGH-WARN. Blank = none. ``` ### Step 2: Answer each question independently @@ -67,6 +73,7 @@ Each per-claim finding now carries a **severity tier** (v1.9.0): - **HIGH-WARN** — fabricated reference (cited paper does NOT exist at the named venue/year), direct contradiction between draft and source, or `not_found` retrieval that you interpret as a hallucinated citation. These block `/commit` via `/verify-claims`. - **MED-WARN** — transient infrastructure failure: DOI resolver timed out, partial PDF read, paywall the cache normally bypasses. The author should re-run the verification later or supply a local copy. - **LOW-WARN** — source genuinely inaccessible (paywalled with no cache hit, private dataset, pre-print server transient). Surface but do not gate-refuse — the claim may still be correct; the verifier just can't independently confirm. +- **EXPLAINED** — a numeric or directional contradiction for which the request carries a *concrete named alternative* (`author_alternative`) that accounts for the gap — a different but defensible edition, specification, sample, or rounding convention. Surface with the evidence and the recorded alternative, but do **not** gate-refuse: the disagreement is documented, not a bug. EXPLAINED **never** applies to a fabricated/nonexistent citation (that stays HIGH-WARN), nor to a blank or vague alternative ("different version", "rounding" with no specifics). ```markdown ## Claim Verification Report @@ -80,7 +87,7 @@ Each per-claim finding now carries a **severity tier** (v1.9.0): | ID | Claim (draft) | Independent answer | Evidence | Match? | Tier | |----|--------------|---------------------|----------|--------|------| -| C1 | [quoted claim] | [what source says] | [quote + loc] | yes / partial / no / cannot-verify | — / LOW / MED / HIGH | +| C1 | [quoted claim] | [what source says] | [quote + loc] | yes / partial / no / cannot-verify | — / LOW / MED / HIGH / EXPLAINED | ### HIGH-WARN (gate-refuse `/commit`) @@ -102,12 +109,12 @@ Each per-claim finding now carries a **severity tier** (v1.9.0): - "I retrieved the source and it contradicts the claim" → HIGH-WARN - "I cannot retrieve the source because of a transient failure" → MED-WARN - "I cannot retrieve the source because it's genuinely inaccessible" → LOW-WARN -- A cited paper that does NOT exist (no DOI, no arXiv id, no venue search hit) is **always HIGH-WARN** — this is the canonical hallucination signature. Do not soften to MED-WARN on the grounds that "maybe my search missed it." -- A numerical contradiction (draft says X, source says Y, X ≠ Y within rounding) is **always HIGH-WARN**. -- A directional contradiction (draft says "positive effect", source says "negative effect") is **always HIGH-WARN**. +- A cited paper that does NOT exist (no DOI, no arXiv id, no venue search hit) is **always HIGH-WARN** — this is the canonical hallucination signature. Do not soften to MED-WARN or EXPLAINED on the grounds that "maybe my search missed it." This is the hard floor: a fabricated citation is never downgradable. +- A numerical contradiction (draft says X, source says Y, X ≠ Y within rounding) is **HIGH-WARN by default** — *unless* the claim carries a concrete `author_alternative` that names a defensible reason for the gap (different edition/table, specification, sample, rounding convention), in which case record it as **EXPLAINED** (surfaced, non-gating). A blank or vague `author_alternative` does not soften — stay HIGH-WARN. +- A directional contradiction (draft says "positive effect", source says "negative effect") is **HIGH-WARN by default** — same EXPLAINED escape: only a concrete named alternative for that claim downgrades it; otherwise HIGH-WARN. - A paraphrase mismatch where the draft's gloss is a reasonable summary of the source — not HIGH-WARN. Flag as `partial` with no tier (or LOW if you want to surface the gloss difference). -Be conservative on HIGH-WARN. It blocks `/commit`. False positives erode the gate's authority; false negatives let known-bad claims ship. +Be conservative on HIGH-WARN. It blocks `/commit`. False positives erode the gate's authority; false negatives let known-bad claims ship. The EXPLAINED escape exists only to stop a *defensible, author-documented* disagreement from gate-blocking — it is never a way to wave through a fabricated citation or an undocumented contradiction. ## What you DO NOT do diff --git a/.claude/agents/domain-referee.md b/.claude/agents/domain-referee.md index 67512c767..bbfeb79cf 100644 --- a/.claude/agents/domain-referee.md +++ b/.claude/agents/domain-referee.md @@ -2,7 +2,8 @@ name: domain-referee description: Substantive referee for a manuscript. Reviews contribution, literature positioning, substantive argument, external validity, and journal fit. Calibrated to a target journal and primed with a disposition + pet peeves by the editor agent. Used by `/review-paper --peer`. tools: Read, Grep, Glob -model: inherit +model: opus +effort: high --- ` tables in `README.md` and `CLAUDE.md` must have exactly one data row per skill on disk. Adding a skill without a row fails the gate. + +REMIND the user to: + +1. Add a row to the **CLAUDE.md** "Skills Quick Reference" table: `` | `/ [args]` | | ``. +2. Add a row to the **README.md** skills table: `` | `/` | | ``. +3. Run `./scripts/check-surface-sync.sh` and `python3 scripts/check-skill-integrity.py` — both must exit 0. + +Print the two ready-to-paste rows so the user can drop them in. + +## Output / report format + +- A new file at `.claude/skills//SKILL.md`. +- A chat summary: the resolved name, the design brief, the gate results (integrity + a reminder that surface-sync still needs the two table rows), and the two paste-ready table rows. +- With `--dry-run`: emit the proposed SKILL.md to chat only and write nothing. + +## Exit behavior + +- **Skill written, gates green:** exit 0 with the path, the two table rows, and the explicit "now add those rows + run the two checks" reminder. +- **Name collision or non-kebab-case:** stop in Phase 0 with the conflict named; write nothing. +- **`check-skill-integrity.py` reports P0/P1:** fix in-place and re-run before returning; never hand back a skill that fails its own gate. +- **`--dry-run`:** print the draft, write nothing, exit 0. + +## Flags + +- `--from-learn` — Seed the interview from an existing `/learn`-style stub (or the current session's discovery) and upgrade it into a full convention-compliant skill rather than starting blank. +- `--dry-run` — Produce the SKILL.md content in chat for review without writing it to disk or touching any surface table. + +## Cross-references + +- [`templates/skill-template.md`](../../../templates/skill-template.md) — the canonical structure, frontmatter-field reference, and the "when to set `disable-model-invocation`" rule this skill follows. +- [`.claude/skills/learn/SKILL.md`](../learn/SKILL.md) — capture a session discovery (the lighter sibling); `--from-learn` upgrades its output. +- [`.claude/skills/coauthor-brief/SKILL.md`](../coauthor-brief/SKILL.md) — a gold-standard skill to imitate (interview → write → flags → exit-behavior shape). +- [`.claude/rules/orchestrator-protocol.md`](../../rules/orchestrator-protocol.md) — why the interview collects all interactivity *before* writing. +- `.claude/scripts/` and `scripts/check-skill-integrity.py` / `scripts/check-surface-sync.sh` — the gates this skill is built to pass on the first try. + +## What this skill does NOT do + +- **Capture a session discovery** — that is [`/learn`](../learn/SKILL.md). This skill designs an interface; `/learn` records a finding. +- **Edit the README / CLAUDE.md surface tables for you.** It *prints* the two rows and reminds you; registering them (and re-running `./scripts/check-surface-sync.sh`) is a deliberate human step so the surface gate is never silently satisfied. +- **Write agents, rules, or hooks.** It scaffolds a skill only; an agent goes in `.claude/agents/`, a rule in `.claude/rules/`. +- **Commit anything.** Branch / PR / merge is [`/commit`](../commit/SKILL.md)'s job. diff --git a/.claude/skills/power-analysis/SKILL.md b/.claude/skills/power-analysis/SKILL.md new file mode 100644 index 000000000..5c9583d3f --- /dev/null +++ b/.claude/skills/power-analysis/SKILL.md @@ -0,0 +1,129 @@ +--- +name: power-analysis +description: Compute statistical power, required sample size, and minimum detectable effect (MDE) for a study design, then write a registry-ready power section. Handles two-arm RCTs (with clustering / ICC and unequal allocation), multiple-arm corrections, and a simulation-based power option for non-standard designs (DiD/event-study, IV, panel). Use when user says "power analysis", "power calculation", "MDE", "minimum detectable effect", "how big a sample do I need", "is my study powered", "power for an RCT", or when /preregister needs a power section for an experiment. Produces a power/MDE table, power curves, and a methods paragraph to paste into a preregistration. +author: Claude Code Academic Workflow +version: 1.0.0 +argument-hint: "[--mode mde|n|power] [--design rct|cluster|multiarm|sim] [--input ]" +disable-model-invocation: true +allowed-tools: ["Read", "Write", "Edit", "Bash", "Task"] +effort: high +--- + +# `/power-analysis` — Power / MDE for study design + +Compute the three interlocking quantities of an ex-ante design calculation — **power**, **required N**, and **minimum detectable effect (MDE)** — and emit a power section the user can paste straight into a preregistration. Analytical for standard designs; simulation-based (reusing the `/simulation-study` harness pattern) for non-standard ones. + +**Core principle:** a power calculation is a *design-time commitment made before the data exist*. Fix any two of {effect size, N, power} and solve for the third; never back out a "power" number from a realised estimate (that is post-hoc power, and it is uninformative — see "What this skill does NOT do"). + +## When to use + +- **Before launching an RCT / field / survey experiment** — to choose N (or clusters) for a target MDE at 80–90% power. +- **Invoked by [`/preregister`](../preregister/SKILL.md) for RCTs** — the AEA RCT Registry and most IRBs require a power/MDE justification; `/preregister`'s `aea-rct` style calls this skill to fill that section. +- **During R&R** — when a referee asks "was this study adequately powered to detect the effect you claim?" +- **Designing a Monte Carlo** — to set `R` and sample sizes before handing off to `/simulation-study`. + +## Inputs + +`$ARGUMENTS` may carry flags; missing pieces are elicited in Phase 0. + +- `--mode mde|n|power` — solve for MDE given N+power, N given MDE+power, or power given N+MDE. Default `mde`. +- `--design rct|cluster|multiarm|sim` — two-arm RCT, clustered RCT (ICC), multiple arms, or simulation-based. Default inferred from the elicited design. +- `--input ` — a spec from [`/interview-me`](../interview-me/SKILL.md) (under `quality_reports/specs/`) to pull the RQ, outcome, and design from. + +## Workflow + +### Phase 0 — Elicit the design + +Gather the design parameters; **ask once** for anything missing rather than fabricating. Required: + +- **Estimand & test:** primary outcome, one- vs two-sided test, `alpha` (default 0.05), and whether the target is a difference in means, a proportion, or a regression coefficient. +- **Two of {effect size, N, power}:** the effect as a raw difference *and* in standardized units (Cohen's d = effect / SD) — record both; `power` default 0.80. +- **Baseline mean and SD** (or baseline proportion for a binary outcome) — needed to translate raw ↔ standardized effects. +- **Allocation:** treated:control ratio (default 1:1; unequal allocation costs power — note it). +- **Clustering:** if randomization is at a group level (village, school, clinic), the **ICC** (ρ), the average cluster size (m), and number of clusters. Compute the design effect `DEFF = 1 + (m − 1)·ρ` and the effective N. +- **Multiplicity:** number of arms / primary outcomes; the correction (Bonferroni, Holm, or none) and whether power is per-comparison or familywise. + +Echo a **Pre-Flight Report** (design, the two fixed quantities, the one being solved for, alpha, power, allocation, ICC/clusters, multiplicity) before computing. If the estimand or the SD source is ambiguous, stop and ask. + +### Phase 1 — Analytical power (standard designs) + +For two-arm RCTs, clustered RCTs, and multi-arm comparisons, compute analytically. Prefer R `pwr` / `WebPower` (or a closed-form `power.t.test` / `power.prop.test`); for clustered designs inflate variance by `DEFF`, or use `pwr` on the effective N. Stata users: `power twomeans` / `power twoproportions` / `power, cluster`; Python: `statsmodels.stats.power`. Emit a short script to `scripts/R/power_.R` (or `.do` / `.py`) so the calc is reproducible, not a one-off console number. + +- **MDE mode:** `MDE = (z_{1−α/2} + z_{1−β}) · SE(effect)`, where `SE` is built from the SD, N, allocation, and `DEFF`. Report MDE in raw and standardized units. +- **N mode:** invert the above for total N (and #clusters when clustered) given the target MDE. +- **Power mode:** given N and a hypothesized effect, return achieved power. +- **Multi-arm:** divide `alpha` by the **number of comparisons in the family** `m` (Bonferroni `alpha/m`): `m = K−1` for all-vs-control, `m = K(K−1)/2` for all-pairwise. Report per-comparison *and* familywise power. + +Sweep a grid (N or #clusters × effect size) so Phase 3 can draw a power curve and an MDE-vs-N curve. + +### Phase 2 — Simulation-based power (non-standard designs) + +When the design is **not** a clean two-arm comparison — DiD / staggered event-study, IV / 2SLS (weak-instrument-aware), panel with serial correlation, a non-normal or censored outcome, or any estimator with no closed-form SE — switch to simulation. **Reuse the `/simulation-study` harness** exactly (see [`simulation-study`](../simulation-study/SKILL.md) and [`.claude/rules/simulation-conventions.md`](../../rules/simulation-conventions.md)): + +1. **Seeded, parameterized DGP** that embeds the hypothesized effect (and the null DGP for size). `set.seed(YYYYMMDD)` once; L'Ecuyer streams if parallel. +2. **Estimator** = the one you will actually use on the real data (e.g. `fixest::feols` two-way FE, `did::att_gt`, `AER::ivreg`), returning `est, se, ci, p, reject`. +3. **Power = share of reps rejecting H0** at `alpha`; **size = rejection rate under the null DGP** (verify it is near nominal before trusting power). Report each with its **Monte Carlo SE** = `sqrt(p(1−p)/R)`. +4. Sweep N (or #clusters / #periods) to trace the power curve; save the raw per-rep tibble via `saveRDS()` to `scripts/R/_outputs/`. + +A simulated power number without an MCSE, or without a verified size check, is not yet an answer. + +### Phase 3 — Write the power section + +Produce the deliverables under `quality_reports/power/`: + +- **`power_.md`** — a table and a methods paragraph (below). +- **`power_curve_.png`** — power vs N (and/or MDE vs N), with reference lines at the target power and the design's planned N. +- The reproducible script under `scripts/R/` (or `.do` / `.py`). + +```markdown +# Power Analysis: +**Date:** YYYY-MM-DD · **Design:** · **Method:** + +| Quantity | Value | +|---|---| +| alpha (sided) | 0.05 (two-sided) | +| Target power | 0.80 | +| Baseline mean (SD) | () | +| Allocation (T:C) | 1:1 | +| ICC / cluster size / #clusters | <ρ> / / (DEFF = <…>) | +| Total N (analysis sample) | | +| **MDE (raw / standardized)** | **<Δ> / ** | +| Achieved power at planned N | <…> (± MCSE <…> if simulated) | + +## Methods paragraph (paste into preregistration) +> Assuming a baseline outcome mean of (SD ), 1:1 allocation, and a two-sided +> test at α = 0.05, a total sample of [ clusters of , ICC = <ρ>] yields 80% +> power to detect a minimum effect of <Δ> ( SD). [Simulation: under the hypothesized +> DGP,

% of replications rejected H0 (MCSE <…>); size under the null was <…>.] +``` + +### Phase 4 — Handoff + +If invoked by `/preregister`, return the methods paragraph + MDE row for the preregistration's power section. If standalone, print the save paths and remind the user the MDE is a *design commitment* to record before data collection. + +## Exit behavior + +- **Computation succeeds:** exit 0; print the MDE / N / power result, the save paths, and (for simulation mode) the size-check value next to power. +- **Under-identified design (only one of {effect, N, power} supplied) or ambiguous SD source:** halt in Phase 0 with a single specific question — never guess the SD or the ICC. +- **Simulation size check fails** (empirical size far from nominal under the null DGP): report power as **UNRELIABLE** and surface the size value; the estimator/DGP must be fixed before the power number is trustworthy. + +## Flags + +- `--mode` `` — What to solve for: minimum detectable effect, required N, or achieved power. +- `--design` `` — Design family — two-arm RCT, clustered/ICC, multi-arm with corrections, or simulation-based for non-standard designs. +- `--input` `` — Path to an `/interview-me` spec or preregistration draft to read design parameters from. + +## Cross-references + +- [`.claude/skills/preregister/SKILL.md`](../preregister/SKILL.md) — invokes this skill to fill the power/MDE section of an `aea-rct` (and OSF) preregistration; this skill returns the methods paragraph. +- [`.claude/skills/simulation-study/SKILL.md`](../simulation-study/SKILL.md) — the Monte Carlo harness Phase 2 reuses (seeded DGP, estimator grid, % rejecting H0). +- [`.claude/rules/simulation-conventions.md`](../../rules/simulation-conventions.md) — the simulation contract (truth from DGP, MCSE, size-under-the-null) that Phase 2 must honor. +- [`.claude/skills/data-analysis/SKILL.md`](../data-analysis/SKILL.md) · [`.claude/skills/stata-replication/SKILL.md`](../stata-replication/SKILL.md) — where the realised analysis (and its actual estimator/SE) lives; the power calc should use the same estimator. +- [`.claude/rules/confidential-data.md`](../../rules/confidential-data.md) — when baseline mean/SD/ICC are taken from restricted-access data, disclosure-avoidance limits apply; cite published or pilot moments rather than embedding raw confidential statistics in the (externally-uploaded) preregistration. + +## What this skill does NOT do + +- **Post-hoc / observed power.** It refuses to compute "the power we had to detect our estimate" from a realised result — that is a deterministic function of the p-value and tells you nothing. Power is ex-ante only. +- **Pick your effect size for you.** The MDE is *your* design commitment; the skill computes consequences of an assumed effect (from theory, a pilot, or a meta-analysis), it does not invent a plausible one. +- **Submit to a registry.** Like `/preregister`, it writes a document; the user uploads it. +- **Replace `/simulation-study`.** Phase 2 borrows the harness for a single power question; a full bias/RMSE/coverage study is `/simulation-study`'s job. diff --git a/.claude/skills/preregister/SKILL.md b/.claude/skills/preregister/SKILL.md index a4f9ef4ea..789449acf 100644 --- a/.claude/skills/preregister/SKILL.md +++ b/.claude/skills/preregister/SKILL.md @@ -90,7 +90,7 @@ Refuse to mark the document "ready" if any of these fails: - **Hypothesis directionality.** Each hypothesis must contain a direction ("higher than", "increases", "negatively predicts", "no effect" is acceptable as a directional claim under equivalence-testing). Reject "is associated with" without a sign. - **Estimator named.** Analysis plan names a specific estimator (OLS, logit, fixest::feols, lme4::lmer, ATT difference-in-means …) and a primary outcome variable. "Regression" alone is insufficient. -- **Sample plan numeric.** Target N, stopping rule, or power-calc target appear. "As many as possible" is not a sample plan. +- **Sample plan numeric.** Target N, stopping rule, or power-calc target appear. "As many as possible" is not a sample plan. For RCTs and prospective designs, run [`/power-analysis`](../power-analysis/SKILL.md) to produce the MDE / required-N and a ready-to-paste power paragraph for this field. - **Exclusions ex ante.** Outlier and exclusion rules are stated *before* the data is seen ("we will exclude observations with completion time < 1 minute"). Vague "we'll deal with outliers" fails. - **Internal consistency.** If the design is randomised, the unit of randomisation matches the unit of analysis OR the analysis plan addresses clustering. If observational, identification strategy is stated. @@ -126,6 +126,7 @@ Include the registry URL: OSF → `osf.io/registries`, AsPredicted → `aspredic - `templates/preregistration-template.md` — the three style templates this skill consumes. - `templates/requirements-spec.md` — MUST/SHOULD/MAY annotation language re-used here. - `.claude/skills/interview-me/SKILL.md` — produces the spec this skill consumes via `--input`. +- `.claude/skills/power-analysis/SKILL.md` — supplies the MDE / required-N + power paragraph for the sample-plan field (RCTs). - `.claude/skills/verify-claims/SKILL.md` — Phase 5 invokes this for citation post-flight. - `.claude/references/discipline-cards.md` — field defaults that drive `--style` selection. - `.claude/rules/replication-protocol.md` — preregistration is the *forward* commitment; replication-protocol is the *backward* contract. diff --git a/.claude/skills/prompt-only/SKILL.md b/.claude/skills/prompt-only/SKILL.md deleted file mode 100644 index 96f75d8d3..000000000 --- a/.claude/skills/prompt-only/SKILL.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -name: prompt-only -description: Reformat an informal or dictated request into a structured prompt (Role / Task / Context / Constraints / Output format / Bookend) and emit it as a reusable artifact — does NOT execute. Companion to `/prompt`. Use when user says "format this prompt", "give me a clean version of this ask", "save this prompt for later", "I want a reusable prompt", or wants a prompt to use elsewhere (different model, different conversation, recurring task). -author: Claude Code Academic Workflow -version: 1.0.0 -argument-hint: "[informal-text-or-pointer-to-text] [depth:light|standard|deep] [--save path]" -allowed-tools: ["Read", "Write", "Glob"] ---- - - - -# `/prompt-only` — format input into a structured prompt without executing - -Same six-section skeleton as [`/prompt`](../prompt/SKILL.md), but **does not execute**. Returns the formatted prompt as a reusable artifact. - -## When to use - -- You want a prompt you'll run later — in a different conversation, against a different model, or as the basis for a recurring task. -- You're drafting a prompt template (e.g., for a future skill, a co-author, a teaching exercise). -- The execution itself should happen elsewhere (a long batch, a different tool, an offline workflow). - -## When NOT to use - -- You want the formatted prompt run immediately. Use [`/prompt`](../prompt/SKILL.md). -- You want a research project formalised. Use [`/interview-me`](../interview-me/SKILL.md). - -## How it works - -### Step 1: Read the input - -`$ARGUMENTS` is the informal text or a pointer to a file. Strip a trailing `depth:` token and `--save` if present. - -### Step 2: Calibrate depth - -Same heuristic as `/prompt` (see [`.claude/references/prompt-formatting-core.md`](../../references/prompt-formatting-core.md)): - -- Light: < 40 words, no domain jargon, "quick" / "small". -- Standard: 40–200 words, or 1+ domain-specific terms. -- Deep: > 200 words, or specific paper/dataset/submission mentioned, or "thorough" / "for submission". - -User override honoured via `depth:light|standard|deep`. - -### Step 3: Emit the formatted prompt - -Use the six-section skeleton from [`.claude/references/prompt-formatting-core.md`](../../references/prompt-formatting-core.md): - -- Light → Role + Task + Output format + Bookend. -- Standard → All six + Assumptions block. -- Deep → All six + Assumptions + Investigation pre-step. - -Output the formatted prompt **in a fenced code block** so the user can copy it cleanly. - -### Step 4 (optional): Save - -If `--save ` was supplied, write the formatted prompt to that path. If the path is a directory, default the filename to `quality_reports/prompts/YYYY-MM-DD_.md`. The slug is derived from the first ~8 words of the Task section. - -### Step 5: Return - -Standard output: - -- The formatted prompt in a code block. -- If saved: a one-line "saved to " confirmation. - -**No execution.** That's the whole point. - -## Boundary - -`/prompt-only` is for **reusable prompt artifacts**. It does not produce the output of the prompt — that's `/prompt`'s job (single-run) or your job (when you run the saved prompt elsewhere). - -## Anti-pattern - -Do not use `/prompt-only` when you want immediate execution. The friction of "format then run separately" is wasteful in interactive contexts. Reach for `/prompt` instead. - -## Output - -- The formatted prompt (fenced code block). -- Optionally: "saved to " confirmation. -- No execution result. diff --git a/.claude/skills/prompt/SKILL.md b/.claude/skills/prompt/SKILL.md deleted file mode 100644 index d78bd6ac4..000000000 --- a/.claude/skills/prompt/SKILL.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -name: prompt -description: Reformat an informal or dictated request into a structured prompt (Role / Task / Context / Constraints / Output format / Bookend) at Light / Standard / Deep depth, then execute it immediately. Use when user has a conversational ask that would benefit from being made specific before Claude runs it, or says "prompt", "structure this", "format this ask", "reshape this request", "make this rigorous". Single-shot input shaping — distinct from `/interview-me` (multi-turn project specification). -author: Claude Code Academic Workflow -version: 1.0.0 -argument-hint: "[informal-text-or-pointer-to-text] [depth:light|standard|deep]" -allowed-tools: ["Read", "Write", "Glob", "Grep", "Task", "WebSearch", "WebFetch"] ---- - - - -# `/prompt` — format informal input into a structured prompt, then execute - -Take an informal or dictated request, reformat it into a structured six-section prompt (Role / Task / Context / Constraints / Output format / Bookend) at the appropriate depth, **show the formatted prompt to the user**, and execute it. - -## When to use - -- The user dictates a long, meandering ask and wants a clean run. -- The request mentions specific domain terminology and would benefit from explicit Constraints + Output-format sections before execution. -- The user has tried an unformatted run and got a generic answer; reformatting before re-running is cheaper than iterating in dialogue. - -## When NOT to use - -- The request is already a clean, specific ask. Skip the formatter; run the request directly. -- The request is for a multi-turn project specification (hypotheses, identification strategy, data needs). Use [`/interview-me`](../interview-me/SKILL.md) instead. -- The request is for ad-hoc fact-checking. Use [`/verify-claims`](../verify-claims/SKILL.md). -- The request is a side question that shouldn't enter conversation history. Use the built-in `/btw` instead. - -## How it works - -### Step 1: Read the input - -`$ARGUMENTS` is either the informal text itself or a pointer to a file containing it (`.md`, `.txt`, `.qmd`). If it's a file, read it. Strip a trailing `depth:light|standard|deep` token if present and record the override. - -### Step 2: Calibrate depth - -Default heuristic (from [`.claude/references/prompt-formatting-core.md`](../../references/prompt-formatting-core.md)): - -- **Light** — input < 40 words, no domain-specific terminology, or explicit "quick" / "small". -- **Standard** — input 40–200 words, OR contains 1+ domain-specific terms (econometrics / political science / methods jargon). -- **Deep** — input > 200 words, OR mentions a specific paper / dataset / submission target, OR says "thorough" / "detailed" / "for submission". - -User override (`depth:light|standard|deep`) is honoured. - -### Step 3: Emit the formatted prompt - -Use the structured-prompt skeleton from [`.claude/references/prompt-formatting-core.md`](../../references/prompt-formatting-core.md): - -- **Light:** Role + Task + Output format + Bookend. -- **Standard:** All six sections + Assumptions/Rationale block (2–4 bullets). -- **Deep:** All six sections + Assumptions + Investigation pre-step (one sentence describing what Claude should research/read/verify before producing the output). - -**Always show the formatted prompt to the user before executing.** Use a fenced code block. The user sees what you reshaped their ask into and can correct an inferred assumption with one message. - -### Step 4: Execute the formatted prompt - -Immediately after emitting the formatted prompt, **execute it** — run the request as if the user had typed the formatted version. Use the appropriate tools (Read, Grep, WebFetch, etc.) per the formatted prompt's Constraints. - -If the formatted Bookend asked a clarifying question and the answer would change the output, surface that question first and wait for the answer. - -### Step 5: Return the output - -Standard prose, table, or code output per the formatted prompt's Output-format section. - -## Boundary with `/interview-me` - -`/prompt` is **input shaping** (single-shot). `/interview-me` is **project specification** (multi-turn Socratic). They don't compete — use `/prompt` when you want a well-formed ask; use `/interview-me` when you want a well-formed research project. - -## Companion: `/prompt-only` - -[`/prompt-only`](../prompt-only/SKILL.md) is the same skill **without** Step 4 — it emits the formatted prompt as a reusable artifact and stops. Use it when you want a prompt you'll run later (e.g., in a different conversation, against a different model, or as the basis for a recurring task). - -## Anti-pattern - -Do not run `/prompt` on a request that is already specific. If the user has already typed a six-section prompt manually, reformatting it via `/prompt` adds friction without value. - -## Output - -- The formatted prompt (visible in the conversation). -- The execution result (the answer to the formatted prompt's Task). -- Optionally: a one-sentence note if an Assumptions item turned out to be wrong during execution, so the user can correct for the next run. diff --git a/.claude/skills/qa-quarto/SKILL.md b/.claude/skills/qa-quarto/SKILL.md index 92bc333ee..be9afac4b 100644 --- a/.claude/skills/qa-quarto/SKILL.md +++ b/.claude/skills/qa-quarto/SKILL.md @@ -49,9 +49,13 @@ If not APPROVED, launch `quarto-fixer` agent to apply fixes (Critical → Major Re-launch critic to verify fixes. Loop back to Phase 2 if needed. -## Iteration Limits +## Iteration Limits — loop-until-dry -Max 5 fix rounds. After that, escalate to user with remaining issues. +This is the **loop-until-dry** primitive from [`orchestrator-protocol.md`](../../rules/orchestrator-protocol.md): the critic returns `FINDING`s (the hard-gate table is the CRITICAL roll-up, per [`orchestration-schemas.md`](../../references/orchestration-schemas.md)); the loop **converges when a round adds 0 new CRITICAL/MAJOR** findings (deduped on `location`+`finding`), not at a fixed round count. + +- **Fallback cap:** 5 rounds bounds a non-converging loop, then escalate to the user with remaining issues. +- **Two-strikes:** the same gate failing in rounds N and N+2 is flagged for the user, not patched again ([`summary-parity.md`](../../rules/summary-parity.md)). +- APPROVED iff every hard gate passes (zero CRITICAL). ## Final Report diff --git a/.claude/skills/replication-package/SKILL.md b/.claude/skills/replication-package/SKILL.md new file mode 100644 index 000000000..4528de9ca --- /dev/null +++ b/.claude/skills/replication-package/SKILL.md @@ -0,0 +1,135 @@ +--- +name: replication-package +description: Assemble a submission-ready replication package to the AEA Data and Code Availability Standard (DCAS) / openICPSR / Social Science Reproduction Platform expectations — standard replication README, dataset manifest, computational-requirements capture, a Table/Figure → script:line map, and a confidential-data deposit plan. Use when user says "build the replication package", "prepare the openICPSR deposit", "make the AEA data and code package", "DCAS compliance", "assemble the deposit for the journal", or after a paper is accepted and the journal's data editor needs the package. NOT a numeric verifier — it calls /audit-reproducibility to confirm claims reproduce before packaging. +argument-hint: "[manuscript path] [outputs-dir] (outputs-dir defaults to scripts/R/_outputs/)" +allowed-tools: ["Read", "Grep", "Glob", "Write", "Bash", "Task"] +effort: high +--- + +# Replication Package + +Produce the deposit an economist hands a journal at acceptance: a directory tree (`data/`, `code/`, `output/`, `README`) plus a DCAS compliance checklist, built to the [AEA Data and Code Availability Standard](https://datacodestandard.org/), openICPSR deposit expectations, and the [Social Science Reproduction Platform](https://www.socialsciencereproduction.org/) reproduction protocol. This skill **moves the repo from auditing reproducibility to producing the deposit** — `/audit-reproducibility` proves the numbers; this skill packages everything a third party needs to regenerate them from scratch. + +**Core principle:** the package is reproducible by a stranger with the data and the README — no tacit knowledge, no "ask the author" steps. Every table and figure maps to the exact script and line that produces it. + +## When to use + +- **At acceptance.** The journal's data editor (AEA, REStud, JPE, EJ, ...) requests a DCAS-compliant deposit before the paper is typeset. +- **Before an openICPSR / Zenodo / Dataverse upload.** Build the tree and README once, locally, before the web upload. +- **Pre-submission dry run.** Catch the "I never wrote down where Table 3 comes from" gap while it is cheap to fix. +- **Confidential-data papers.** Produce the access-restricted-data note and a runnable-on-restricted-data package even when the data itself cannot be deposited. + +## Inputs + +- `$0` — path to the manuscript (`.tex`, `.qmd`, `.md`, `.pdf`). Required (the source of the Table/Figure inventory). +- `$1` — outputs directory. Defaults to `scripts/R/_outputs/`. Recognised alternatives: `scripts/stata/_outputs/`, `scripts/python/_outputs/`, `_targets/objects/`. + +## Workflow + +### Phase 0: Pre-flight — detect language(s) and outputs + +1. Detect the analysis language(s) by scanning for `scripts/R/*.R` (+ `renv.lock` / `DESCRIPTION`), `scripts/stata/*.do`, `scripts/python/*.py` (+ `requirements.txt` / `environment.yml` / `pyproject.toml`). A project may be **polyglot** — record all detected languages. +2. Locate the outputs directory (`$1`) and the one-command entry point (`00_run_all.R`, `99_run_all.do`, `run.py`, `Makefile`). If none exists, flag it — DCAS requires a single master script. +3. If `quality_reports/passports/.yaml` exists, load it; its `claims:` entries are the authoritative Table/Figure → `source_file:source_line` map for Phase 1. + +### Phase 1: Generate the standard replication README + +Write `replication_package/README.md` (the AEA template, fields below). Leave a `[FILL]` marker on any field you cannot infer — never fabricate a data source or license. + +- **Overview / paper citation** — title, authors, abstract one-liner. +- **Data Availability Statement** — for each dataset: public / restricted / proprietary, and whether it is redistributed in the package. This is the single most-rejected DCAS field; be explicit. +- **Dataset manifest** — a table, one row per file: `filename | description | source (URL/citation) | access (public / DUA / purchase) | license | provided in package? (Y/N)`. +- **Computational requirements** — OS, software + versions (R / Stata / Python), key packages, approximate runtime, RAM, any HPC/cluster need. +- **Step-by-step run instructions** — the single master-script invocation, then the expected outputs. +- **Table/Figure → script:line map** — one row per exhibit: `Exhibit | Program | Line | Output file`. Read from the passport if present; otherwise grep the manuscript for `\input{}` / `\includegraphics{}` and trace each to the producing script. This map is what a reproducer follows; it is the heart of the package. + +### Phase 2: Capture the computational environment + +Generate the dependency lockfile(s) and an environment snapshot for each detected language. Prefer [`/capture-environment`](../capture-environment/SKILL.md) if available; otherwise produce them directly: + +- **R** — `renv::snapshot()` → `renv.lock`; `sessionInfo()` → `output/sessionInfo.txt`. +- **Python** — `pip freeze` → `requirements.txt` (or export the conda `environment.yml`); record `python --version`. +- **Stata** — `creturn list` / `about` → `output/stata_version.txt`; confirm every `.do` pins `version NN` (per [`stata-code-conventions.md`](../../rules/stata-code-conventions.md)). +- **Container (recommended by DCAS for non-trivial setups)** — scaffold a `Dockerfile` pinning the base image + language version. + +### Phase 3: Confirm claims reproduce before packaging + +Run [`/audit-reproducibility`](../audit-reproducibility/SKILL.md) `$0 $1` (passport-aware if the YAML exists). + +- **Any FAIL** (out of tolerance, no named alternative) → **block**: do not assemble a package around numbers that do not reproduce. Surface the failing claims and stop. +- **EXPLAINED** (out of tolerance with a recorded named alternative) → allowed; carry the note into the README's known-discrepancies section. +- **All PASS / PASS + EXPLAINED** → proceed to Phase 4. + +### Phase 4: Assemble the tree + DCAS checklist + +Create the deposit skeleton (copy/symlink real files where they exist; leave `[FILL]` placeholders otherwise): + +``` +replication_package/ +├── README.md # Phase 1 +├── data/ +│ ├── raw/ # as-obtained (or a pointer + DUA note if restricted) +│ └── analysis/ # constructed analysis files +├── code/ # numbered scripts + master script (00_run_all.* / 99_run_all.do) +└── output/ # tables/, figures/, logs/, sessionInfo.txt, renv.lock / requirements.txt +``` + +Then emit the **DCAS compliance checklist** (`replication_package/DCAS_checklist.md`): Data Availability Statement present · every dataset has source + access + license · master script present and one-command · computational requirements stated · every Table/Figure mapped to program:line · no absolute/machine-specific paths in code · seeds set for any stochastic step · license file (a code license such as BSD/MIT + a data-usage statement). Mark each PASS / FAIL / `[FILL]`. + +### Phase 5: Confidential-data handling + +Per [`.claude/rules/confidential-data.md`](../../rules/confidential-data.md), scan the manifest for restricted, proprietary, or PII-bearing inputs (administrative records, IRS/Census RDC, proprietary panels, linked health data). + +- **Never copy restricted data into `replication_package/data/`.** Replace it with a pointer: the provider, the application/DUA process, the access cost, and the expected wait time. +- Generate `replication_package/data/access-restricted-data.md` — the access-restricted-data note a reproducer follows to obtain the same inputs. +- Confirm the **code still ships** (DCAS requires runnable-on-restricted-data code even when the data cannot be deposited), and that any committed extracts pass disclosure-avoidance (cell suppression / rounding) before they enter `output/`. + +## Output / Report format + +Write `quality_reports/replication_package_[paper-slug].md`: + +```markdown +# Replication Package: [Paper Title] +**Date:** [YYYY-MM-DD] **Languages:** [R / Stata / Python] **Deposit target:** [openICPSR / Zenodo / Dataverse] + +## DCAS checklist +| Item | Status | +|---|---| +| Data Availability Statement | PASS / FAIL / [FILL] | +| Dataset manifest (source · access · license) | ... | +| One-command master script | ... | +| Computational requirements | ... | +| Table/Figure → program:line map | ... | +| No machine-specific paths · seeds set | ... | +| Reproducibility audit (Phase 3) | PASS / EXPLAINED-only / FAIL (blocker) | +| Confidential-data note (if applicable) | ... | + +## Skeleton built at +replication_package/ (tree + README + checklist) + +## Open [FILL] items +[one line per unresolved field] +``` + +## Exit behavior + +- **All checklist items PASS (or PASS + `[FILL]`) and audit PASS/EXPLAINED-only:** exit 0; print the tree location and any `[FILL]` items for the author to complete. +- **Any audit FAIL (Phase 3):** exit 1; package assembly halts. Numbers that do not reproduce do not get deposited. +- **Restricted data detected but no access note generated:** exit 1 with the confidential-data blocker — packaging cannot proceed until Phase 5 runs. + +## Cross-references + +- [`.claude/rules/replication-protocol.md`](../../rules/replication-protocol.md) — tolerance contract + passport schema (the upstream verification this skill packages). +- [`.claude/skills/audit-reproducibility/SKILL.md`](../audit-reproducibility/SKILL.md) — the Phase 3 gate; proves claims reproduce. +- [`.claude/rules/confidential-data.md`](../../rules/confidential-data.md) — restricted-data deposit rules driving Phase 5. +- [`templates/passport-template.yaml`](../../../templates/passport-template.yaml) — source of the Table/Figure → program:line map when present. +- [`.claude/skills/data-analysis/SKILL.md`](../data-analysis/SKILL.md) · [`.claude/skills/stata-replication/SKILL.md`](../stata-replication/SKILL.md) — the R / Stata pipelines whose outputs this skill packages. +- [`.claude/skills/simulation-study/SKILL.md`](../simulation-study/SKILL.md) — seeded Monte Carlo outputs are packaged the same way (seeds + per-rep raw results belong in `output/`). +- [`.claude/skills/preregister/SKILL.md`](../preregister/SKILL.md) — for RCTs, the PAP belongs in the deposit alongside the analysis. + +## What this skill does NOT do + +- **Verify the numbers.** That is `/audit-reproducibility` (called in Phase 3). This skill packages a *verified* result; it blocks rather than re-derives on FAIL. +- **Upload to the repository.** It builds the local tree and README; the author performs the openICPSR / Zenodo / Dataverse upload and gets the DOI. Web deposit is deliberately out of scope. +- **Judge the research.** Whether the identification strategy (DiD / event-study, IV, RCT, panel FE) is sound is a `/review-paper` question. A reproducible package can still house a flawed design. +- **De-identify your data.** It flags restricted inputs and refuses to deposit them; it does not run disclosure-avoidance algorithms on raw microdata — that is the author's (and the RDC's) responsibility. diff --git a/.claude/skills/respond-to-eval/SKILL.md b/.claude/skills/respond-to-eval/SKILL.md new file mode 100644 index 000000000..736fd2dd2 --- /dev/null +++ b/.claude/skills/respond-to-eval/SKILL.md @@ -0,0 +1,128 @@ +--- +name: respond-to-eval +description: Turn student course evaluations (free-text + numeric) into an actionable teaching-improvement plan — the teaching analogue of /respond-to-referees. Clusters comments into themes, separates signal from noise, classifies each theme Keep / Change / Investigate / Out-of-scope, and drafts concrete changes mapped to the syllabus and slide decks. Use when user says "respond to my evals", "what do these course evaluations tell me", "turn my teaching feedback into a plan", or after a semester's evals arrive. +argument-hint: "[eval-file(s)] [prior-plan-path] [--min-mentions N] [--no-verify]" +allowed-tools: ["Read", "Write", "Grep", "Glob", "Bash", "Task"] +effort: high +--- + +# Respond to Evaluations + +Convert a semester's course evaluations into a defensible teaching-improvement plan. Cluster free-text comments into themes, weight each theme by how many independent students raised it, classify what to do about it, and draft specific changes pointed at the syllabus and deck — so next semester's revision is a checklist, not a vibe. + +**Posture (echoes `/respond-to-referees`):** one angry comment is not a trend, and a comment you disagree with is a signal to *investigate*, not a license to ignore or to auto-act. A single student's frustration may be the only one willing to say what twenty felt — frequency weights the theme, it does not gate it. Ground truth here is a process: the plan records *why* a theme was kept or changed, so the reasoning survives to the next round. + +## When to use + +- End of term, when numeric scores + open-text comments land and you want a revision plan, not a mood. +- Assembling a teaching dossier / tenure file where you must *show* you acted on feedback. +- Mid-stream (early-semester feedback) to course-correct before the term ends. + +Not for: writing the syllabus from scratch (compose with `/create-lecture` and a course outline), or reviewing one deck's pedagogy (use `/pedagogy-review`). + +## Inputs + +- `$0` — the evaluation file(s): a CSV/TSV export, `.txt`/`.md` of pasted comments, or a `.pdf`/`.docx` report. +- `$1` (optional) — the **prior** improvement plan, so this round is a diff (did last term's changes land?). + +| Format | How to read | +| --- | --- | +| `.csv`, `.tsv`, `.txt`, `.md` | `Read` directly; for CSV, note which column is numeric vs free-text. | +| `.pdf` | `TMP=$(mktemp -t evals).txt && pdftotext "$0" "$TMP"` (poppler). Read/grep `"$TMP"`. | +| `.docx` | `TMP=$(mktemp -t evals).txt && pandoc "$0" -t plain -o "$TMP"`. | + +If extraction fails or a tool is missing, ask for a plain-text export and stop. + +## Phases + +### Phase 0: Load evals + prior plan (Pre-Flight) + +Read the eval file(s) and the prior plan (if given). Produce a short Pre-Flight block before clustering: + +```markdown +## Pre-Flight Report +**Evals loaded:** N responses (M with free-text), instrument: [name/term] +**Numeric items:** [list each scale item + mean, and the institution/department mean if present] +**Prior plan:** [path, or "none — first round"] — changes promised last term: [bullet list] +**Course artifacts in scope:** [syllabus path] · [deck(s) under Slides/ or Quarto/] +``` + +Numbers anchor the read but do not override text: a 4.2/5 with ten "I was lost by week 6" comments is a problem the mean is hiding. + +### Phase 1: Theme-cluster + weight by frequency + +1. Split free-text into atomic comments (one student may raise several themes; one comment may belong to several themes). +2. Cluster into themes (e.g., *pacing*, *problem-set difficulty*, *grading clarity*, *real-world relevance*, *office hours*, *prerequisite gaps*). Name each theme in the instructor's words, not the student's. +3. For each theme record: **mention count** (distinct students), **representative verbatim quote** (~25 words, anonymized — strip names/identifying detail), **valence** (positive / negative / mixed), and **numeric corroboration** (which scale item, if any, moves with it). +4. **Signal vs noise:** a theme with `< --min-mentions` (default 2) distinct students is tagged **low-frequency**, not dropped — it carries to Phase 2 for a Keep/Investigate call. Frequency weights; it never silences. + +### Phase 2: Classify + propose changes + +Assign each theme exactly one label (the teaching analogue of `/respond-to-referees`' coverage matrix): + +| Label | Meaning | Drives | +| --- | --- | --- | +| **Keep** | Working well; protect it from collateral damage when you change other things. | A "do not break" note. | +| **Change** | Clear, agreed problem with a concrete fix you can name. | A specific syllabus/deck edit. | +| **Investigate** | Real signal but root cause unclear, or you disagree with the proposed remedy — gather evidence (a mid-term pulse poll, a look at the grade distribution, peer observation) before acting. | An investigation step, not an edit. | +| **Out-of-scope** | Outside your control (room, time slot, required textbook, prerequisite course) or contrary to a deliberate pedagogical choice you stand behind. | A documented rationale, not a change. | + +For each **Change**, write a concrete revision mapped to a target: `syllabus §X` / `Slides/LectureNN.tex slide K` / a new worked example / an assessment reweighting — the same point-to-the-location discipline `/respond-to-referees` uses for "we added X on page Y". For **Investigate**, name the evidence you'll collect and the decision rule. For **Out-of-scope**, write the one-sentence rationale you'd stand behind in a dossier. + +Disagreement is explicit and reasoned: "Students asked to drop proofs; retained because the course's stated objective is derivation fluency — added two scaffolded worked examples (LectureNN slide K) to ease the on-ramp instead" is a Keep-with-mitigation, not an Out-of-scope dismissal. + +### Phase 3: Save the improvement plan + +Write the plan to `quality_reports/teaching/YYYY-MM-DD_[course]_improvement-plan.md`. Structure: + +1. **Header** — course, term, instrument, response rate, numeric summary vs benchmark. +2. **Prior-plan retrospective** (if `$1` given) — for each change promised last term: Landed / Partial / Not done, with the evidence from this term's evals. +3. **Theme matrix** — one row per theme: theme · mentions · valence · numeric corroboration · classification · target (syllabus §/deck slide) · representative quote. +4. **Change list** — the concrete edits, ordered by mention count then severity, each pointing at a syllabus section or deck/slide. +5. **Investigate list** — open questions + the evidence to collect next. + +The plan is a deliverable, not a transient report, so it lives under `quality_reports/teaching/` and feeds next term's `$1`. + +### Phase 3.5: Post-Flight Verification (quotes + targets) + +The plan's hallucination-prone content is (a) verbatim quotes attributed to students and (b) "edit syllabus §X / LectureNN slide K" targets that must actually exist. Run the forked-verifier protocol in [`.claude/rules/post-flight-verification.md`](../../rules/post-flight-verification.md): spawn `claim-verifier` (`context: fork`) with the quotes + the eval source and the edit-targets + the syllabus/deck paths. Reconcile — a quote that isn't in the source, or a "slide K" that doesn't exist, is corrected or dropped before the plan is final. Opt-out: `--no-verify` (not recommended). + +## Output / Report + +After writing the plan, surface this in your **final chat message** (not inside the plan file): + +``` +## Teaching-improvement summary — [course], [term] +Themes: K total — C Change · I Investigate · P Keep · O Out-of-scope +Top 3 changes (by mentions): 1) … 2) … 3) … +Open investigations: … +Prior plan: x of y promised changes landed. +``` + +If all themes are classified and every Change names a target, say `All themes classified; every Change mapped to a syllabus or deck target.` + +## Exit behavior + +- A theme with no classification halts the report — there are no orphans, exactly as `/respond-to-referees` admits no unclassified concern. +- Read-only on the syllabus and decks: this skill **plans** edits and writes the plan file; it does not edit teaching materials. Apply changes deliberately afterward (with `/create-lecture` or direct edits). +- Numbers never auto-override text and text never auto-overrides numbers; conflicts become **Investigate**, not a silent winner. + +## Flags + +- `--min-mentions` — distinct-student threshold below which a theme is tagged *low-frequency* (default `2`). Lowering it surfaces more singletons; it never drops them. +- `--no-verify` — skip Phase 3.5 Post-Flight Verification of quotes and edit-targets. Not recommended for a dossier-bound plan. + +## Cross-references + +- [`.claude/skills/respond-to-referees/SKILL.md`](../respond-to-referees/SKILL.md) — the research analogue; this skill borrows its map-classify-respond shape and "signal to investigate, not auto-act" posture. +- [`.claude/skills/pedagogy-review/SKILL.md`](../pedagogy-review/SKILL.md) — once a **Change** targets a specific deck, run pedagogy-review on it before re-teaching. +- [`.claude/skills/create-lecture/SKILL.md`](../create-lecture/SKILL.md) — to execute deck-level changes the plan proposes. +- [`.claude/rules/post-flight-verification.md`](../../rules/post-flight-verification.md) — the forked-verifier protocol Phase 3.5 reuses. +- [`templates/skill-template.md`](../../../templates/skill-template.md) — house style for skills. + +## What this skill does NOT do + +- It does **not** edit the syllabus or any deck — it produces a plan; you (or `/create-lecture`) apply it. +- It does **not** compute new numeric scores or re-weight the instrument; it reads the institution's numbers as given. +- It does **not** identify students or attempt to de-anonymize comments — quotes are stripped of identifying detail. +- It does **not** auto-act on disagreement or on a single comment; both route to **Investigate** or a documented rationale. diff --git a/.claude/skills/review-paper/SKILL.md b/.claude/skills/review-paper/SKILL.md index 8449b35d1..08f659c2b 100644 --- a/.claude/skills/review-paper/SKILL.md +++ b/.claude/skills/review-paper/SKILL.md @@ -288,11 +288,14 @@ Phase 3: Re-audit → Jump back to Phase 1. ``` -### Iteration limits +### Iteration limits — loop-until-dry -- **Max 5 rounds.** After round 5, halt regardless of verdict. -- **Fix round limits:** if the same Concern label appears in rounds N and N+2, flag as "author disagreement" and let the user decide (keep-as-is with rationale vs. another fix attempt). -- **Budget escape:** if cumulative token cost across all rounds exceeds a configurable spend cap (default ~500k — a spend ceiling, not a context-window limit, since each re-audit runs in fresh context), warn and let the user cap further rounds. +Same **loop-until-dry** primitive as `/qa-quarto` ([`orchestrator-protocol.md`](../../rules/orchestrator-protocol.md)): the critic returns `FINDING`s in the shared schema ([`orchestration-schemas.md`](../../references/orchestration-schemas.md)) and the loop **converges when a round adds 0 new CRITICAL/MAJOR concerns** (deduped on `location`+`finding`), not at a fixed count. + +- **Convergence:** APPROVED when a round produces zero Major Concerns and zero fatal Referee Objections. +- **Fallback cap:** 5 rounds bounds a non-converging loop; after round 5, halt and list remaining concerns. +- **Two-strikes:** if the same Concern label appears in rounds N and N+2, flag as "author disagreement" and let the user decide (keep-as-is with rationale vs. another fix attempt) — see [`summary-parity.md`](../../rules/summary-parity.md). +- **Budget escape:** if cumulative token cost exceeds the spend cap (default ~500k — a spend ceiling, not a context-window limit, since each re-audit runs in fresh context), warn and let the user cap further rounds. ### Stopping criteria @@ -356,7 +359,7 @@ Reports: `quality_reports/cross_artifact_[paper]/reproducibility.md`. Opt-out: `--no-novelty-check` already skips the probe entirely. If the probe runs, Post-Flight is mandatory. -**Pre-Flight Report (required before Phase 1).** Before spawning the editor, output a Pre-Flight Report so the user can verify the inputs are read correctly: +**Pre-Flight Report (required before Phase 1).** This is the `RUN_CONFIG` echo from [`orchestrator-protocol.md`](../../rules/orchestrator-protocol.md) — every interactive choice (journal, dispositions, peeve budget, N referees, cross-artifact/novelty toggles, round) is resolved **before** the forked editor/referees spawn, because a forked subagent cannot stop to ask. Output it so the user can verify inputs, and halt here on any unresolved required field (unknown journal, missing script) rather than mid-run: ```markdown ## Pre-Flight Report — /review-paper --peer @@ -393,9 +396,11 @@ Spawn in parallel: Each referee must include "What would change my mind: [specific ask]" on every MAJOR concern. -### Phase 3: Editor synthesis +### Phase 3: Editor synthesis (reduce → judge, with the hallucination gate) + +Read both referee reports. **Reduce** their `FINDING`s, classify each MAJOR concern as FATAL / ADDRESSABLE / TASTE, and produce the editorial decision using the decision rule table in `editor.md`. -Read both referee reports. Classify each MAJOR concern as FATAL / ADDRESSABLE / TASTE. Produce editorial decision using the decision rule table in `editor.md`. +**Post-judge hallucination gate** ([`orchestration-schemas.md` §4](../../references/orchestration-schemas.md)): the editor reduces the referees — it must not desk-reject or escalate on a CRITICAL reason **neither referee raised**. Any editor-introduced blocker that is not traceable to a referee finding is re-verified in a fresh `claim-verifier` fork or dropped to `[JUDGE-HALLUCINATED]` and the decision recomputed. (The editor may always downgrade or de-duplicate referee concerns.) Report: `quality_reports/peer_review_[paper]/editorial_decision.md`. @@ -431,3 +436,10 @@ The shipped `journal-profiles.md` covers 5 econ journals (AER, QJE, JPE, ECMA, R For non-econ paper types in `methods-referee.md`, extend the paper-type list (e.g., biology: `observational / experimental / computational / review`). +## Cross-references + +- [`.claude/skills/audit-reproducibility/SKILL.md`](../audit-reproducibility/SKILL.md) — numeric-claim verification (auto-invoked on referenced scripts). +- [`.claude/skills/replication-package/SKILL.md`](../replication-package/SKILL.md) — assemble the AEA DCAS deposit once the paper passes review. +- [`.claude/skills/capture-environment/SKILL.md`](../capture-environment/SKILL.md) · [`.claude/skills/disclosure-check/SKILL.md`](../disclosure-check/SKILL.md) — environment capture + restricted-data screening for the deposit. +- [`.claude/skills/seven-pass-review/SKILL.md`](../seven-pass-review/SKILL.md) — heavier 7-lens pass for submission-ready drafts. + diff --git a/.claude/skills/scaffold-exercises/SKILL.md b/.claude/skills/scaffold-exercises/SKILL.md new file mode 100644 index 000000000..2ed882945 --- /dev/null +++ b/.claude/skills/scaffold-exercises/SKILL.md @@ -0,0 +1,140 @@ +--- +name: scaffold-exercises +description: Scaffold a graded problem set with sections, problems, worked solutions, and short "why this matters" explainers across analytical, empirical, and coding types. Use when user says "make a problem set on X", "scaffold exercises for this lecture", "create practice problems", "generate homework with a solution key", "build a graded assignment on topic Y". Emits a clean student set plus a separate solution key — NOT for grading submissions or auto-checking student answers. +argument-hint: "[topic] [--difficulty intro|core|advanced] [--count N] [--types analytical,empirical,coding] [--dataset path] [--no-solutions]" +allowed-tools: ["Read", "Grep", "Glob", "Write", "Bash"] +effort: medium +--- + +# `/scaffold-exercises` — Problem Set Scaffolder + +Generate a graded problem set as two files: a clean **student set** (problems only) and a **solution key** (worked solutions + a one-line explainer per problem). Pattern imported from `mattpocock/skills`, adapted for economics teaching — the primary lens is graded coursework that mixes derivation, estimation, and code. + +**Input:** `$ARGUMENTS` — a topic (e.g., "instrumental variables", "consumer theory", "staggered DiD") and optional flags. See [Flags](#flags). + +--- + +## When to use + +- You have a lecture or reading and want a matching assignment with an answer key. +- You want a mix of problem types (derive, estimate, code) at a controlled difficulty, with solutions emitted **separately** so the student file stays clean. + +Do **not** use this to grade submissions, auto-check answers, or build a timed exam — it scaffolds *practice/graded* material, not assessment infrastructure. + +--- + +## Problem types + +| Type | What the student does | Solution artifact | +| --- | --- | --- | +| **analytical** | Derive / prove / characterize (theory: optimization, identification, comparative statics) | Step-by-step derivation with the key lemma named | +| **empirical** | Estimate + interpret on a provided or simulated dataset | Expected estimate, sign/magnitude reasoning, common-mistake note | +| **coding** | Implement an estimator or simulation in R or Stata | Runnable reference snippet + expected output shape | + +If no dataset is supplied for an empirical problem, generate a small **simulated** one with a fixed seed (YYYYMMDD) so the answer key is deterministic and reproducible. + +--- + +## Workflow + +### Phase 0: Set topic, difficulty, counts, types (Pre-Flight) + +Read any source material the user points at (lecture `.tex`/`.qmd`, a paper, a dataset header) and produce a Pre-Flight Report **before** generating problems: + +```markdown +## Pre-Flight Report — Problem Set + +**Topic:** [topic] +**Source(s) read:** [lecture/paper/dataset — one-line takeaway each] +**Difficulty:** intro | core | advanced +**Counts by type:** analytical=N, empirical=N, coding=N (total = `--count`) +**Dataset:** [provided path | simulated with seed YYYYMMDD | none] +**Learning objectives:** [2-4 bullets the set should exercise] +``` + +Resolve every flag here (interactive choices are gathered before generation, not mid-run). If the topic is too vague to write objectives, ask one clarifying question and stop. Otherwise proceed. + +### Phase 1: Generate problems + +For each problem, write a number, a section heading, the prompt, and any data/notation it needs. Conventions: + +- **Motivation before mechanics** — one sentence on why the problem is worth solving, matching `create-lecture`'s pedagogy. +- **Notation reuse** — match symbols to the source lecture; never introduce a clashing symbol for an already-defined object. +- **Difficulty calibration** — *intro* checks one concept; *core* chains 2-3 steps; *advanced* requires a non-obvious insight or identification argument. +- **Self-contained** — each problem states its own assumptions; no "as in lecture 4" dangling references. + +### Phase 2: Generate worked solutions + explainers + +For every problem, write: + +1. A **worked solution** — full derivation, expected estimate, or runnable code (depending on type). Coding solutions must actually run; if `Bash` + R/Stata are available, execute the snippet and paste real output. +2. A **"why this matters" explainer** — 1-2 sentences linking the answer to the broader concept (the imported pattern's signature: every problem ships with a short rationale, not just a number). + +### Phase 3: Write student set + solution key + +Emit two files (paths configurable; default under the working directory): + +- `exercises/_problems.md` — the **student set**: sections, problems, any data, NO answers. +- `exercises/_solutions.md` — the **solution key**: each problem restated, its worked solution, and its explainer. + +The split is load-bearing: never leak a solution into the student file. With `--no-solutions`, write only the student set and stop. + +--- + +## Output / Report format + +Student set: + +```markdown +# Problem Set: [Topic] (Difficulty: core) + +## Section 1 — Analytical +**1.** [Motivation sentence.] [Prompt.] + +## Section 2 — Empirical +**2.** Using `data/` (vars: ...), [estimate + interpret prompt]. + +## Section 3 — Coding (R) +**3.** [Implement-X prompt.] +``` + +Solution key mirrors the numbering, adding `### Solution` and `> Why this matters:` blocks per problem. Close your chat reply with a one-line manifest: files written, problem count by type, and whether code solutions were executed or only drafted. + +--- + +## Exit behavior + +- Print the two output paths (absolute), the per-type counts, and the seed if a dataset was simulated. +- If a coding solution could not be executed (no R/Stata, or it errored), flag it as **DRAFTED — NOT RUN** rather than implying it was verified. +- If any empirical problem references variables not present in the supplied dataset, stop and surface the mismatch instead of inventing columns. + +--- + +## Flags + +- `--difficulty` — `intro` | `core` | `advanced` (default `core`); calibrates step depth as in Phase 1. +- `--count` — total number of problems (default 6); split across types per the Pre-Flight counts. +- `--types` — comma-separated subset of `analytical,empirical,coding` (default all three). +- `--dataset` — path to a real dataset for empirical problems; omit to simulate one with a seeded DGP. +- `--no-solutions` — write only the student set; skip the solution key (Phase 2/3 key file). + +--- + +## Cross-references + +- [`.claude/skills/create-lecture/SKILL.md`](../create-lecture/SKILL.md) — build the lecture these exercises practice; shares notation-reuse + motivation-first conventions. +- [`.claude/skills/data-analysis/SKILL.md`](../data-analysis/SKILL.md) — for empirical problems whose reference solution needs a full R estimation pipeline. +- [`.claude/skills/simulation-study/SKILL.md`](../simulation-study/SKILL.md) — when a problem demonstrates an estimator's finite-sample behavior; reuse its seeded-DGP discipline. +- [`.claude/skills/lit-review/SKILL.md`](../lit-review/SKILL.md) — source advanced problems from current papers on the topic. +- [`.claude/skills/interview-me/SKILL.md`](../interview-me/SKILL.md) — turn a fuzzy "I want a set on…" into concrete learning objectives first. +- [`templates/skill-template.md`](../../../templates/skill-template.md) — house style for authoring/extending this skill. + +--- + +## What this skill does NOT do + +- **Does not grade** student submissions or auto-check answers against a key. +- **Does not run a timed exam** or enforce assessment policy (point weights, rubrics, proctoring). +- **Does not invent data** — empirical problems use a supplied dataset or an explicitly seeded simulation, never fabricated numbers. +- **Does not leak solutions** into the student file, and does not deploy/publish anything (no `/deploy`). +- **Does not auto-invoke** other skills — it references siblings; it does not call them. diff --git a/.claude/skills/seven-pass-review/SKILL.md b/.claude/skills/seven-pass-review/SKILL.md index 182207561..e950839d2 100644 --- a/.claude/skills/seven-pass-review/SKILL.md +++ b/.claude/skills/seven-pass-review/SKILL.md @@ -47,7 +47,9 @@ In a single message, spawn 7 Task tool calls (one per lens). Each subagent gets: - The manuscript path (to re-read with its own context). - The lens-specific prompt (below). - Instructions to write to `quality_reports/seven_pass_[stem]/lens_[N]_[lens-name].md`. -- Severity tagging: CRITICAL / MAJOR / MINOR. +- A closing `findings:` + `scorecard:` block in the shared schema ([`orchestration-schemas.md`](../../references/orchestration-schemas.md)): `severity: CRITICAL | MAJOR | MINOR`, with `evidence` and `change_my_mind` on every CRITICAL/MAJOR. Phase 2 reduces over these typed findings — it does not re-read the prose. + +This is the **fan-out** primitive from [`orchestrator-protocol.md`](../../rules/orchestrator-protocol.md); `Task` subagents are the portable mechanism (the agents that fill lenses 3/6 are in [`agent-fleet.md`](../../references/agent-fleet.md)). Lens prompt rubrics are embedded inline below — one summary paragraph per lens. Each forked subagent receives its lens's rubric plus the manuscript path. @@ -61,9 +63,11 @@ Lens prompt rubrics are embedded inline below — one summary paragraph per lens - **Lens 6 (Prose):** Sentences under 30 words? Active voice dominant? Hedging proportionate (neither overclaiming nor endless "may suggest")? Paragraph topic sentences? - **Lens 7 (Citations):** Invoke `/validate-bib --semantic`. For top-10 cited works, does the in-text claim match the cited paper's actual finding direction? Are contemporary / competing works cited? -### Phase 2: Synthesize +### Phase 2: Synthesize (reduce → judge, with the hallucination gate) + +Wait for all 7 lens reports. **Reduce, don't re-review:** stack the seven `scorecard`s and apply the gate predicate from [`orchestration-schemas.md` §3](../../references/orchestration-schemas.md) — the Executive verdict is a function of the typed findings, not a fresh eighth opinion. Then **run the post-judge hallucination gate** ([§4](../../references/orchestration-schemas.md)): any CRITICAL the synthesis introduces that **no lens raised** must be re-verified in a fresh `claim-verifier` fork, or dropped to `[JUDGE-HALLUCINATED]` and the verdict recomputed. A synthesis may freely downgrade or de-duplicate lens findings; it may not invent a new blocker. -Wait for all 7 lens reports. Then read them and produce: +Then produce: `quality_reports/seven_pass_[stem]/_SYNTHESIS.md` diff --git a/.claude/skills/slide-excellence/SKILL.md b/.claude/skills/slide-excellence/SKILL.md index 9b2d9a850..4e7936c66 100644 --- a/.claude/skills/slide-excellence/SKILL.md +++ b/.claude/skills/slide-excellence/SKILL.md @@ -154,7 +154,9 @@ Spawn only the agents whose conditions hold: **De-duplication:** if the user has already run one of these skills on this file in the current session (e.g., ran `/proofread` first, now running `/slide-excellence`), ask whether to reuse the existing report or re-run. Default: reuse (saves tokens). -## Step 5: Synthesize Combined Summary +## Step 5: Synthesize Combined Summary (reduce typed findings) + +This is **fan-out → reduce** ([`orchestrator-protocol.md`](../../rules/orchestrator-protocol.md)): each agent returns `FINDING`s + a `SCORECARD` in the shared schema ([`orchestration-schemas.md`](../../references/orchestration-schemas.md)), and this step **stacks the typed scorecards** rather than re-reading each report by eye. The Overall Quality Score is the gate predicate over summed CRITICAL/MAJOR/MINOR counts. (Conditional dispatch means a skipped lens contributes no findings, not zeros to average.) Only include sections for agents that actually ran. diff --git a/.claude/skills/syllabus/SKILL.md b/.claude/skills/syllabus/SKILL.md new file mode 100644 index 000000000..5d24ae922 --- /dev/null +++ b/.claude/skills/syllabus/SKILL.md @@ -0,0 +1,119 @@ +--- +name: syllabus +description: Build or restructure a course syllabus from a topic list or reading list — course description + prerequisites, week-by-week schedule (topic → readings → deliverables), measurable learning objectives, an assessment scheme + rubric, standard policies (late work, AI use, academic integrity, accessibility), and a per-week work-list to hand to `/create-lecture`. Use when user says "build a syllabus", "structure my course", "turn this reading list into a schedule", "draft a course outline", "make a syllabus for Econ 7xx", or "map weeks to lectures". Economics-aware (PhD metrics/micro/macro sequences, undergrad); generic enough for any field. +argument-hint: "[course title or topic/reading list] [--weeks N] [--level phd|grad|undergrad] [--sessions-per-week N] [--no-policies]" +allowed-tools: ["Read", "Grep", "Glob", "Write"] +effort: medium +--- + +# Build a Course Syllabus + +Turn a bare topic list or reading list into a teachable syllabus: a course description, a sequenced weekly schedule, measurable objectives, an assessment scheme with a rubric, the boilerplate policies every syllabus needs, and a week→lecture work-list you can feed straight into `/create-lecture`. The instructor owns the academic judgment; this skill structures it. + +## When to use + +- You have a pile of topics or papers and need them ordered into a term. +- You are converting last year's syllabus to a new length, level, or modality. +- You want the schedule's weeks pre-mapped to lecture decks before you start building slides. + +Not for building the slides themselves (`/create-lecture`), reviewing a deck's pedagogy (`/pedagogy-review`), or assembling the reading list from scratch (`/lit-review`). + +## Phase 0: Intake (elicit before drafting) + +A syllabus is shaped almost entirely by three parameters. Resolve them first — from flags, then by asking. Do **not** start sequencing until all three are pinned. + +1. **Level + audience** — `--level` (`phd` / `grad` / `undergrad`). For economics, name the sequence (first-year metrics, micro theory, macro, field course, undergrad intermediate). Level sets reading depth, proof-vs-application balance, and assessment type. +2. **Length + cadence** — `--weeks` (default 14) and `--sessions-per-week` (default 2). A reading seminar and a problem-set course at the same length need very different schedules. +3. **Material** — the topic list or reading list (`$ARGUMENTS`, a file path, or a `.bib`). If the user points at a `.bib` or a folder of PDFs, `Glob`/`Read` to inventory it; if topics are bare, ask for 1-2 anchor texts per topic. + +Echo a short **Intake Report** (level, weeks, cadence, N topics / N readings, any gaps) and get confirmation before Phase 1. + +## Phase 1: Sequence topics into weeks + +1. **Order by dependency, not by the reading list's order.** Build a prerequisite chain — foundational tools before the topics that use them (e.g., asymptotics → GMM → applications; consumer theory → GE → welfare). Front-load what later weeks assume. +2. **Allocate readings to weeks.** Distribute material so each week's load is realistic for the level (a PhD field week carries 2-4 papers; an undergrad week, 1 chapter + 1 application). Flag any week that is overloaded or thin. +3. **Place deliverables on the calendar.** Problem sets after the tools they exercise; the midterm at a natural conceptual break; referee report / replication / paper proposal milestones spaced so they don't collide. Mark reading-break and no-class weeks. +4. **Emit the schedule table** (topic → readings → deliverable per week) and get sign-off before writing objectives — re-sequencing is cheap now, expensive later. + +## Phase 2: Objectives + assessment + +1. **Write measurable learning objectives.** One course-level set plus per-unit objectives, each using an observable verb (derive, estimate, replicate, critique, prove) — not "understand" or "be familiar with". Tie each objective to the week(s) that deliver it. +2. **Design the assessment scheme.** Pick instruments that fit the level (problem sets + final for a methods course; referee report + replication + paper for a field/PhD course; exams + project for undergrad). State weights summing to 100%, and check every objective is assessed by something. +3. **Write one rubric** for the highest-stakes deliverable (the paper, project, or referee report) — criteria × levels, with point bands. Keep it short and concrete. + +## Phase 3: Emit the syllabus + the lecture work-list + +1. **Assemble the syllabus document** in the Output format below (skip policies if `--no-policies`). +2. **Generate the per-week `/create-lecture` work-list** — one row per teaching week mapping it to a deck name, its objective(s), and its anchor reading, so the instructor can hand each row to `/create-lecture` in order. This is the bridge from syllabus to slides. + +## Output format + +Write to `syllabus.md` (or a user-specified path): + +```markdown +# [Course title] — [Term, Year] +[Level · meeting cadence · units] + +## Course description +[2-4 sentences: what the course is, what students leave able to do.] + +## Prerequisites +[Courses / skills assumed. Be specific — "first-year metrics or equivalent".] + +## Learning objectives +By the end of this course, students will be able to: +- [observable verb] … (Weeks N-M) + +## Weekly schedule +| Week | Topic | Readings | Deliverable | +|------|-------|----------|-------------| +| 1 | … | … | — | + +## Assessment +| Component | Weight | Maps to objectives | +|-----------|-------:|--------------------| +| … | …% | … | + +### Rubric — [highest-stakes deliverable] +| Criterion | Excellent | Adequate | Weak | +|-----------|-----------|----------|------| + +## Policies +Late work · AI / LLM use · academic integrity · accessibility / accommodations · attendance. + +## Week → lecture work-list (hand to `/create-lecture`) +| Week | Deck name | Objective(s) | Anchor reading | +|------|-----------|--------------|----------------| +``` + +End the chat message (not the file) with a **gap summary**: weeks with no reading, objectives not assessed, deliverable collisions, or topics dropped for lack of time — the instructor decides how to resolve each. + +## Flags + +- `--weeks` — term length in teaching weeks (default 14). +- `--level` — `phd` / `grad` / `undergrad`; sets reading depth and assessment type. +- `--sessions-per-week` — meetings per week (default 2); affects per-week load. +- `--no-policies` — omit the boilerplate Policies section (use when an institutional template supplies it). + +## Exit behavior + +- Phase 0 halts until level, length, and material are all resolved — a forked or unattended run never guesses these. If the material is empty, ask for a topic/reading list and stop. +- Each phase gates on user sign-off (schedule before objectives, objectives before policies); re-ordering is offered cheaply at Phase 1. +- The deliverable is the syllabus file plus the work-list; the chat message always carries the gap summary, even when empty (`No gaps — every week has a reading and every objective is assessed.`). +- This skill is text-only; it never compiles, renders, or builds slides. + +## Cross-references + +- `/create-lecture` ([`.claude/skills/create-lecture/SKILL.md`](../create-lecture/SKILL.md)) — the natural next step; feed it the work-list rows one week at a time. +- `/lit-review` ([`.claude/skills/lit-review/SKILL.md`](../lit-review/SKILL.md)) — build or extend the reading list *before* sequencing. +- `/interview-me` ([`.claude/skills/interview-me/SKILL.md`](../interview-me/SKILL.md)) — if the course goals are still fuzzy, formalize them first. +- `/pedagogy-review` ([`.claude/skills/pedagogy-review/SKILL.md`](../pedagogy-review/SKILL.md)) — review narrative/pacing once decks exist; the syllabus arc is the input it checks against. +- Authoring conventions: [`templates/skill-template.md`](../../../templates/skill-template.md). + +## What this skill does NOT do + +- Does not write slide content or scaffold `.tex`/`.qmd` decks — that is `/create-lecture`. +- Does not search for or vet readings — pair with `/lit-review`. +- Does not grade, generate problem sets, or build answer keys. +- Does not assert institutional policy as binding — the Policies section is editable boilerplate the instructor must reconcile with their department's rules. +- Does not invent citations; every reading in the schedule comes from the material the user supplied. diff --git a/.claude/skills/teach-from-paper/SKILL.md b/.claude/skills/teach-from-paper/SKILL.md new file mode 100644 index 000000000..927940586 --- /dev/null +++ b/.claude/skills/teach-from-paper/SKILL.md @@ -0,0 +1,116 @@ +--- +name: teach-from-paper +description: Turn a research paper into teaching materials — a lecture outline, the 3-5 results worth presenting (with intuition), a slide skeleton ready for `/create-lecture`, discussion questions, and a problem-set brief. Reads the paper end-to-end and pitches to a stated audience level. Use when user says "turn this paper into a lecture", "teach from this paper", "build slides from this PDF", "make teaching materials from X", "I'm presenting this paper to my class". +argument-hint: "[paper-path] [--level undergrad|phd|seminar] [--minutes N] [--no-exercises]" +allowed-tools: ["Read", "Grep", "Glob", "Write", "Bash"] +effort: high +--- + +# Teach From Paper + +Convert one research paper into a ready-to-build teaching package: a lecture outline, a shortlist of teachable results with the intuition spelled out, a slide skeleton, discussion questions, and an exercise brief. The deliverable is an outline and a brief — **not** a finished deck; the slide skeleton is shaped to hand straight to `/create-lecture`, and the exercise brief to `/scaffold-exercises`. + +## When to use + +- You have a paper (PDF, `.tex`, `.md`) and a slot to teach it — a lecture, a reading group, a job-market practice talk. +- You want the teachable core extracted and pitched to a level, not a generic summary. +- You're prepping the *plan* for a deck and want `/create-lecture` to do the drafting. + +Not for: literature surveys across many papers (use `/lit-review`); refereeing the paper's correctness (use `/review-paper`); drafting the actual Beamer slides (use `/create-lecture`). + +## Inputs + +- `$0` — path to the paper. + +| Format | How to read it | +| --- | --- | +| `.tex`, `.qmd`, `.md`, `.txt` | Read directly with the `Read` tool. | +| `.pdf` | `TMP=$(mktemp -t paper).txt && pdftotext "$0" "$TMP"` (poppler-utils), then Read/Grep `"$TMP"`. | + +If extraction fails or the tool is missing, ask the user for a plain-text version and stop. The full paper goes in the context window (1M) — read it end-to-end before extracting; do not skim the abstract and guess. + +## Phases + +### Phase 0: Read the Paper + Set Audience Level (Pre-Flight Report) + +Read the paper start to finish. Then resolve the audience level and time budget — from `--level` / `--minutes` if given, otherwise ask once. Echo a Pre-Flight Report before extracting: + +```markdown +## Pre-Flight Report +**Paper:** [title, authors, year] +**One-line thesis:** [the paper's central claim in your words] +**Audience level:** undergrad | phd | seminar (drives notation depth + which proofs survive) +**Time budget:** N minutes (~N/2 slides) +**Prerequisites assumed:** [concepts students must already have] +**Running example candidate:** [the paper's application that can thread the lecture] +``` + +Get a nod on level + thesis, then proceed. Level governs everything downstream: undergrad keeps intuition and drops proofs; phd keeps the identifying assumptions and one key derivation; seminar foregrounds the contribution-vs-literature framing. + +### Phase 1: Extract the Teachable Core + Intuition + +1. Identify the paper's **3-5 results worth presenting** — not every proposition, the ones a student should leave remembering. Prefer the headline result, the identifying assumption that makes it credible, and one surprise or limitation. +2. For each, capture: the **formal statement** (trimmed to the audience level), the **intuition** (why it's true in one breath, no algebra), and the **failure mode** (when it breaks). +3. Map the paper's notation to something teachable; flag any symbol clash a student would trip on. +4. Separate **method** (how they get the result) from **takeaway** (what we now believe) — students conflate these; the lecture must not. + +### Phase 2: Build the Lecture Outline + Slide Skeleton + +Produce a motivation → setup → key result → method → takeaways arc, then a slide skeleton matching the time budget (~2 min/slide). Each skeleton entry is a title + one-line content note + figure/diagram placeholder — enough for `/create-lecture` to draft from, no prose. Honor the project's pedagogy invariants in shape: motivation before formalism, a worked example near each definition, a transition slide at each act break. + +### Phase 3: Discussion Questions + Exercise Brief + +1. Write **4-6 discussion questions** graded by depth: comprehension → application → critique ("where would this identification fail?"). Pitch to the level. +2. Write an **exercise brief** — 2-4 problems sketched (prompt + what skill it drills + expected-answer shape), NOT full solutions. This is the hand-off to `/scaffold-exercises`, which fleshes out problems, data, and answer keys. Skip if `--no-exercises`. + +## Output / Report format + +Write to `quality_reports/teach_from_paper_[sanitized-title].md`: + +```markdown +# Teaching Package: [Paper Title] +**Audience:** [level] · **Budget:** [N min] · **Date:** [YYYY-MM-DD] + +## 1. Lecture Outline +Motivation → Setup → Key Result → Method → Takeaways (one line each) + +## 2. Results Worth Presenting +### R1 — [name] +- **Statement:** … · **Intuition:** … · **Breaks when:** … +[R2..R5] + +## 3. Slide Skeleton (→ /create-lecture) +| # | Title | Content note | Figure/diagram | + +## 4. Discussion Questions +1. [comprehension] … 5. [critique] … + +## 5. Exercise Brief (→ /scaffold-exercises) +- **E1:** [prompt] — drills [skill] — answer shape: [form] +``` + +## Exit behavior + +- Present the package in chat and confirm the file path written. +- Name the two hand-offs explicitly: "Slide skeleton ready — run `/create-lecture [Topic]`. Exercise brief ready — run `/scaffold-exercises`." +- If the audience level was never confirmed, do **not** ship — a deck pitched at the wrong level is wasted work. Halt and ask. + +## Flags + +- `--level` — `undergrad` | `phd` | `seminar`. Sets notation depth, which proofs survive, and question difficulty. Asked interactively if omitted. +- `--minutes` — target lecture length; the slide count is roughly `--minutes`/2. +- `--no-exercises` — skip Phase 3's exercise brief (keep discussion questions). + +## Cross-references + +- `/create-lecture` — consumes the Phase 2 slide skeleton to draft the actual Beamer deck. See [`.claude/skills/create-lecture/SKILL.md`](../create-lecture/SKILL.md). +- `/review-paper` — referee the paper's correctness *before* teaching it if you're unsure the result holds. See [`.claude/skills/review-paper/SKILL.md`](../review-paper/SKILL.md). +- `/lit-review` — for situating the paper among many, rather than teaching one deeply. See [`.claude/skills/lit-review/SKILL.md`](../lit-review/SKILL.md). +- The Phase 5 exercise brief is the input contract for `/scaffold-exercises` (a downstream skill that fleshes out problem sets); this skill stops at the brief. + +## What this skill does NOT do + +- Does **not** draft finished slides — that's `/create-lecture`. The slide skeleton is an outline, not Beamer. +- Does **not** write full problem-set solutions — the exercise brief is a sketch for `/scaffold-exercises`. +- Does **not** verify the paper is correct — pair with `/review-paper` if the result's validity is in doubt. +- Does **not** read multiple papers or survey a field — one paper in, one teaching package out. diff --git a/.claude/skills/triage-inbox/SKILL.md b/.claude/skills/triage-inbox/SKILL.md new file mode 100644 index 000000000..87536f33a --- /dev/null +++ b/.claude/skills/triage-inbox/SKILL.md @@ -0,0 +1,120 @@ +--- +name: triage-inbox +description: Triage academic email and calendar (Gmail / Google Calendar via the session's MCP) into a prioritized digest plus a referee-obligations tracker — classifying referee requests, R&R and editor correspondence, co-author threads, seminar and conference invites, and grant/admin deadlines, and proposing a human-gated action for each (draft reply, calendar hold, scaffold a project, snooze). Use when user says "triage my inbox", "check my academic email", "what needs my attention this week", "go through my mail", or when run as a scheduled routine. NEVER auto-sends or auto-accepts anything. +argument-hint: "[--since ] [--cap ] [--no-calendar] [--dry-run]" +disable-model-invocation: true +allowed-tools: ["Read", "Write", "Glob", "Bash"] +effort: medium +--- + +# /triage-inbox — Academic Inbox + Calendar Triage + +Turn a noisy academic inbox into a short, decision-ready digest. Fetch recent mail and calendar context through the session's MCP servers (Gmail / Google Calendar), classify each thread into the categories an academic actually acts on, and propose **one** action per thread — always human-gated. The companion artifact is a running **referee-obligations tracker** so you never silently overcommit to reviews. + +**Core principle:** this skill *reads, classifies, and proposes*. It drafts; it never sends, accepts, declines, or books anything without you. That boundary is what makes it safe to run unattended as a [`/schedule`](#cross-references) routine. + +## When to use + +- **Weekly / daily sweep** — "what landed that needs a decision?" without reading every thread yourself. +- **As a scheduled routine** — wired to `/schedule` to run each morning and leave a digest waiting. +- **Referee-load management** — keep an honest count of outstanding reviews against a standing cap before you say yes to one more. +- **R&R / editor deadline capture** — turn "minor revision due in 6 weeks" buried in an email into a calendar hold proposal. + +## When NOT to use + +- To actually send a reply, accept an invite, or book an event — this skill stops at *proposals*. You confirm and execute. +- To handoff a project to a co-author — that's [`/coauthor-brief`](../coauthor-brief/SKILL.md). +- To draft the R&R response document itself — that's [`/respond-to-referees`](../respond-to-referees/SKILL.md). + +## Phases + +### Phase 0 — Pre-flight (MCP check, window, referee cap) + +1. **Confirm MCP access.** This skill reaches mail/calendar **only** through the session's MCP tools (`Gmail` search/read, `Google Calendar` list/suggest). They are session-scoped — in a headless `claude -p` or cron run they may be **absent**. Probe once (e.g. list labels / list calendars). If unavailable, **degrade gracefully**: emit a tracker-only digest from the on-disk tracker (Phase 3) plus a one-line "MCP servers not reachable in this run — skipped fetch" note, and exit cleanly. Never fail the routine over a missing server. +2. **Resolve the lookback window** — `--since` (an ISO date or `Ndays`), else the timestamp of the last digest in `quality_reports/inbox/`, else default **7 days**. Echo it back. +3. **Set the referee-load cap** — `--cap` if given, else read the standing cap from the tracker header, else default **3** concurrent reviews. This cap gates the recommendation in Phase 2, not your inbox. +4. **Echo a one-line pre-flight** before fetching: window, cap, calendar on/off (`--no-calendar`), dry-run on/off. + +### Phase 1 — Fetch + classify + +1. **Fetch** recent threads via the MCP Gmail search tool over the window; if calendar is on, pull existing events/free-busy for the deadline-conflict check. +2. **Classify** each thread into exactly one bucket: + + | Bucket | Signals | + |---|---| + | **Referee request** | "invite you to review", journal/editor sender, manuscript ID, "would you be willing" | + | **R&R / editor correspondence** | "revise and resubmit", "minor/major revision", decision letter, due-date language | + | **Co-author thread** | known collaborator, shared-paper subject, "can you", "your section", attachment churn | + | **Seminar / conference invite** | "invited talk", "submit by", CFP, "seminar series", scheduling polls | + | **Grant / admin deadline** | funder name, "submission deadline", reporting/compliance, IRB/DUA renewals | + | **Noise** | newsletters, receipts, auto-notifications — counted, not itemized | + +3. Capture per thread: sender, subject, a one-line gist, any **explicit deadline**, and the bucket. + +### Phase 2 — Propose one action per thread (NEVER auto-send) + +For each non-noise thread, propose exactly one of: + +- **Draft reply** — write a courteous draft *for review*. Do not send. If the Gmail MCP exposes a create-draft tool, you MAY stage a Gmail draft (which still requires the user to hit send) — otherwise inline the text in the digest. +- **Calendar hold** — for an R&R / grant / talk deadline, propose a hold (title, date, lead-time reminder). Surface conflicts against existing events. **Propose only** — booking is the user's click. +- **Scaffold a referee project** — for an *accepted* (or leaning-yes) referee request under the cap, offer to run `/new-referee-project` on the attached manuscript. Over the cap → recommend a polite decline draft instead, and say why ("4 reviews already open vs. cap of 3"). +- **Summarize + offer a brief** — for a co-author thread, distill the asks and offer to generate a [`/coauthor-brief`](../coauthor-brief/SKILL.md). +- **Snooze** — defer with a re-surface date; nothing else happens. + +**Hard gate:** every outbound action (send, accept, decline, book, scaffold) waits for explicit user confirmation. Drafts and holds are *proposals*. Honor `--dry-run` by proposing without staging even drafts. + +### Phase 3 — Emit digest + update the obligations tracker + +1. **Digest** → `quality_reports/inbox/YYYY-MM-DD_triage.md` (create the dir). Buckets ordered by urgency; each item is a one-liner + its proposed action. Noise is a count, not a list. +2. **Referee-obligations tracker** → `quality_reports/inbox/referee-obligations.md` (a persistent ledger, not dated). Append/refresh rows for any review accepted, declined, or completed this run; recompute open count vs. cap; flag overdue rows. + +## Output / report format + +```markdown +# Inbox Triage — YYYY-MM-DD (window: last N days · referee cap: K) + +## Needs a decision (M) +- **[R&R]** *J. of X* — minor revision, **due 2026-07-15**. → Propose calendar hold (−14d reminder); conflicts: none. +- **[Referee]** *Econometrica* — review request, manuscript 12-345. Open reviews 2/3 → under cap. → Offer `/new-referee-project`. +- **[Co-author]** A. Smith — "can you redo Table 3 with not-yet-treated controls?" → Summarized; offer `/coauthor-brief`. + +## FYI / snoozed (P) +- **[Seminar]** Dept. brown-bag poll — snoozed to 2026-06-16. + +## Noise: 24 threads (newsletters, receipts) — not itemized. + +## Referee load: 2 open / cap 3 (see referee-obligations.md) +``` + +Plus the one-line chat summary: digest path, counts per bucket, open-reviews-vs-cap, and whether the MCP fetch ran or was skipped. + +## Exit behavior + +- **Normal run:** write the digest, refresh the tracker, print the summary line. No mail sent, no event booked, no project scaffolded — those await your confirmation. +- **MCP unavailable (headless/cron):** tracker-only digest + "fetch skipped" note; exit 0. The routine must not error just because a session server is absent. +- **Over the referee cap:** still surface the request, but the proposed action is a decline draft with the count as rationale — never a silent scaffold. +- **`--dry-run`:** propose everything, stage nothing (not even a draft). + +## Flags + +- `--since` `` — Lookback window. Default: the last digest's timestamp, else 7 days. +- `--cap` `` — Standing concurrent-review cap that gates referee scaffolding. Default: the tracker header value, else 3. +- `--no-calendar` — Skip the Calendar MCP entirely; classify mail only, no holds proposed. +- `--dry-run` — Propose actions without staging anything (no Gmail drafts created). + +## Cross-references + +- [`.claude/skills/coauthor-brief/SKILL.md`](../coauthor-brief/SKILL.md) — the handoff brief offered for co-author threads. +- [`.claude/skills/respond-to-referees/SKILL.md`](../respond-to-referees/SKILL.md) — drafts the R&R response document once a revision deadline surfaces here. +- `/new-referee-project` — scaffolds a review repo from an accepted referee request (the action this skill proposes, never auto-runs). +- `/schedule` — wire this skill into a cron routine; the human-gated design is what makes unattended runs safe. +- [`.claude/rules/orchestrator-protocol.md`](../../rules/orchestrator-protocol.md) — the "no daemon, user/skill-initiated, human-in-the-loop" contract this skill honors for outbound actions. +- [`.claude/rules/confidential-data.md`](../../rules/confidential-data.md) — never copy attachment contents, restricted data, or credentials into a digest that may be committed. + +## What this skill does NOT do + +- **Send, reply, accept, decline, or book.** It drafts and proposes; you execute. No exceptions, including in scheduled runs. +- **Auto-scaffold a referee project.** It *offers* `/new-referee-project`; scaffolding waits for your yes and respects the cap. +- **Run unattended with side effects.** Outbound actions are always human-gated — the only thing a cron run writes is the digest and the tracker. +- **Read or store message bodies wholesale.** It extracts gists, deadlines, and senders; it does not archive email contents or attachment data into the repo. +- **Reach mail/calendar without MCP.** No direct IMAP/API credentials — everything goes through the session's MCP servers, and their absence degrades gracefully. diff --git a/.claude/skills/validate-bib/SKILL.md b/.claude/skills/validate-bib/SKILL.md index 701a1edbb..0f251c144 100644 --- a/.claude/skills/validate-bib/SKILL.md +++ b/.claude/skills/validate-bib/SKILL.md @@ -102,6 +102,8 @@ For each file, count citation commands (`\citet` vs `\citep` vs `\cite`; `@key` Gated behind `--cite-claim`. For the top-10 most-cited works per file, WebFetch the crossref abstract and surface it beside the in-text context. **No auto-judgment** — humans decide if the claim matches. +> **This is existence/structure, not appropriateness.** Deciding whether the cited paper *actually says* what the in-text claim attributes to it is [`/verify-claims`](../verify-claims/SKILL.md)'s job — it reads the source and grounds a supports / partial / contradicts verdict in quotes + pages (with the EXPLAINED escape for a defensible named alternative). `--cite-claim` only surfaces the abstract; for the verdict, run `/verify-claims`. + ### Report structure (`quality_reports/bib_audit_semantic.md`) ```markdown @@ -148,9 +150,10 @@ Gated behind `--cite-claim`. For the top-10 most-cited works per file, WebFetch - `.claude/skills/review-paper/SKILL.md` — pair for full pre-submission. - `.claude/skills/audit-reproducibility/SKILL.md` — numeric-claims counterpart. +- `.claude/skills/verify-claims/SKILL.md` — citation **appropriateness** counterpart (does the cited paper support the claim?). This skill checks that a citation *exists and is well-formed*; `/verify-claims` checks that it *holds*. ## What this skill does NOT do -- Judge whether a citation is used in the right *context* (`--cite-claim` surfaces abstracts but does not judge). +- Judge whether a citation is used in the right *context* — whether the cited paper actually *says* what the claim attributes to it. That is `/verify-claims`'s job (see 2d); this skill stays existence-and-structure only. - Auto-fix your `.bib` file — all edits are recommendations. - Check non-DOI identifiers (ISBN, arXiv, SSRN) — roadmap. diff --git a/.claude/skills/verify-claims/SKILL.md b/.claude/skills/verify-claims/SKILL.md index f529d7a08..ca65c4e70 100644 --- a/.claude/skills/verify-claims/SKILL.md +++ b/.claude/skills/verify-claims/SKILL.md @@ -20,6 +20,7 @@ Fact-check a draft using the **Post-Flight Verification protocol** ([`.claude/ru - **Other skills that auto-run Post-Flight internally** (`/lit-review`, `/research-ideation`, `/respond-to-referees`, `/review-paper --peer`) — no need to call this separately; they already run it. - **`/proofread`** — grammar, typos, overflow. Different lens. - **`/review-paper`** (default mode) — full manuscript review, not just claim verification. +- **`/validate-bib`** — checks citations *exist* and are well-formed (structural + DOI). This skill checks they *hold* (the cited paper supports the attributed claim). Complementary — run both before submission. ## How it works @@ -48,6 +49,8 @@ Read the draft. Identify factual assertions of these types: Skip: opinions, forward-looking suggestions, definitions the draft introduces. +**For citation-type claims, extract the claim↔citation PAIR — not just the citation.** Capture *what* the draft attributes to *which* work, so the verifier checks **appropriateness** (does Smith 2019 actually *show* X?), not merely existence. "Smith (2019) shows a positive wage effect" becomes `{cite: Smith2019, attributed: "positive wage effect"}`. This is the layer `/validate-bib` explicitly defers here: validate-bib confirms the citation *exists and is well-formed*; this skill confirms it *holds*. A mis-citation (the paper exists but says something else, or the opposite) is exactly a numeric/directional contradiction → HIGH-WARN unless a concrete `author_alternative` is recorded (then EXPLAINED). + Output a claims table: ```markdown @@ -72,18 +75,19 @@ The forked agent runs the CoVe independent-answer step. It has never seen the dr ### Phase 4 — Reconcile -The verifier returns a per-claim verdict in **one of three severity tiers** (v1.9.0): +The verifier returns a per-claim verdict in one of these severity tiers: - **HIGH-WARN** — fabricated reference (the cited paper doesn't exist at the named venue/year), draft claim directly contradicted by the source, or `not_found` retrieval that the verifier interprets as a hallucinated citation. **Gate-refuse** — these block `/commit` for any file `/verify-claims` was just run against, unless the user explicitly overrides with `--no-fail-closed` or sets `verifyClaims.allowHighWarn: true` in `.claude/settings.json`. - **MED-WARN** — transient infrastructure / retrieval failure (paywall the verifier can normally bypass via cached metadata; DOI resolver timeout; partial PDF read). Surface for the author; do not gate-refuse. - **LOW-WARN** — source genuinely inaccessible (paywalled and not in cache; private dataset; pre-print server transient). Surface with `cannot-verify` flag; do not gate-refuse. +- **EXPLAINED** (v2.0) — a numeric/directional contradiction the author has *pre-justified* with a concrete named alternative (different defensible edition, specification, sample, or rounding convention), passed to the verifier via the claim's `author_alternative` field. Surfaced with the evidence and the recorded reason; **non-gating**. The hard floor holds: a *fabricated* citation is never EXPLAINED, and a blank/vague alternative stays HIGH-WARN. This mirrors `audit-reproducibility`'s EXPLAINED disposition for numeric claims — a mismatch is not always a failure when a defensible alternative is named. -Verdict aggregation by tier across all extracted claims: +Verdict aggregation by tier across all extracted claims (EXPLAINED counts as non-gating, like LOW): | Tier counts | Outcome | `/commit` behaviour | |---|---|---| -| 0 HIGH, 0 MED, ≥ 0 LOW | **PASS** (green block) | proceeds | -| 0 HIGH, ≥ 1 MED, any LOW | **PARTIAL** (yellow block) | proceeds with warning | +| 0 HIGH, 0 MED, ≥ 0 LOW/EXPLAINED | **PASS** (green block) | proceeds | +| 0 HIGH, ≥ 1 MED, any LOW/EXPLAINED | **PARTIAL** (yellow block) | proceeds with warning | | ≥ 1 HIGH | **FAIL** (red block) | **halts** unless override | `--no-fail-closed` opts out of the gate-refuse behaviour on HIGH-WARN. Use sparingly — it's there for offline / hallucination-sensitive contexts where the user accepts the risk in writing. diff --git a/.githooks/pre-commit b/.githooks/pre-commit new file mode 100755 index 000000000..63fd4ec02 --- /dev/null +++ b/.githooks/pre-commit @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# +# Pre-commit quality gate for the academic-workflow template. +# +# This closes the "a direct `git commit` bypasses every gate" hole: the +# gates that `/commit` runs (surface-sync + quality score) now run on EVERY +# commit, not only when you go through the skill. +# +# Install once per clone: ./scripts/install-hooks.sh +# (sets core.hooksPath=.githooks so git finds this version-controlled hook.) +# +# It runs two gates against the STAGED changes and aborts the commit on +# failure: +# 1. Surface-sync (ALWAYS): count + enumerative-table-row consistency, +# skill integrity, model-version currency — deterministic and fast. +# 2. Quality score (>= 80) on staged .qmd / .tex / .R files with rubrics. +# +# Escape hatches (use sparingly; record the reason in the commit message): +# SKIP_QUALITY_GATE=1 git commit ... # skip only the quality score +# git commit --no-verify ... # skip ALL hooks (last resort) +# +set -uo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" || { + echo "pre-commit: not inside a git work tree" >&2 + exit 2 +} +cd "$REPO_ROOT" || exit 2 + +# Validate the STAGED index, not the working tree, by stashing unstaged +# *tracked* changes — but ONLY when it is safe. If any single file has BOTH +# staged and unstaged changes (the `git add -p` workflow), a `stash --keep-index` +# + `pop` round-trip can conflict and write literal conflict markers into your +# file. In that case we skip the stash and validate the working tree as-is (with +# a warning) rather than risk corrupting your work. Untracked files are never +# stashed (avoids slow stashes of large data dirs). A failed pop is loud + +# recoverable, never silently swallowed. +_STASHED=0 +_staged="$(git diff --cached --name-only 2>/dev/null | sort)" +_unstaged="$(git diff --name-only 2>/dev/null | sort)" +_overlap="$(comm -12 <(printf '%s\n' "$_staged") <(printf '%s\n' "$_unstaged") | grep -c . || true)" +if [ "${_overlap:-0}" -gt 0 ] 2>/dev/null; then + echo "ℹ pre-commit: ${_overlap} file(s) have both staged and unstaged changes —" >&2 + echo " validating the WORKING TREE (not the pure staged index) to avoid a stash conflict." >&2 +elif [ -n "$_unstaged" ]; then + if git stash push --keep-index --quiet --message "pre-commit-gate" 2>/dev/null; then + _STASHED=1 + trap 'if [ "$_STASHED" = "1" ]; then git stash pop --quiet 2>/dev/null || echo "⚠ pre-commit: could not auto-restore unstaged changes — recover with: git stash pop" >&2; fi' EXIT + fi +fi + +FAIL=0 + +echo "── pre-commit: surface-sync gates ──" +./scripts/check-surface-sync.sh +[ $? -ne 0 ] && FAIL=1 + +if [ "${SKIP_QUALITY_GATE:-0}" = "1" ]; then + echo "" + echo "── pre-commit: quality gate SKIPPED (SKIP_QUALITY_GATE=1) ──" +else + # Staged, added-or-modified files that have quality rubrics. + STAGED="$(git diff --cached --name-only --diff-filter=ACM | grep -Ei '\.(qmd|tex|R)$' || true)" + if [ -n "$STAGED" ]; then + n="$(printf '%s\n' "$STAGED" | grep -c .)" + echo "" + echo "── pre-commit: quality score (>= 80) on ${n} staged file(s) ──" + # Score each file individually: portable (no mapfile/bash4), handles + # spaces in paths, and gives a per-file exit code we can classify. + while IFS= read -r f; do + [ -z "$f" ] && continue + [ -f "$f" ] || continue + python3 scripts/quality_score.py "$f" + rc=$? + # quality_score.py: rc 2 = auto-fail (compile/syntax FAILURE), rc 1 = + # score < 80 (or scorer error / file-not-found). BOTH must block — a + # file that fails to compile is the worst case, not an "infra error". + if [ "$rc" -ne 0 ]; then + echo "✗ quality gate: '$f' failed (rc=$rc — 2=compile/syntax failure, 1=score<80 or scorer error)." >&2 + FAIL=1 + fi + done <<< "$STAGED" + else + echo "" + echo "── pre-commit: no staged .qmd/.tex/.R files to score ──" + fi +fi + +if [ "$FAIL" -ne 0 ]; then + echo "" >&2 + echo "✗ pre-commit gate failed — commit aborted." >&2 + echo " Fix the findings above, or (sparingly) bypass with SKIP_QUALITY_GATE=1 / --no-verify." >&2 + exit 1 +fi + +echo "" +echo "✓ pre-commit gates passed." +exit 0 diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 18e2fc5ff..b2fbf19d2 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -30,7 +30,8 @@ This repository is a **template** designed for academic researchers to fork and python3 scripts/quality_score.py path/to/file ``` 5. **Test against ≥2 domains** when adding skills/agents — show that your contribution generalizes. -6. **Update both README and the guide** when adding features. Skill counts must agree across `CLAUDE.md`, `README.md`, `docs/index.html`, and `guide/workflow-guide.qmd`. +6. **Install the gate once:** `./scripts/install-hooks.sh` points `core.hooksPath` at `.githooks/pre-commit`, so every commit runs the surface-sync + quality (≥80) checks locally — the same gates CI runs. +7. **Keep the surfaces in sync** when adding features. Adding a skill means **adding its row to the README `` table** *and* keeping the prose counts (e.g. "51 skills") in sync across `README.md`, `docs/index.html`, the guide, and `templates/skill-template.md`. `./scripts/check-surface-sync.sh` enforces **both** the counts and the table rows — run it before you open a PR. ## PR style diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 14815b535..3da13505b 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -16,7 +16,8 @@ - [ ] `/deep-audit` finds no new inconsistencies - [ ] Manually exercised the changed skill/agent/hook on a real file - [ ] Updated **both** `README.md` and `guide/workflow-guide.qmd` if user-facing -- [ ] Skill counts agree across `CLAUDE.md`, `README.md`, `docs/index.html`, and the guide +- [ ] `./scripts/check-surface-sync.sh` passes — prose counts **and** `surface-sync-table` rows (added the new skill's row to the README table) are in sync +- [ ] Ran `./scripts/install-hooks.sh` so the pre-commit gate is active locally ## Generality (for new skills/agents/rules) diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml new file mode 100644 index 000000000..5e04478ee --- /dev/null +++ b/.github/workflows/deploy.yml @@ -0,0 +1,36 @@ +name: deploy-guide + +# Renders the workflow guide to docs/ on every push to main that touches the +# guide source, so GitHub Pages stays in sync without a manual `quarto render`. +# Opt-in: enable Actions on your fork and grant Pages write. Forkers who don't +# publish a guide can delete this file. + +on: + push: + branches: [main] + paths: + - "guide/**" + +permissions: + contents: write + +jobs: + render: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: quarto-dev/quarto-actions/setup@v2 + - name: Render guide + run: quarto render guide/workflow-guide.qmd + - name: Sync to docs/ + run: | + cp guide/workflow-guide.html docs/workflow-guide.html + if [ -n "$(git status --porcelain docs/)" ]; then + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git add docs/workflow-guide.html + git commit -m "ci: re-render guide → docs/ [skip ci]" + git push + else + echo "docs/ already up to date." + fi diff --git a/.github/workflows/gates.yml b/.github/workflows/gates.yml new file mode 100644 index 000000000..662198b86 --- /dev/null +++ b/.github/workflows/gates.yml @@ -0,0 +1,27 @@ +name: gates + +# Runs the deterministic, toolchain-free gates on every PR and push to a +# feature branch — the same surface-sync + skill-integrity + model-version +# checks the local pre-commit hook runs. Quality scoring (which needs +# Quarto / R) stays local; CI guards the cheap, high-signal invariants. + +on: + pull_request: + push: + branches-ignore: + - main # main is protected by the PRs that merge into it + +permissions: + contents: read + +jobs: + surface-sync: + name: surface-sync + skill-integrity + model-versions + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Run gate suite + run: ./scripts/check-surface-sync.sh diff --git a/.gitignore b/.gitignore index 2653dba01..434862818 100644 --- a/.gitignore +++ b/.gitignore @@ -55,9 +55,11 @@ CLAUDE.local.md # Quarto rendering artifacts .quarto/ -# quality_reports/ — per-project session logs, plans, specs, merges. -# The directory structure ships in the template (via .gitkeep), but the -# content is PROJECT-INTERNAL and shouldn't be inherited by forks. +# quality_reports/ — per-project session logs, plans, specs, merges, audits, +# diagnoses. The directory structure ships in the template (via .gitkeep), but +# the content is PROJECT-INTERNAL and shouldn't be inherited by forks. +# (diagnoses/ in particular may hold real data values/paths — never ship them; +# audits/ are repo-specific gap analyses, noise for a forker.) quality_reports/plans/* quality_reports/specs/* quality_reports/session_logs/* @@ -65,6 +67,8 @@ quality_reports/merges/* quality_reports/decisions/* quality_reports/checkpoints/* quality_reports/preregistrations/* +quality_reports/audits/* +quality_reports/diagnoses/* !quality_reports/plans/.gitkeep !quality_reports/specs/.gitkeep !quality_reports/session_logs/.gitkeep @@ -72,6 +76,8 @@ quality_reports/preregistrations/* !quality_reports/decisions/.gitkeep !quality_reports/checkpoints/.gitkeep !quality_reports/preregistrations/.gitkeep +!quality_reports/audits/.gitkeep +!quality_reports/diagnoses/.gitkeep # R analysis outputs (generated artifacts; keep scripts, ignore outputs) scripts/R/_outputs/ diff --git a/CHANGELOG.md b/CHANGELOG.md index c86ea7434..f18ff44bf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,83 @@ If you have forked this template, see the **Upgrading** section at the bottom fo --- +## v2.0.0 — 2026-06-09 + +A **paradigm-shift major release.** The template moves from a *prompt-craft contractor* — craft a prompt, invoke a skill, read a report — to a **verification-gated research lab**: you state a goal, a fleet of specialist agents does the labor under **gates that enforce themselves**, and you act as the **auditor of the disagreements they surface**. Two ideas converge: *"loops, not prompts"* (Boris Cherny / the Claude Code team) and *"ground truth is a process, not a dataset"* (Amazon Science). The result modernizes the **orchestration**, not the substance — the passport, simulation contract, and journal-calibrated referees are untouched. Shipped against `quality_reports/plans/2026-06-09_v2.0-modernization-master-plan.md`. + +### ⭐ Why upgrade — read this even if you've used the template before + +If you forked v1.x, here is what is *materially different* now (one-stop summary): + +| Before (v1.x) | Now (v2.0) | +|---|---| +| Quality gate ran **only inside `/commit`** — a direct `git commit` bypassed it | A real **git pre-commit hook** (`./scripts/install-hooks.sh`) runs surface-sync + quality (≥80) on **every** commit | +| The review loop was *"a pattern, not a runtime"* — prose a human triggered | A **real runtime**: fan-out → reduce → judge **+ hallucination gate** → **loop-until-dry** (`orchestrator-protocol.md`) | +| A numeric mismatch was always a **FAIL** (so people disabled the gate) | A defensible, **named** alternative is recorded as **`EXPLAINED`** and flows into your response-to-referees; real errors stay fail-closed | +| Hooks **nagged** ("you should update the log / compile") | Hooks **act**: the Stop hook **auto-writes** the session log; `git-guardrails` **blocks** destructive git; `claim-reconcile` flags stale claims the moment analysis changes | +| 17 of 18 agents ran on `model: inherit` (cost savings unrealized) | Every agent **pinned** to model + effort (Opus referees / Sonnet reviewers / Haiku mechanical) | +| You **audited** reproducibility | You **produce** the deposit: `/replication-package` builds the AEA DCAS / openICPSR package | +| Onboarding said *"paste this starter prompt"* | Onboarding is **goal-first**; the `/prompt` and `/prompt-only` skills are retired (shaping is now ambient) | + +**If you do only one thing after pulling v2.0:** run `./scripts/install-hooks.sh` to activate the pre-commit gate. + +**Inventory at release: 51 skills, 18 agents, 32 rules, 7 hooks** (was 38 / 18 / 28 / 6 at v1.10.0). Net: +15 skills, −2 retired; +4 rules; +2 hooks, −1 retired. New: 9 references (3 added), 2 output-styles, CI. + +### Added — economist "producer" skills (the biggest gold-standard gap) + +- **`/replication-package`** — assembles a submission-ready **AEA DCAS / openICPSR** deposit: standard README, dataset manifest, computational-requirements capture, a Table/Figure → `script:line` map (from the passport), and a confidential-data deposit note. Gates on `/audit-reproducibility` (blocks on FAIL, allows EXPLAINED). *Move from auditing reproducibility to producing the deliverable.* +- **`/capture-environment`** — snapshots the computational environment: `renv.lock` + `sessionInfo` (R), `requirements.txt`/`uv.lock` (Python), Stata version + ado list, seeds/RNG, and an optional pinning `Dockerfile`. +- **`/did-event-study`** — a **thin wrapper** over canonical staggered-DiD packages (Callaway–Sant'Anna `did`, Sun–Abraham `fixest::sunab`, HonestDiD; Stata twins) that surfaces each package's *native* diagnostics and **never reimplements an estimator**. Warns on the never-treated vs not-yet-treated control choice (the contested decision the EXPLAINED disposition exists for). +- **`/power-analysis`** — power / required-N / MDE for two-arm RCTs (clustering/ICC), multi-arm, and simulation-based designs; feeds `/preregister`. +- **`/disclosure-check`** — statistical-disclosure-limitation pre-screen for restricted-data outputs (small cells, dominance, PII); gates on CRITICAL. +- **`/grant-proposal`** + **`/data-management-plan`** — compose the confidential-data + environment-capture primitives into NSF/NIH/ERC-shaped drafts + requirements checklists. +- **`/coauthor-brief`** — a collaborator/multi-machine handoff brief (git delta, per-artifact state, reproduce-locally + restricted-data access steps). +- **`confidential-data.md` rule** — restricted/IRB-data protocol (never commit raw data; disclosure clearance; restricted-data-safe multi-author git topology). **AEA Data Editor / DCAS policy** added to `journal-profiles.md`. + +### Added — teaching-at-scale (the repo's origin, previously a blind spot) + +- **`/syllabus`**, **`/teach-from-paper`**, **`/respond-to-eval`** (the teaching analogue of `/respond-to-referees`), and **`/scaffold-exercises`** (graded problem sets with worked solutions + explainers, adapted from `mattpocock/skills`). + +### Added — autonomy, runtime, and authoring + +- **Real orchestration runtime** — `.claude/references/orchestration-schemas.md` (the `FINDING`/`SCORECARD`/`RUN_CONFIG` contracts + the post-judge **hallucination gate**) and `agent-fleet.md` (the 18-agent manifest with tiers). `orchestrator-protocol.md` rewritten from *pattern* to *runtime*. +- **Event-driven + scheduled autonomy** — `claim-reconcile.py` (flags stale numeric claims when an analysis script/output changes), `.claude/references/scheduled-routines.md` + `scripts/nightly-repro-check.sh`, and **`/triage-inbox`** (schedulable, human-gated email/calendar triage + referee-obligations tracker). +- **`/new-skill`** — scaffolds a convention-compliant skill that passes `check-skill-integrity` on the first try (adapted from `mattpocock/skills`' write-a-skill). +- **`/diagnose`** — root-cause a wrong/failing empirical result with a disciplined **reproduce → minimise → hypothesise → instrument → fix** loop (the net-new idea from a fresh pass over `mattpocock/skills`, reshaped for research code where the bug is usually a *silent wrong number*, not a crash; competing-hypothesis fan-out via `Task`, `--no-fix` to localize without editing). +- **Output styles** — `.claude/output-styles/academic-writing.md` and `referee.md` (the template shipped zero before). +- **CI** — `.github/workflows/gates.yml` (surface-sync + integrity + model-version checks on every PR) and `deploy.yml` (re-render the guide → `docs/` on push to `main`). +- **Enforcing pre-commit hook** — `.githooks/pre-commit` + `scripts/install-hooks.sh` (opt-in via `core.hooksPath`). +- **`git-guardrails.py`** (PreToolUse) — blocks `git reset --hard`, `git clean -f`, `git push --force`, and blanket `git add -A`; warns on hardcoded machine paths in code (`CLAUDE_STRICT_PATHS=1` to hard-block). +- **Table-row sync gate** — `check-surface-sync.py` now verifies that the enumerative skills/agents *tables* (not just prose counts) carry one row per item on disk — the drift that hid the v1.5.0 peer-review trio for three releases. + +### Changed + +- **Verification 2.0 — "ground truth is a process."** `audit-reproducibility` and `claim-verifier` gain an **`EXPLAINED`** disposition: an out-of-tolerance numeric/citation mismatch stays a fail-closed FAIL **unless** the author records a *concrete named alternative* (e.g. "never-treated vs not-yet-treated comparison group"), in which case it is surfaced, non-blocking, and carried into the response-to-referees. The hard floor holds (fabricated citations, unmatched claims, vague notes never downgrade). `audit-reproducibility` also no longer presumes the manuscript is the oracle — a mismatch means "one of {paper, code} must change." +- **Loop-until-dry.** `qa-quarto`, `review-paper --adversarial`, `deep-audit`, and `seven-pass-review` converge when a round surfaces no new CRITICAL/MAJOR finding (2 dry rounds), with the old "max 5 rounds" demoted to a fallback cap; synthesizers run the **post-judge hallucination gate** (an editor can't desk-reject on a reason no referee raised). +- **Hooks act instead of nagging.** `log-reminder.py` now **auto-writes** a structured session-log entry on each meaningful change-set (was a stderr reminder). `pre-compact.py`'s DRAFT-plan block is **ON by default**. The statusline now shows context-% + dirty-file count + plan status. +- **Per-agent model routing realized.** All 18 agents pinned to `model:` + `effort:` per `model-routing.md` (9 Opus / 8 Sonnet / 1 Haiku). +- **Goal-first onboarding.** The README "How It Works" leads with *goal-first, gate-enforced*; the guide and landing page reframed accordingly. "Contractor mode" remains, augmented. + +### Removed + +- **`/prompt` and `/prompt-only` skills** — retired. Prompt-craft is 2023-era; a loop-first workflow shapes every ambiguous request automatically, so this is now the ambient **`prompt-shaping.md`** rule, not a command. (`prompt-formatting-core.md` re-pointed; `/interview-me` remains for multi-turn specification.) +- **`verify-reminder.py` hook** — retired. Its per-edit "compile reminder" is superseded by the Stop-hook completion note in the auto-writing session log. + +### Upgrading (forkers pulling v2.0) + +1. **`./scripts/install-hooks.sh`** — activate the pre-commit gate (the single highest-value step). +2. If you customized `/prompt` or `/prompt-only`, move that logic into your own rule or `/interview-me` — the skills are gone. +3. If you reference `verify-reminder.py` in a fork, remove it; the Stop hook (`log-reminder.py`) now writes logs. +4. Pin `model:`/`effort:` in any custom agents (`model-routing.md`), and re-run `./scripts/check-surface-sync.sh` after adding skills — it now checks table rows too. +5. No data migrations; the passport schema is backward-compatible (`EXPLAINED` is additive). + +### Notes + +- Count surfaces (README, CLAUDE.md, guide source + rendered HTML, landing page, skill template) updated to 51 / 18 / 32 / 7; `check-surface-sync.sh` (now including the table-row gate), `check-skill-integrity.py`, and `check-model-versions.sh` all pass. +- Explicit **non-goals** (documented, not omissions): no autonomous daemon, no plugin marketplace, no multi-estimator production fleet (one `/did-event-study` proof-of-concept), no challenger→auditor→ledger pipeline (the zero-cost `EXPLAINED` mechanism instead), `MEMORY.md` stays the committed memory backend. + +--- + ## v1.10.0 — 2026-05-31 A **hub-expansion + currency-refresh** minor release. The template gains a Monte Carlo simulation capability and an R package-development release gate, refreshes the model / effort / cost guidance for **Opus 4.8**, and reframes itself as a hub for an entire research program — not just slides and papers. Shipped against the plan at `quality_reports/plans/2026-05-31_v1.10.0-simulation-and-hub.md` (local-only per the `quality_reports/plans/*` ignore rule). No breaking changes. diff --git a/CLAUDE.md b/CLAUDE.md index 7acf2fe23..befd2de3e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -78,52 +78,21 @@ python scripts/quality_score.py Quarto/file.qmd | 90 | PR | Ready for deployment | | 95 | Excellence | Aspirational | -Enforced by `/commit` (halts + asks for override); not enforced by a git pre-commit hook. +Enforced by `/commit` (halts + asks for override) **and** — once you run `./scripts/install-hooks.sh` — by a real git pre-commit hook (`.githooks/pre-commit`) that runs the surface-sync + quality (≥80) gates on every commit. Bypass sparingly with `SKIP_QUALITY_GATE=1` or `--no-verify`. --- ## Skills Quick Reference -| Command | What It Does | -|---------|-------------| -| `/compile-latex [file]` | 3-pass XeLaTeX + bibtex | -| `/deploy [LectureN]` | Render Quarto + sync to docs/ | -| `/extract-tikz [LectureN]` | TikZ → PDF → SVG | -| `/new-diagram [snippet] [output.tex]` | Scaffold a TikZ diagram from the gallery with prevention + review | -| `/proofread [file]` | Grammar/typo/overflow review | -| `/visual-audit [file]` | Slide layout audit | -| `/pedagogy-review [file]` | Narrative, notation, pacing review | -| `/review-r [file]` | R code quality review | -| `/qa-quarto [LectureN]` | Adversarial Quarto vs Beamer QA | -| `/slide-excellence [file]` | Combined multi-agent review | -| `/translate-to-quarto [file]` | Beamer → Quarto translation | -| `/validate-bib` | Cross-reference citations | -| `/devils-advocate` | Challenge slide design | -| `/create-lecture` | Full lecture creation | -| `/commit [msg]` | Stage, commit, PR, merge | -| `/lit-review [topic]` | Literature search + synthesis | -| `/research-ideation [topic]` | Research questions + strategies | -| `/interview-me [topic]` | Interactive research interview | -| `/review-paper [file]` | Manuscript review (single-pass / `--adversarial` / `--peer ` simulated pipeline) | -| `/respond-to-referees [report] [manuscript]` | R&R cross-reference + response draft | -| `/data-analysis [dataset]` | End-to-end R analysis | -| `/audit-reproducibility [paper]` | Enforce replication tolerance thresholds on paper ↔ code | -| `/learn [skill-name]` | Extract discovery into persistent skill | -| `/context-status` | Show session health + context usage | -| `/deep-audit` | Repository-wide consistency audit | -| `/permission-check` | Diagnose permission layers when prompts fire unexpectedly | -| `/seven-pass-review` | Seven-pass adversarial manuscript review (parallel forked subagents) | -| `/verify-claims [file]` | Chain-of-Verification fact-check (forked verifier, fresh context) | -| `/checkpoint [topic]` | Save a structured state snapshot (active plan, decisions, file pointers, next actions) before stopping or handing off | -| `/preregister [--style osf|aspredicted|aea-rct]` | Draft a preregistration document (OSF / AsPredicted / AEA RCT Registry) from a research spec | -| `/humanize [file]` | Detect AI-voice tells in academic prose (read-only audit; no rewrite) | -| `/prompt [text] [depth:light|standard|deep]` | Reformat informal input into a structured six-section prompt, then execute | -| `/prompt-only [text] [depth] [--save path]` | Same formatting as `/prompt`, but emits the prompt as a reusable artifact (no execution) | -| `/compress-session [slug]` | Distil current session into structured notes before auto-compaction (vs `/checkpoint` for natural stops) | -| `/promote-memory [filter]` | Five-critic council that votes on which `[LEARN]` entries graduate from personal-memory.md to MEMORY.md | -| `/stata-replication [paper-or-data]` | End-to-end Stata pipeline scaffold + execution via `stata-mcp` (mirrors `/data-analysis` for R) | -| `/simulation-study [estimator+DGP]` | Reproducible Monte Carlo study: DGP, estimator grid, seeded reps, bias/RMSE/coverage/size/power + Monte Carlo SEs | -| `/r-package-check [pkg path]` | R package release gate: `devtools::document()` + tests + `R CMD check --as-cran`, CRAN-policy triage, `r-package-reviewer` pass | +The full table of all skills lives in [README.md](README.md#skills-claudeskills). Most-used, by workflow: + +- **Slides / teaching:** `/create-lecture` `/compile-latex` `/deploy` `/qa-quarto` `/slide-excellence` `/syllabus` `/teach-from-paper` `/scaffold-exercises` +- **Papers / review:** `/review-paper` (`--peer`) `/seven-pass-review` `/respond-to-referees` `/verify-claims` `/proofread` `/humanize` +- **Data / reproducibility:** `/data-analysis` `/did-event-study` `/simulation-study` `/audit-reproducibility` `/diagnose` `/replication-package` `/capture-environment` `/power-analysis` `/disclosure-check` +- **Research / writing:** `/interview-me` `/lit-review` `/research-ideation` `/preregister` `/grant-proposal` `/data-management-plan` +- **Meta / workflow:** `/commit` `/learn` `/new-skill` `/checkpoint` `/context-status` `/deep-audit` `/coauthor-brief` `/triage-inbox` + +Stata (`/stata-replication`), R packages (`/r-package-check`), TikZ (`/extract-tikz`, `/new-diagram`), and more — see the README for the complete index. --- diff --git a/README.md b/README.md index f846516b7..f7109e568 100644 --- a/README.md +++ b/README.md @@ -72,9 +72,19 @@ If both succeed, delete `Slides/HelloWorld.tex` and `Quarto/HelloWorld.qmd` and ## How It Works +### Goal-first, gate-enforced (the v2.0 shift) + +You don't craft a perfect prompt — you **state a goal and let the work loop toward it under gates**. Specialist agents do the labor; enforcing gates decide when it's good enough; you adjudicate the disagreements they surface. Three things make that trustworthy: + +- **Real gates, not reminders.** A version-controlled pre-commit hook (run `./scripts/install-hooks.sh` once) runs the surface-sync + quality (≥80) checks on *every* commit — bypassing the skill no longer bypasses the review. A `git-guardrails` hook blocks destructive git (`reset --hard`, `clean -f`, `push --force`, `add -A`); the review runtime re-checks any reviewer-introduced "fatal" finding before it counts. +- **A real orchestration runtime.** Reviews fan out to forked specialist agents, reduce over a shared finding schema, judge with a hallucination gate, and loop until dry — see [`orchestrator-protocol.md`](.claude/rules/orchestrator-protocol.md). +- **Ground truth as a process.** A mismatch isn't always a failure: a defensible, *named* alternative is recorded as `EXPLAINED` and carried into your response-to-referees, while genuine errors stay fail-closed. + +This is **not** an autonomous daemon — the loop is always you- or skill-initiated, and you stay the auditor. Scheduled [Routines](.claude/references/scheduled-routines.md) handle recurring chores (nightly reproducibility, weekly lit-delta, inbox triage) and notify only when they find something. + ### Contractor Mode -You describe a task. For complex or ambiguous requests, Claude first creates a requirements specification with MUST/SHOULD/MAY priorities and clarity status (CLEAR/ASSUMED/BLOCKED). You approve the spec, then Claude plans the approach and invokes the right skill (e.g. `/create-lecture`, `/qa-quarto`, `/review-paper --adversarial`). That skill implements the orchestrator pattern internally — implement, verify, review, fix, re-verify, score — and returns a summary when the work meets quality standards. Say "just do it" and it auto-commits when the score clears 80. +You describe a task. For complex or ambiguous requests, Claude first creates a requirements specification with MUST/SHOULD/MAY priorities and clarity status (CLEAR/ASSUMED/BLOCKED). You approve the spec, then Claude plans the approach and invokes the right skill (e.g. `/create-lecture`, `/qa-quarto`, `/review-paper --adversarial`). That skill implements the orchestrator runtime internally — implement, verify, review, fix, re-verify, score — and returns a summary when the work meets quality standards. Say "just do it" and it runs the full loop; commits still require an explicit `/commit` (which the pre-commit hook then gates). ### Specialized Agents @@ -91,7 +101,7 @@ Each is better at its narrow task than a generalist would be. The `/slide-excell ### Adversarial QA -Two agents work in opposition: the **critic** reads both Beamer and Quarto and produces harsh findings. The **fixer** implements exactly what the critic found. They loop until the critic says "APPROVED" (or 5 rounds max). This catches errors that single-pass review misses. +Two agents work in opposition: the **critic** reads both Beamer and Quarto and produces harsh findings. The **fixer** implements exactly what the critic found. They **loop until dry** — converging when a round surfaces no new issue (a 5-round cap is the fallback, not the primary stop). This catches errors that single-pass review misses. ### Quality Review @@ -101,7 +111,7 @@ Every artifact gets a score (0–100). Scores below threshold halt the workflow - **90** — PR threshold - **95** — excellence (aspirational) -> **Framing honesty:** Thresholds are advisory at the harness level — the `/commit` skill runs quality checks and halts on failure, but there is no pre-commit git hook that blocks a direct `git commit`. If you bypass the skill, you bypass the review. For hard enforcement, configure a git pre-commit hook. +> **Framing honesty:** Thresholds are advisory at the harness level — the `/commit` skill runs quality checks and halts on failure. **And** as of v2.0, running `./scripts/install-hooks.sh` once installs a real pre-commit hook (`.githooks/pre-commit`) that runs the surface-sync + quality (≥80) gates on *every* commit, so bypassing the skill no longer bypasses the review. Opt out per-commit with `SKIP_QUALITY_GATE=1` or `git commit --no-verify`. ### Context Survival @@ -178,10 +188,11 @@ This workflow is designed as a **single hub for an entire research program** — ## What's Included

-18 agents, 38 skills, 28 rules, 6 hooks (click to expand) +18 agents, 51 skills, 32 rules, 7 hooks (click to expand) ### Agents (`.claude/agents/`) + | Agent | What It Does | |-------|-------------| | `proofreader` | Grammar, typos, overflow, consistency review | @@ -205,6 +216,7 @@ This workflow is designed as a **single hub for an entire research program** — ### Skills (`.claude/skills/`) + | Skill | What It Does | |-------|-------------| | `/compile-latex` | 3-pass XeLaTeX compilation with bibtex | @@ -214,7 +226,7 @@ This workflow is designed as a **single hub for an entire research program** — | `/visual-audit` | Launch slide-auditor on a file | | `/pedagogy-review` | Launch pedagogy-reviewer on a file | | `/review-r` | Launch R code reviewer | -| `/qa-quarto` | Adversarial critic-fixer loop (max 5 rounds) | +| `/qa-quarto` | Adversarial critic-fixer loop (loops until dry; 5-round cap is a fallback) | | `/slide-excellence` | Combined multi-agent review | | `/translate-to-quarto` | Full 11-phase Beamer-to-Quarto translation | | `/validate-bib` | Cross-reference citations against bibliography | @@ -238,13 +250,26 @@ This workflow is designed as a **single hub for an entire research program** — | `/preregister` | Generate a preregistration document (OSF / AsPredicted / AEA RCT Registry style) from a research spec | | `/verify-claims` (v1.7.0) | Chain-of-Verification fact-check (forked verifier, fresh context). HIGH/MED/LOW-WARN severity tiers (v1.9.0); HIGH-WARN gate-refuses `/commit`. | | `/humanize` (v1.9.0) | Detect AI-voice tells in academic prose (10 detection categories; read-only, no rewrite) | -| `/prompt` (v1.9.0) | Reformat informal/dictated input into a structured six-section prompt, then execute (ported from Blattman with stripping) | -| `/prompt-only` (v1.9.0) | Same formatting as `/prompt` but emits the prompt as a reusable artifact (no execution) | | `/compress-session` (v1.9.0) | Distil current session into structured notes (decisions, next actions, *discarded-as-noise*) before auto-compaction | | `/promote-memory` (v1.9.0) | Five-critic council that votes on which `[LEARN]` entries graduate from personal-memory.md to MEMORY.md | | `/stata-replication` (v1.9.0) | End-to-end Stata pipeline via the `stata-mcp` MCP server (mirrors `/data-analysis` for R-first projects) | | `/simulation-study` (v1.10.0) | Scaffold + run a reproducible Monte Carlo study — parameterized DGP, estimator grid, seeded replications, bias/RMSE/coverage/size/power with Monte Carlo SEs | | `/r-package-check` (v1.10.0) | R package release gate — `devtools::document()` + tests + `R CMD check --as-cran`, triage ERROR/WARNING/NOTE vs CRAN policy, `r-package-reviewer` pass | +| `/replication-package` (v2.0) | Assemble a submission-ready DCAS / openICPSR replication package — standard README, dataset manifest, computational-requirements capture, Table/Figure → script:line map, confidential-data deposit note (blocks on `/audit-reproducibility` FAIL) | +| `/capture-environment` (v2.0) | Snapshot the computational environment for a replication package — renv.lock + sessionInfo.txt (R), requirements.txt / environment.yml / uv.lock (Python), Stata version + ado list, seeds/RNG, optional pinning Dockerfile | +| `/did-event-study` (v2.0) | Thin wrapper for staggered DiD / event-study via canonical packages (Callaway–Sant'Anna `did`, Sun–Abraham `fixest::sunab`, HonestDiD sensitivity; Stata equivalents) — surfaces each package's native diagnostics, never reimplements an estimator | +| `/power-analysis` (v2.0) | Power / required-N / minimum-detectable-effect for study design — two-arm RCT (clustering/ICC, unequal allocation), multi-arm corrections, simulation-based power for non-standard designs; feeds `/preregister` | +| `/disclosure-check` (v2.0) | Statistical-disclosure-limitation pre-screen for restricted/confidential-data outputs (small cells, complementary-suppression gaps, dominance, PII); CRITICAL/WARNING/OK + gate | +| `/grant-proposal` (v2.0) | Scaffold an NSF/NIH/ERC/foundation grant proposal by composing primitives (spec → aims/methods, delegated DMP + facilities, coherence pass + requirements checklist) | +| `/data-management-plan` (v2.0) | Funder-compliant Data Management Plan (NSF / NIH DMS 2023 / ERC / Horizon Europe) — folds in disclosure-avoidance + IRB constraints and a replication-package/environment plan; outputs a draft + funder checklist | +| `/coauthor-brief` (v2.0) | Collaborator handoff brief — what changed since last brief, per-artifact state, open questions, reproduce-locally + restricted-data access steps | +| `/triage-inbox` (v2.0) | Schedulable academic inbox + calendar triage via Gmail/Calendar MCP — classifies referee requests, R&R/editor, co-author threads, seminar/conference invites, grant/admin deadlines; proposes one human-gated action each (draft reply, calendar hold, `/new-referee-project`, `/coauthor-brief`, snooze); emits a digest + referee-obligations tracker; degrades gracefully when MCP is absent; never auto-sends | +| `/diagnose` (v2.0) | Root-cause a wrong/failing empirical result — disciplined reproduce → minimise → hypothesise → instrument → fix loop; tuned for research-code bugs (type coercion, NA/merge blow-ups, clustering/SE choice, seed/package-version drift); `--no-fix` localizes without editing | +| `/syllabus` (v2.0) | Build/restructure a course syllabus from a topic or reading list — course description + prerequisites, week-by-week schedule (topic→readings→deliverables), measurable learning objectives, assessment scheme + rubric, standard policies (late work / AI use / integrity / accessibility), and a per-week work-list mapping weeks to `/create-lecture` decks; economics-aware (PhD metrics/micro/macro sequences, undergrad) | +| `/teach-from-paper` (v2.0) | Reads a paper end-to-end and pitches it to a stated audience level — lecture outline (motivation → setup → key result → method → takeaways), the 3-5 results worth presenting with intuition, a slide skeleton for `/create-lecture`, discussion questions, and a problem-set brief for `/scaffold-exercises` | +| `/respond-to-eval` (v2.0) | Teaching analogue of `/respond-to-referees` — clusters course-eval comments into themes, weights by frequency (signal vs noise), classifies Keep / Change / Investigate / Out-of-scope, and drafts concrete changes mapped to the syllabus + slide decks; saves the plan to `quality_reports/teaching/` | +| `/scaffold-exercises` (v2.0) | Scaffold a graded problem set across analytical/empirical/coding types, with worked solutions and "why this matters" explainers emitted to a separate solution key | +| `/new-skill` (v2.0) | Scaffold a new skill that follows this repo's conventions — interviews for purpose, triggers, and tools, writes `.claude/skills//SKILL.md` from the template with frontmatter/body that pass `check-skill-integrity.py` first try, then reminds to add the surface-table rows | ### Research Workflow @@ -267,9 +292,10 @@ Rules use path-scoped loading: **always-on** rules load every session (~100 line | Rule | What It Enforces | |------|-----------------| | `plan-first-workflow` | Plan mode for non-trivial tasks + context preservation | -| `orchestrator-protocol` | Contractor mode: implement → verify → review → fix → score | +| `orchestrator-protocol` | Goal-first review runtime: fan-out → reduce → judge (+ hallucination gate) → loop-until-dry (the contractor loop, now a real runtime) | | `session-logging` | Three logging triggers: post-plan, incremental, end-of-session | | `meta-governance` | Template vs. working project distinctions | +| `prompt-shaping` (v2.0) | Ambient habit — shape informal/ambiguous requests before acting (replaces the retired `/prompt` + `/prompt-only` skills) | **Path-scoped** (load only when working on matching files): @@ -299,6 +325,9 @@ Rules use path-scoped loading: **always-on** rules load every session (~100 line | `stata-code-conventions` (v1.9.0) | `**/*.do`, `scripts/stata/**` | Stata header scaffold, numbered pipeline, esttab, clustering discipline, AEA compliance | | `simulation-conventions` (v1.10.0) | `**/*simulation*.R`, `**/*_sim.R`, `explorations/**` | Monte Carlo discipline: DGP/estimand, L'Ecuyer seeding, Monte Carlo SE, coverage-vs-truth, raw-result storage | | `r-package-conventions` (v1.10.0) | `R/**`, `tests/**`, `DESCRIPTION`, `NAMESPACE`, `man/**` | R package-source standards: no `library()` in `R/`, roxygen NAMESPACE, Imports/Suggests, testthat 3e, CRAN policy | +| `confidential-data` (v2.0) | `data/**`, `**/*.dta`, `**/restricted/**`, `**/confidential/**` | Restricted/IRB-data protocol: never commit raw data, disclosure clearance before release, restricted-data-safe multi-author git topology | +| `did-conventions` (v2.0) | `**/*did*.R`, `**/*event*study*.R`, `**/*att_gt*`, `**/*csdid*.do`, `**/*drdid*` | DiD/event-study standards (Sant'Anna): LONG data + gname coding, doubly-robust default, control-group rule, uniform-band inference, mandatory pre-trend/HonestDiD/didFF diagnostics, replicate-and-verify-to-1e-6 | +| `inference-robustness` (v2.0) | `scripts/**/*.R`, `**/*.do`, `**/*.py` | Multiple-testing (FWER/Romano-Wolf vs FDR/Anderson sharpened-q, pre-register the family) + specification-curve / leave-one-out / wild-cluster-bootstrap robustness | ### Templates (`templates/`) @@ -387,7 +416,7 @@ See the [guide's ecosystem section](https://psantanna.com/claude-code-my-workflo - **What's new:** see [CHANGELOG.md](CHANGELOG.md). We follow loose semver — breaking changes get major bumps so you can decide when to pull updates. - **How to contribute:** see [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md). PRs welcome for generalizable improvements; fork-specific work stays in your fork. -- **Pin to a version:** `git checkout v1.10.0` (current as of 2026-05-31). +- **Pin to a version:** `git checkout v2.0.0` (current as of 2026-06-09). --- diff --git a/docs/index.html b/docs/index.html index 88b2405c1..2d812e0d1 100644 --- a/docs/index.html +++ b/docs/index.html @@ -12,7 +12,7 @@ - + @@ -194,7 +194,7 @@

My Claude Code Setup

Actively developed and maintained. This summarizes how I use Claude Code for academic work — slides, papers, data analysis, and more. Adopted by 15+ research groups across economics, energy, political science, and engineering. I keep learning and updating these files. -

A foundation for AI-assisted academic work using Claude Code. You describe what you want — lecture slides, a research paper, a data analysis, a replication package — and Claude plans the approach, runs specialized agents, fixes issues, verifies quality, and presents results. Like a contractor who handles the entire job.

+

A foundation for AI-assisted academic work using Claude Code. You describe what you want — lecture slides, a research paper, a data analysis, a replication package — and Claude plans the approach, runs specialized agents, fixes issues, verifies quality, and presents results — goal-first and gate-enforced. Like a contractor who handles the entire job, with you as the auditor of the disagreements the review loop surfaces. It is deliberately not an autonomous daemon: every loop is started by you or a skill, never on its own.

Read the full guide @@ -204,20 +204,20 @@

What's in the template

  • Full academic lifecycle — not just slides: research papers, data analysis, Monte Carlo simulation studies, replication packages, proposals, and presentations
    • -
  • Contractor mode via skills — you describe the task; Claude picks a skill (/create-lecture, /review-paper, /qa-quarto, …) that runs the orchestrator pattern internally: plan, implement, review with agents, fix, re-verify, score against quality thresholds
    • +
  • Contractor mode via skills — you describe the task; Claude picks a skill (/create-lecture, /review-paper, /qa-quarto, …) that runs the orchestrator runtime internally: plan, implement, review with agents via fan-out → reduce → judge (with a post-judge hallucination gate), fix, re-verify, score against quality thresholds — looping until dry
  • Beamer → Quarto pipeline — 11-phase translation with TikZ-to-SVG and ggplot-to-plotly conversion
    • -
  • 18 specialized agents — proofreader, slide auditor, pedagogy reviewer, R reviewer, TikZ critic, domain reviewer, adversarial QA pair, translator, verifier, claim-verifier (Chain-of-Verification), the simulated-peer-review trio (editor, domain referee, methods referee), the Monte Carlo sim-reviewer, the CRAN-focused R package-reviewer, the AI-voice humanize-auditor, and the promote-memory council
    • -
  • Adversarial critic-fixer loop — two agents that check each other's work across up to 5 rounds — the critic can't fix, the fixer can't approve
    • +
  • 18 specialized agents — proofreader, slide auditor, pedagogy reviewer, R reviewer, TikZ critic, domain reviewer, adversarial QA pair, translator, verifier, claim-verifier (Chain-of-Verification), the simulated-peer-review trio (editor, domain referee, methods referee), the Monte Carlo sim-reviewer, the CRAN-focused R package-reviewer, the AI-voice humanize-auditor, and the promote-memory council — all pinned to a fixed model + effort tier (no longer inherit)
    • +
  • Adversarial critic-fixer loop — two agents that check each other's work and loop until dry (converge after 2 consecutive clean rounds; a 5-round cap is only a fallback) — the critic can't fix, the fixer can't approve
  • Quality scoring with mandatory verification — every file scored 0–100; /commit halts below 80 (user can override with an explicit reason); skills that implement the orchestrator pattern verify every output before reporting done
    • -
  • 38 slash commands + 28 context-aware rules — /compile-latex, /proofread, /deploy, /commit, /qa-quarto, /lit-review, /review-paper, /respond-to-referees, /new-diagram, /data-analysis, /simulation-study, /r-package-check, /audit-reproducibility, /checkpoint, /preregister, and more — plus quality gates, TikZ prevention/measurement, notation consistency, and R conventions
    • +
  • 51 slash commands + 32 context-aware rules + 7 hooks — /compile-latex, /proofread, /deploy, /commit, /qa-quarto, /lit-review, /review-paper, /respond-to-referees, /new-diagram, /data-analysis, /simulation-study, /r-package-check, /audit-reproducibility, /checkpoint, /preregister, /replication-package, /disclosure-check, /grant-proposal, /syllabus, /triage-inbox, and more — plus quality gates, TikZ prevention/measurement, notation consistency, R conventions, and the ambient prompt-shaping rule (no /prompt command)
  • Simulated peer review — /review-paper --peer <journal> runs a full editorial pipeline: editor desk review, two blind referees with deliberately different dispositions (STRUCTURAL / CREDIBILITY / MEASUREMENT / POLICY / THEORY / SKEPTIC), editorial synthesis with FATAL / ADDRESSABLE / TASTE classification. Calibrated to 5 econ journals (AER / QJE / JPE / ECMA / ReStud) plus a template for adding your own field. Adapted from Hugo Sant’Anna’s clo-author with permission.
  • Research workflow skills — /lit-review for literature synthesis, /research-ideation for hypothesis generation, /interview-me to formalize ideas, /review-paper for manuscript review, /data-analysis for end-to-end R analysis
    • -
  • Smart hooks — Desktop notifications (macOS/Linux), pre-compaction context snapshots to session logs, progressive context-usage warnings, post-edit verify reminders
    • +
  • Smart hooks (7) — desktop notifications (macOS/Linux); pre-compaction context snapshots to session logs (DRAFT-block default ON); progressive context-usage warnings; log-reminder.py auto-writes the session log on every meaningful change-set; git-guardrails.py blocks dangerous git ops (reset --hard, clean -f, push --force, add -A); claim-reconcile.py flags stale numeric claims when scripts change; plus a real git pre-commit hook (.githooks/pre-commit via ./scripts/install-hooks.sh) that runs the surface-sync + quality gates
  • TikZ diagram pipeline — 8-snippet gallery (DAGs, DiD, event study, timeline, regression scatter, flowchart, supply-demand), /new-diagram scaffold with prevention pre-checks, production Beamer preamble with palette-synced Quarto theme, measurement-based collision audits via tikz-reviewer
  • Plan-first workflow with continuous learning — non-trivial tasks start in plan mode; [LEARN:tag] corrections persist in MEMORY.md across sessions; plans survive context compression
  • Research exploration workflow — structured explorations/ sandbox with fast-track prototyping (60/100 threshold), simplified orchestrator for R/research, and merge-only quality reporting
    • -
  • Automated session log enforcement — a lightweight Stop hook reminds Claude to keep session logs current; Beamer-Quarto sync enforced via auto-loaded rules
    • -
  • 2026 features — effort levels for cost control, permission modes, expanded hook events, skill frontmatter (effort, context: fork), /batch for parallel refactoring, plugins, and headless CLI mode
    • +
  • Automated session logging — a Stop hook (log-reminder.py) auto-writes the session log on every meaningful change-set rather than merely nagging; Beamer-Quarto sync enforced via auto-loaded rules
    • +
  • 2026 features — effort levels for cost control, permission modes, expanded hook events, skill frontmatter (effort, context: fork), /batch for parallel refactoring, plugins, headless CLI mode, output-styles (academic-writing, referee), CI via .github/workflows, and scheduled Routines (nightly-repro-check.sh)
  • Community ecosystem — extended by clo-author, claudeblattman, MixtapeTools, autoresearch, and ClaudeCodeTools — 15+ research groups have adopted this workflow
diff --git a/docs/workflow-guide.html b/docs/workflow-guide.html index 568371f97..630538711 100644 --- a/docs/workflow-guide.html +++ b/docs/workflow-guide.html @@ -2380,7 +2380,7 @@

Table of contents

  • 3.1 Why Specialized Agents Beat One-Size-Fits-All
  • 3.2 The Adversarial Pattern: Critic + Fixer
    • -
  • 3.3 The Orchestrator: A Pattern, Not a Daemon
    • +
  • 3.3 The Orchestrator: A Runtime, Not a Daemon
  • 3.4 Quality Review: The 80/90/95 System
    • 3.4.1 How Scores Are Calculated
      • @@ -2422,7 +2422,7 @@

      Table of contents

    • 4.8 Memory — Cross-Session Persistence
      • 4.8.1 Plans — Compression-Resistant Task Memory
        • -
      • 4.8.2 Session Logs — Why-Not-Just-What History (with Automated Reminders)
        • +
      • 4.8.2 Session Logs — Why-Not-Just-What History (auto-written)
      • 4.8.3 How It All Fits Together
      • 4.8.4 Hooks — Automated Enforcement
      • 4.8.5 Context Survival System (Advanced)
        • @@ -2558,7 +2558,7 @@

        My Claude Code Setup

        Modified
        -

        May 31, 2026

        +

        June 9, 2026

        @@ -2614,8 +2614,8 @@

        -Orchestrator pattern -Specific skills (/qa-quarto, /review-paper --adversarial, /create-lecture, …) run a 6-step verify-review-fix-score loop internally +Orchestrator runtime +Specific skills (/qa-quarto, /review-paper --adversarial, /create-lecture, …) run a real fan-out → reduce → judge → loop-until-dry runtime internally Multi-agent workflows @@ -2757,7 +2757,7 @@

        -

        You talk, Claude orchestrates. The 18 agents, 38 skills, and 28 rules exist so you don’t have to think about them. Describe your goal, approve the plan, and let the system work.

        +

        You talk, Claude orchestrates. The 18 agents, 51 skills, and 32 rules exist so you don’t have to think about them. Describe your goal, approve the plan, and let the system work.

        @@ -2770,7 +2770,7 @@

        -

        This guide describes the full system — 18 agents, 38 skills, 28 rules. That is the ceiling, not the floor. Start with just CLAUDE.md and 2–3 skills (/compile-latex, /proofread, /commit). Add rules and agents as you discover what you need. The template is designed for progressive adoption: fork it, fill in the placeholders, and start working. Everything else is there when you’re ready.

        +

        This guide describes the full system — 18 agents, 51 skills, 32 rules. That is the ceiling, not the floor. Start with just CLAUDE.md and 2–3 skills (/compile-latex, /proofread, /commit). Add rules and agents as you discover what you need. The template is designed for progressive adoption: fork it, fill in the placeholders, and start working. Everything else is there when you’re ready.

        @@ -3124,7 +3124,7 @@

        @@ -3154,15 +3154,15 @@

        -

        3.3 The Orchestrator: A Pattern, Not a Daemon

        +
        +

        3.3 The Orchestrator: A Runtime, Not a Daemon

        Individual agents are specialists. Skills like /slide-excellence and /qa-quarto coordinate a few agents for specific tasks. In day-to-day work, you don’t have to think about which agents to run — the right skill runs the right agents for you.

        -

        The orchestrator protocol (.claude/rules/orchestrator-protocol.md) defines the 6-step loop (IMPLEMENT → VERIFY → REVIEW → FIX → RE-VERIFY → SCORE) that skills implement internally. It is not a runtime daemon: plan approval does NOT auto-trigger this loop. You invoke a skill (e.g., /create-lecture, /qa-quarto, /review-paper --adversarial) and the skill runs the pattern within its own scope.

        +

        The orchestrator protocol (.claude/rules/orchestrator-protocol.md) is a real runtime, built on the Task subagent primitive every Claude Code session has: skills fan out to forked reviewers, reduce their typed findings through a deterministic gate, judge with a post-judge hallucination guard, and loop until dry. What is not automatic is the trigger: nothing launches this loop on its own, and there is no background daemon — plan approval does NOT auto-trigger it. You invoke a skill (e.g., /create-lecture, /qa-quarto, /review-paper --adversarial) and the skill runs the runtime within its own scope. The human is the auditor of the disagreements the loop surfaces; this is explicitly not an autonomous daemon.

        What’s mechanically implemented today: /commit (verifier + quality_score), /qa-quarto (critic-fixer loop), /review-paper --adversarial (critic-fixer loop), /slide-excellence (multi-agent fanout), /review-paper --peer (editor + 2 referees + cross-artifact). See Pattern 2 for the complete workflow.

        3.4 Quality Review: The 80/90/95 System

        -

        The quality-gates rule (quality-gates.md) defines scoring thresholds that /commit and review skills apply. Thresholds are advisory — enforced inside specific skills, not by a repo-wide git pre-commit hook. A direct git commit bypasses the review. Every file gets a quality score from 0 to 100:

        +

        The quality-gates rule (quality-gates.md) defines scoring thresholds that /commit and review skills apply. Thresholds are advisory — enforced inside specific skills and, once you run ./scripts/install-hooks.sh, by a real git pre-commit hook (.githooks/pre-commit) that runs the surface-sync + quality (≥80) gates on every commit. Bypass it sparingly with SKIP_QUALITY_GATE=1 or --no-verify. Every file gets a quality score from 0 to 100:

        @@ -3459,8 +3459,9 @@

        @@ -3596,7 +3597,7 @@

        -

        Claude Code ships with built-in skills beyond this template’s 38: /batch orchestrates parallel refactoring across your codebase (using git worktrees for isolation), /simplify runs 3-agent code review and applies fixes, and /debug helps troubleshoot sessions. These complement the academic skills above.

        +

        Claude Code ships with built-in skills beyond this template’s 51: /batch orchestrates parallel refactoring across your codebase (using git worktrees for isolation), /simplify runs 3-agent code review and applies fixes, and /debug helps troubleshoot sessions. These complement the academic skills above.

        @@ -3671,7 +3672,7 @@

        TROUBLESHOOTING.md for the migration checklist.

        -

        Not all agents need the same model. Each agent file has a model: field in its YAML frontmatter. By default, all agents use model: inherit (they use whatever model your main session runs). But you can customize this to optimize cost:

        +

        Not all agents need the same model. Each agent file has a model: field in its YAML frontmatter. In v2.0 all 18 agents are pinned to an explicit model + effort tier (no longer inherit), following the 70/20/10 routing pattern below. You can re-tune any agent to optimize cost:

        @@ -3701,18 +3702,18 @@

        -

        - - + + +
        Defaultmodel: inheritUses whatever the main session runsMechanical defaultmodel: haikuBounded, fast work proofreader, slide-auditor
        -

        The principle: Use Opus for tasks that require holding two large documents in mind simultaneously (translation, adversarial comparison). Use Sonnet for tasks with clear, bounded scope (fix these 12 issues, check this R script). Let everything else inherit.

        +

        The principle: Use Opus for tasks that require holding two large documents in mind simultaneously (translation, adversarial comparison). Use Sonnet for tasks with clear, bounded scope (fix these 12 issues, check this R script). Use Haiku for mechanical work. In v2.0 every agent is pinned explicitly rather than inheriting — see agent-fleet.md for the full fleet + tiers.

        To change an agent’s model, edit its YAML frontmatter:

        ---
 name: quarto-critic
-model: opus   # was: inherit
+model: opus   # pinned per agent-fleet.md (v2.0: no agent inherits)
 ---
        @@ -4261,10 +4262,10 @@

        Workflow Patterns for the full protocol.

        -
        -

        4.8.2 Session Logs — Why-Not-Just-What History (with Automated Reminders)

        +
        +

        4.8.2 Session Logs — Why-Not-Just-What History (auto-written)

        Git commits record what changed, but not why. Session logs fill this gap. Claude writes to quality_reports/session_logs/ at three points: right after plan approval, incrementally during implementation (as decisions happen), and at session end. This means the log captures reasoning as it happens, before auto-compression can discard it.

        -

        Because relying on instructions alone is fragile (Claude forgets during long sessions), a Stop hook (.claude/hooks/log-reminder.py) fires after every response. It tracks how many responses have passed since the session log was last updated. After a threshold, it blocks Claude from stopping until the log is current. This turns a best practice into an enforced behavior.

        +

        Because relying on instructions alone is fragile (Claude forgets during long sessions), a Stop hook (.claude/hooks/log-reminder.py) fires after every response. In v2.0 it no longer merely nags — it auto-writes the session log for every meaningful change-set, so the why-history is captured without depending on Claude remembering to do it.

        New sessions can read these logs to understand not just the current state of the project, but the reasoning behind it. See Pattern 1 in Workflow Patterns for the full protocol.

        @@ -4330,7 +4331,7 @@

        4.8.4 Hooks — Automated Enforcement

        The session log reminder above is one example of a broader pattern: using hooks to enforce rules that Claude might otherwise forget during long sessions. Rules live in context and can be compressed away. Hooks live in .claude/settings.json and fire every time, regardless of context state.

        -

        The template includes hooks for logging, notifications, and context survival:

        +

        The template includes 7 hooks for logging, notifications, context survival, and git safety (plus a real git pre-commit hook, .githooks/pre-commit, installed via ./scripts/install-hooks.sh):

        @@ -4371,9 +4372,14 @@

        -

        - - + + + + + + + +
        Verification reminderPostToolUse[Write|Edit]Reminds to compile/render before marking doneGit guardrailsPreToolUse[Bash]Blocks destructive git (reset --hard, clean -f, push --force, add -A)
        Claim reconcilePostToolUse[Edit|Write]Flags stale numeric claims when analysis scripts change
        @@ -4721,9 +4727,10 @@

        5.2.3 Agent Selection

        @@ -4774,7 +4781,7 @@

        “Translate Lecture 5 to Quarto. Just do it.”

        -

        In this mode, the orchestrator still runs the full verify-review-fix loop (quality is non-negotiable), but skips the final approval pause and auto-commits if the score is 80 or above. It still presents the summary so you can see what was done.

        +

        In this mode, the orchestrator still runs the full fan-out → reduce → judge → loop-until-dry runtime (quality is non-negotiable) and skips the final approval pause for the current skill. It does not auto-commit: “just do it” is never commit authorization — commits require an explicit /commit or unambiguous request. It still presents the summary so you can see what was done.

        5.2.5 Relationship to Existing Skills

        @@ -4784,7 +4791,7 @@

        7.4 Step 4: Creating Custom Skills

        -

        The guide includes 24 skills for common academic tasks. But if you have repetitive workflows specific to your domain, you can create your own.

        +

        The guide includes 51 skills for common academic tasks. But if you have repetitive workflows specific to your domain, you can create your own.

        7.4.1 When to Create a Skill

        Create a skill when: - You repeatedly explain the same 3+ step workflow to Claude - You need domain-specific quality checks (citation style, notation consistency, lab protocols) - You enforce field-specific output formats (thesis structure, journal templates) - You coordinate multi-tool workflows (data → analysis → manuscript)

        @@ -6531,7 +6538,7 @@

        -

        The /deep-audit skill was itself extracted from a repeating workflow using /learn. After running 7 rounds of manual consistency audits — each time launching 4 parallel agents to check guide accuracy, hook code quality, skills/rules consistency, and cross-document counts — the pattern was codified into a skill. Now /deep-audit launches those same 4 agents, triages findings, applies fixes, and loops until clean (max 5 rounds). It also encodes a table of known bug patterns from past audits so future rounds catch regressions faster.

        +

        The /deep-audit skill was itself extracted from a repeating workflow using /learn. After running 7 rounds of manual consistency audits — each time launching 4 parallel agents to check guide accuracy, hook code quality, skills/rules consistency, and cross-document counts — the pattern was codified into a skill. Now /deep-audit launches those same 4 agents, triages findings, applies fixes, and loops until clean (loop-until-dry: converges after 2 dry rounds, with a fallback cap). It also encodes a table of known bug patterns from past audits so future rounds catch regressions faster.

        This is the /learn lifecycle in action: discover a repeating workflow → extract it → never repeat the manual steps again.

        @@ -6547,7 +6554,7 @@

        @@ -6856,16 +6863,6 @@

        -/prompt -.claude/skills/prompt/ -Reformat informal input into a structured six-section prompt, then execute (v1.9.0; ported from Blattman v2.1 with stripping) - - -/prompt-only -.claude/skills/prompt-only/ -Same formatting as /prompt but emits the prompt as a reusable artifact (no execution) (v1.9.0) - - /compress-session .claude/skills/compress-session/ Distil the current session into a structured note (decisions, files, open questions, next actions, discarded-as-noise) before auto-compaction (v1.9.0) @@ -6890,6 +6887,76 @@

        +/replication-package +.claude/skills/replication-package/ +Assemble a submission-ready DCAS / openICPSR replication package; calls /audit-reproducibility and blocks on FAIL (v2.0) + + +/capture-environment +.claude/skills/capture-environment/ +Snapshot the computational environment (renv.lock / sessionInfo / requirements.txt / uv.lock + optional Dockerfile + seeds) (v2.0) + + +/did-event-study +.claude/skills/did-event-study/ +Staggered DiD / event-study wrapper over canonical packages (Callaway–Sant’Anna did, fixest::sunab, HonestDiD; Stata csdid/eventstudyinteract/honestdid) (v2.0) + + +/power-analysis +.claude/skills/power-analysis/ +Power / required-N / MDE for study design (RCT, clustered/ICC, multi-arm, or simulation-based) + a methods paragraph for /preregister (v2.0) + + +/disclosure-check +.claude/skills/disclosure-check/ +Pre-screen restricted-data outputs for SDL violations (small cells, dominance, PII); gates on any CRITICAL (v2.0) + + +/grant-proposal +.claude/skills/grant-proposal/ +Scaffold a grant proposal (NSF/NIH/ERC/foundation) from an /interview-me spec; delegates DMP + facilities; aims↔︎methods↔︎budget coherence pass (v2.0) + + +/data-management-plan +.claude/skills/data-management-plan/ +Draft a funder-compliant Data Management Plan (NSF/NIH 2023/ERC/Horizon) (v2.0) + + +/coauthor-brief +.claude/skills/coauthor-brief/ +Generate a collaborator handoff brief: git delta, per-artifact state, open questions, reproduce-locally + restricted-data steps (v2.0) + + +/triage-inbox +.claude/skills/triage-inbox/ +Triage academic email + calendar (Gmail/Calendar MCP) into a prioritized digest + referee-obligations tracker; human-gated actions only (v2.0) + + +/syllabus +.claude/skills/syllabus/ +Build a course syllabus from a topic/reading list (schedule, objectives, assessment + rubric, per-week /create-lecture work-list) (v2.0) + + +/teach-from-paper +.claude/skills/teach-from-paper/ +Turn a paper into a lecture outline, teachable results, slide skeleton (→ /create-lecture), discussion questions, exercise brief (v2.0) + + +/respond-to-eval +.claude/skills/respond-to-eval/ +Turn student course evaluations into a classified teaching-improvement plan (Keep/Change/Investigate/Out-of-scope) (v2.0) + + +/scaffold-exercises +.claude/skills/scaffold-exercises/ +Scaffold a graded problem set (analytical/empirical/coding) with worked solutions; clean student set + separate solution key (v2.0) + + +/new-skill +.claude/skills/new-skill/ +Scaffold a convention-compliant skill (interview → write SKILL.md with gate-passing frontmatter) (v2.0) + @@ -6918,7 +6985,7 @@

        Orchestrator Protocol orchestrator-protocol.md -Contractor mode loop +Review runtime: fan-out → reduce → judge (+ hallucination gate) → loop-until-dry (goal-first / gate-enforced; augments contractor mode) Session Logging @@ -6930,6 +6997,11 @@

        +Prompt Shaping +prompt-shaping.md +Ambient prompt-shaping habit (replaces the retired /prompt and /prompt-only skills) +

        Path-scoped (load only when working on matching files):

        @@ -7067,6 +7139,11 @@

        +Confidential Data +confidential-data.md +restricted/confidential data paths — no raw confidential data in git, disclosure-avoidance discipline, used by /disclosure-check / /replication-package / /data-management-plan (v2.0) + @@ -7087,9 +7164,9 @@

        -Session log reminder +Session log auto-writer Stop (command) -.claude/hooks/log-reminder.py +.claude/hooks/log-reminder.py — auto-writes the session log on every meaningful change-set (no longer merely a reminder) Desktop notification @@ -7099,7 +7176,7 @@

        Context state capture PreCompact (command) -.claude/hooks/pre-compact.py +.claude/hooks/pre-compact.py — DRAFT-block default is ON Context restoration @@ -7112,9 +7189,14 @@

        -Verification reminder -PostToolUse[Write|Edit] (command) -.claude/hooks/verify-reminder.py +Git guardrails +PreToolUse (command) +.claude/hooks/git-guardrails.py — blocks reset --hard / clean -f / push --force / add -A + + +Claim reconcile +PostToolUse (command) +.claude/hooks/claim-reconcile.py — flags stale numeric claims when scripts change @@ -7205,7 +7287,7 @@

        8.5.3 Hooks Not Firing

        -

        Symptom: No context warnings, no verification reminders.

        +

        Symptom: No context warnings, no auto-written session log, no git guardrails firing.

        Fix: 1. Check hooks are configured: cat .claude/settings.json | grep hooks 2. Ensure Python 3 is available: which python3 3. Check hook file permissions: ls -la .claude/hooks/

        diff --git a/guide/workflow-guide.html b/guide/workflow-guide.html index 568371f97..630538711 100644 --- a/guide/workflow-guide.html +++ b/guide/workflow-guide.html @@ -2380,7 +2380,7 @@

        Table of contents

        • 3.1 Why Specialized Agents Beat One-Size-Fits-All
        • 3.2 The Adversarial Pattern: Critic + Fixer
          • -
        • 3.3 The Orchestrator: A Pattern, Not a Daemon
          • +
        • 3.3 The Orchestrator: A Runtime, Not a Daemon
        • 3.4 Quality Review: The 80/90/95 System
          • 3.4.1 How Scores Are Calculated
            • @@ -2422,7 +2422,7 @@

            Table of contents

          • 4.8 Memory — Cross-Session Persistence
            • 4.8.1 Plans — Compression-Resistant Task Memory
              • -
            • 4.8.2 Session Logs — Why-Not-Just-What History (with Automated Reminders)
              • +
            • 4.8.2 Session Logs — Why-Not-Just-What History (auto-written)
            • 4.8.3 How It All Fits Together
            • 4.8.4 Hooks — Automated Enforcement
            • 4.8.5 Context Survival System (Advanced)
              • @@ -2558,7 +2558,7 @@

              My Claude Code Setup

              Modified
              -

              May 31, 2026

              +

              June 9, 2026

              @@ -2614,8 +2614,8 @@

              -Orchestrator pattern -Specific skills (/qa-quarto, /review-paper --adversarial, /create-lecture, …) run a 6-step verify-review-fix-score loop internally +Orchestrator runtime +Specific skills (/qa-quarto, /review-paper --adversarial, /create-lecture, …) run a real fan-out → reduce → judge → loop-until-dry runtime internally Multi-agent workflows @@ -2757,7 +2757,7 @@

              -

              You talk, Claude orchestrates. The 18 agents, 38 skills, and 28 rules exist so you don’t have to think about them. Describe your goal, approve the plan, and let the system work.

              +

              You talk, Claude orchestrates. The 18 agents, 51 skills, and 32 rules exist so you don’t have to think about them. Describe your goal, approve the plan, and let the system work.

              @@ -2770,7 +2770,7 @@

              -

              This guide describes the full system — 18 agents, 38 skills, 28 rules. That is the ceiling, not the floor. Start with just CLAUDE.md and 2–3 skills (/compile-latex, /proofread, /commit). Add rules and agents as you discover what you need. The template is designed for progressive adoption: fork it, fill in the placeholders, and start working. Everything else is there when you’re ready.

              +

              This guide describes the full system — 18 agents, 51 skills, 32 rules. That is the ceiling, not the floor. Start with just CLAUDE.md and 2–3 skills (/compile-latex, /proofread, /commit). Add rules and agents as you discover what you need. The template is designed for progressive adoption: fork it, fill in the placeholders, and start working. Everything else is there when you’re ready.

              @@ -3124,7 +3124,7 @@

              @@ -3154,15 +3154,15 @@

              -

              3.3 The Orchestrator: A Pattern, Not a Daemon

              +
              +

              3.3 The Orchestrator: A Runtime, Not a Daemon

              Individual agents are specialists. Skills like /slide-excellence and /qa-quarto coordinate a few agents for specific tasks. In day-to-day work, you don’t have to think about which agents to run — the right skill runs the right agents for you.

              -

              The orchestrator protocol (.claude/rules/orchestrator-protocol.md) defines the 6-step loop (IMPLEMENT → VERIFY → REVIEW → FIX → RE-VERIFY → SCORE) that skills implement internally. It is not a runtime daemon: plan approval does NOT auto-trigger this loop. You invoke a skill (e.g., /create-lecture, /qa-quarto, /review-paper --adversarial) and the skill runs the pattern within its own scope.

              +

              The orchestrator protocol (.claude/rules/orchestrator-protocol.md) is a real runtime, built on the Task subagent primitive every Claude Code session has: skills fan out to forked reviewers, reduce their typed findings through a deterministic gate, judge with a post-judge hallucination guard, and loop until dry. What is not automatic is the trigger: nothing launches this loop on its own, and there is no background daemon — plan approval does NOT auto-trigger it. You invoke a skill (e.g., /create-lecture, /qa-quarto, /review-paper --adversarial) and the skill runs the runtime within its own scope. The human is the auditor of the disagreements the loop surfaces; this is explicitly not an autonomous daemon.

              What’s mechanically implemented today: /commit (verifier + quality_score), /qa-quarto (critic-fixer loop), /review-paper --adversarial (critic-fixer loop), /slide-excellence (multi-agent fanout), /review-paper --peer (editor + 2 referees + cross-artifact). See Pattern 2 for the complete workflow.

              3.4 Quality Review: The 80/90/95 System

              -

              The quality-gates rule (quality-gates.md) defines scoring thresholds that /commit and review skills apply. Thresholds are advisory — enforced inside specific skills, not by a repo-wide git pre-commit hook. A direct git commit bypasses the review. Every file gets a quality score from 0 to 100:

              +

              The quality-gates rule (quality-gates.md) defines scoring thresholds that /commit and review skills apply. Thresholds are advisory — enforced inside specific skills and, once you run ./scripts/install-hooks.sh, by a real git pre-commit hook (.githooks/pre-commit) that runs the surface-sync + quality (≥80) gates on every commit. Bypass it sparingly with SKIP_QUALITY_GATE=1 or --no-verify. Every file gets a quality score from 0 to 100:

              @@ -3459,8 +3459,9 @@

              @@ -3596,7 +3597,7 @@

              -

              Claude Code ships with built-in skills beyond this template’s 38: /batch orchestrates parallel refactoring across your codebase (using git worktrees for isolation), /simplify runs 3-agent code review and applies fixes, and /debug helps troubleshoot sessions. These complement the academic skills above.

              +

              Claude Code ships with built-in skills beyond this template’s 51: /batch orchestrates parallel refactoring across your codebase (using git worktrees for isolation), /simplify runs 3-agent code review and applies fixes, and /debug helps troubleshoot sessions. These complement the academic skills above.

              @@ -3671,7 +3672,7 @@

              TROUBLESHOOTING.md for the migration checklist.

              -

              Not all agents need the same model. Each agent file has a model: field in its YAML frontmatter. By default, all agents use model: inherit (they use whatever model your main session runs). But you can customize this to optimize cost:

              +

              Not all agents need the same model. Each agent file has a model: field in its YAML frontmatter. In v2.0 all 18 agents are pinned to an explicit model + effort tier (no longer inherit), following the 70/20/10 routing pattern below. You can re-tune any agent to optimize cost:

              @@ -3701,18 +3702,18 @@

              -

              - - + + +
              Defaultmodel: inheritUses whatever the main session runsMechanical defaultmodel: haikuBounded, fast work proofreader, slide-auditor
              -

              The principle: Use Opus for tasks that require holding two large documents in mind simultaneously (translation, adversarial comparison). Use Sonnet for tasks with clear, bounded scope (fix these 12 issues, check this R script). Let everything else inherit.

              +

              The principle: Use Opus for tasks that require holding two large documents in mind simultaneously (translation, adversarial comparison). Use Sonnet for tasks with clear, bounded scope (fix these 12 issues, check this R script). Use Haiku for mechanical work. In v2.0 every agent is pinned explicitly rather than inheriting — see agent-fleet.md for the full fleet + tiers.

              To change an agent’s model, edit its YAML frontmatter:

              ---
 name: quarto-critic
-model: opus   # was: inherit
+model: opus   # pinned per agent-fleet.md (v2.0: no agent inherits)
 ---
              @@ -4261,10 +4262,10 @@

              Workflow Patterns for the full protocol.

              -
              -

              4.8.2 Session Logs — Why-Not-Just-What History (with Automated Reminders)

              +
              +

              4.8.2 Session Logs — Why-Not-Just-What History (auto-written)

              Git commits record what changed, but not why. Session logs fill this gap. Claude writes to quality_reports/session_logs/ at three points: right after plan approval, incrementally during implementation (as decisions happen), and at session end. This means the log captures reasoning as it happens, before auto-compression can discard it.

              -

              Because relying on instructions alone is fragile (Claude forgets during long sessions), a Stop hook (.claude/hooks/log-reminder.py) fires after every response. It tracks how many responses have passed since the session log was last updated. After a threshold, it blocks Claude from stopping until the log is current. This turns a best practice into an enforced behavior.

              +

              Because relying on instructions alone is fragile (Claude forgets during long sessions), a Stop hook (.claude/hooks/log-reminder.py) fires after every response. In v2.0 it no longer merely nags — it auto-writes the session log for every meaningful change-set, so the why-history is captured without depending on Claude remembering to do it.

              New sessions can read these logs to understand not just the current state of the project, but the reasoning behind it. See Pattern 1 in Workflow Patterns for the full protocol.

              @@ -4330,7 +4331,7 @@

              4.8.4 Hooks — Automated Enforcement

              The session log reminder above is one example of a broader pattern: using hooks to enforce rules that Claude might otherwise forget during long sessions. Rules live in context and can be compressed away. Hooks live in .claude/settings.json and fire every time, regardless of context state.

              -

              The template includes hooks for logging, notifications, and context survival:

              +

              The template includes 7 hooks for logging, notifications, context survival, and git safety (plus a real git pre-commit hook, .githooks/pre-commit, installed via ./scripts/install-hooks.sh):

              @@ -4371,9 +4372,14 @@

              -

              - - + + + + + + + +
              Verification reminderPostToolUse[Write|Edit]Reminds to compile/render before marking doneGit guardrailsPreToolUse[Bash]Blocks destructive git (reset --hard, clean -f, push --force, add -A)
              Claim reconcilePostToolUse[Edit|Write]Flags stale numeric claims when analysis scripts change
              @@ -4721,9 +4727,10 @@

              5.2.3 Agent Selection

              @@ -4774,7 +4781,7 @@

              “Translate Lecture 5 to Quarto. Just do it.”

              -

              In this mode, the orchestrator still runs the full verify-review-fix loop (quality is non-negotiable), but skips the final approval pause and auto-commits if the score is 80 or above. It still presents the summary so you can see what was done.

              +

              In this mode, the orchestrator still runs the full fan-out → reduce → judge → loop-until-dry runtime (quality is non-negotiable) and skips the final approval pause for the current skill. It does not auto-commit: “just do it” is never commit authorization — commits require an explicit /commit or unambiguous request. It still presents the summary so you can see what was done.

              5.2.5 Relationship to Existing Skills

              @@ -4784,7 +4791,7 @@

              7.4 Step 4: Creating Custom Skills

              -

              The guide includes 24 skills for common academic tasks. But if you have repetitive workflows specific to your domain, you can create your own.

              +

              The guide includes 51 skills for common academic tasks. But if you have repetitive workflows specific to your domain, you can create your own.

              7.4.1 When to Create a Skill

              Create a skill when: - You repeatedly explain the same 3+ step workflow to Claude - You need domain-specific quality checks (citation style, notation consistency, lab protocols) - You enforce field-specific output formats (thesis structure, journal templates) - You coordinate multi-tool workflows (data → analysis → manuscript)

              @@ -6531,7 +6538,7 @@

              -

              The /deep-audit skill was itself extracted from a repeating workflow using /learn. After running 7 rounds of manual consistency audits — each time launching 4 parallel agents to check guide accuracy, hook code quality, skills/rules consistency, and cross-document counts — the pattern was codified into a skill. Now /deep-audit launches those same 4 agents, triages findings, applies fixes, and loops until clean (max 5 rounds). It also encodes a table of known bug patterns from past audits so future rounds catch regressions faster.

              +

              The /deep-audit skill was itself extracted from a repeating workflow using /learn. After running 7 rounds of manual consistency audits — each time launching 4 parallel agents to check guide accuracy, hook code quality, skills/rules consistency, and cross-document counts — the pattern was codified into a skill. Now /deep-audit launches those same 4 agents, triages findings, applies fixes, and loops until clean (loop-until-dry: converges after 2 dry rounds, with a fallback cap). It also encodes a table of known bug patterns from past audits so future rounds catch regressions faster.

              This is the /learn lifecycle in action: discover a repeating workflow → extract it → never repeat the manual steps again.

              @@ -6547,7 +6554,7 @@

              @@ -6856,16 +6863,6 @@

              -/prompt -.claude/skills/prompt/ -Reformat informal input into a structured six-section prompt, then execute (v1.9.0; ported from Blattman v2.1 with stripping) - - -/prompt-only -.claude/skills/prompt-only/ -Same formatting as /prompt but emits the prompt as a reusable artifact (no execution) (v1.9.0) - - /compress-session .claude/skills/compress-session/ Distil the current session into a structured note (decisions, files, open questions, next actions, discarded-as-noise) before auto-compaction (v1.9.0) @@ -6890,6 +6887,76 @@

              +/replication-package +.claude/skills/replication-package/ +Assemble a submission-ready DCAS / openICPSR replication package; calls /audit-reproducibility and blocks on FAIL (v2.0) + + +/capture-environment +.claude/skills/capture-environment/ +Snapshot the computational environment (renv.lock / sessionInfo / requirements.txt / uv.lock + optional Dockerfile + seeds) (v2.0) + + +/did-event-study +.claude/skills/did-event-study/ +Staggered DiD / event-study wrapper over canonical packages (Callaway–Sant’Anna did, fixest::sunab, HonestDiD; Stata csdid/eventstudyinteract/honestdid) (v2.0) + + +/power-analysis +.claude/skills/power-analysis/ +Power / required-N / MDE for study design (RCT, clustered/ICC, multi-arm, or simulation-based) + a methods paragraph for /preregister (v2.0) + + +/disclosure-check +.claude/skills/disclosure-check/ +Pre-screen restricted-data outputs for SDL violations (small cells, dominance, PII); gates on any CRITICAL (v2.0) + + +/grant-proposal +.claude/skills/grant-proposal/ +Scaffold a grant proposal (NSF/NIH/ERC/foundation) from an /interview-me spec; delegates DMP + facilities; aims↔︎methods↔︎budget coherence pass (v2.0) + + +/data-management-plan +.claude/skills/data-management-plan/ +Draft a funder-compliant Data Management Plan (NSF/NIH 2023/ERC/Horizon) (v2.0) + + +/coauthor-brief +.claude/skills/coauthor-brief/ +Generate a collaborator handoff brief: git delta, per-artifact state, open questions, reproduce-locally + restricted-data steps (v2.0) + + +/triage-inbox +.claude/skills/triage-inbox/ +Triage academic email + calendar (Gmail/Calendar MCP) into a prioritized digest + referee-obligations tracker; human-gated actions only (v2.0) + + +/syllabus +.claude/skills/syllabus/ +Build a course syllabus from a topic/reading list (schedule, objectives, assessment + rubric, per-week /create-lecture work-list) (v2.0) + + +/teach-from-paper +.claude/skills/teach-from-paper/ +Turn a paper into a lecture outline, teachable results, slide skeleton (→ /create-lecture), discussion questions, exercise brief (v2.0) + + +/respond-to-eval +.claude/skills/respond-to-eval/ +Turn student course evaluations into a classified teaching-improvement plan (Keep/Change/Investigate/Out-of-scope) (v2.0) + + +/scaffold-exercises +.claude/skills/scaffold-exercises/ +Scaffold a graded problem set (analytical/empirical/coding) with worked solutions; clean student set + separate solution key (v2.0) + + +/new-skill +.claude/skills/new-skill/ +Scaffold a convention-compliant skill (interview → write SKILL.md with gate-passing frontmatter) (v2.0) +

        @@ -6918,7 +6985,7 @@

        Orchestrator Protocol orchestrator-protocol.md -Contractor mode loop +Review runtime: fan-out → reduce → judge (+ hallucination gate) → loop-until-dry (goal-first / gate-enforced; augments contractor mode) Session Logging @@ -6930,6 +6997,11 @@

        +Prompt Shaping +prompt-shaping.md +Ambient prompt-shaping habit (replaces the retired /prompt and /prompt-only skills) +

        Path-scoped (load only when working on matching files):

        @@ -7067,6 +7139,11 @@

        +Confidential Data +confidential-data.md +restricted/confidential data paths — no raw confidential data in git, disclosure-avoidance discipline, used by /disclosure-check / /replication-package / /data-management-plan (v2.0) + @@ -7087,9 +7164,9 @@

        -Session log reminder +Session log auto-writer Stop (command) -.claude/hooks/log-reminder.py +.claude/hooks/log-reminder.py — auto-writes the session log on every meaningful change-set (no longer merely a reminder) Desktop notification @@ -7099,7 +7176,7 @@

        Context state capture PreCompact (command) -.claude/hooks/pre-compact.py +.claude/hooks/pre-compact.py — DRAFT-block default is ON Context restoration @@ -7112,9 +7189,14 @@

        -Verification reminder -PostToolUse[Write|Edit] (command) -.claude/hooks/verify-reminder.py +Git guardrails +PreToolUse (command) +.claude/hooks/git-guardrails.py — blocks reset --hard / clean -f / push --force / add -A + + +Claim reconcile +PostToolUse (command) +.claude/hooks/claim-reconcile.py — flags stale numeric claims when scripts change @@ -7205,7 +7287,7 @@

        8.5.3 Hooks Not Firing

        -

        Symptom: No context warnings, no verification reminders.

        +

        Symptom: No context warnings, no auto-written session log, no git guardrails firing.

        Fix: 1. Check hooks are configured: cat .claude/settings.json | grep hooks 2. Ensure Python 3 is available: which python3 3. Check hook file permissions: ls -la .claude/hooks/

        diff --git a/guide/workflow-guide.qmd b/guide/workflow-guide.qmd index 4d187ec13..1f3b56894 100644 --- a/guide/workflow-guide.qmd +++ b/guide/workflow-guide.qmd @@ -53,7 +53,7 @@ Claude Code runs on your computer with full access to your file system, terminal | Run shell commands | Compile LaTeX, run R scripts, render Quarto --- directly | | Access git history | Commits, PRs, branches --- all from the conversation | | Persistent memory | CLAUDE.md + MEMORY.md survive across sessions | -| Orchestrator pattern | Specific skills (`/qa-quarto`, `/review-paper --adversarial`, `/create-lecture`, …) run a 6-step verify-review-fix-score loop internally | +| Orchestrator runtime | Specific skills (`/qa-quarto`, `/review-paper --adversarial`, `/create-lecture`, …) run a real fan-out → reduce → judge → loop-until-dry runtime internally | | Multi-agent workflows | 18 specialized agents for proofreading, layout, pedagogy, code review, fact-checking, AI-voice auditing, simulation + R-package review | | Quality review | Advisory scoring inside `/commit` (halt + ask to override); targets: 80 commit, 90 PR, 95 excellence | | CLI/headless mode | Run from scripts: `claude -p "compile all lectures"` | @@ -155,13 +155,13 @@ Most of the time, you just describe what you want and Claude handles the rest. E ::: {.callout-important} ## The Bottom Line -**You talk, Claude orchestrates.** The 18 agents, 38 skills, and 28 rules exist so you don't have to think about them. Describe your goal, approve the plan, and let the system work. +**You talk, Claude orchestrates.** The 18 agents, 51 skills, and 32 rules exist so you don't have to think about them. Describe your goal, approve the plan, and let the system work. ::: ::: {.callout-note} ## You Don't Need All of This on Day One -This guide describes the full system --- 18 agents, 38 skills, 28 rules. That is the ceiling, not the floor. **Start with just CLAUDE.md and 2--3 skills** (`/compile-latex`, `/proofread`, `/commit`). Add rules and agents as you discover what you need. The template is designed for progressive adoption: fork it, fill in the placeholders, and start working. Everything else is there when you're ready. +This guide describes the full system --- 18 agents, 51 skills, 32 rules. That is the ceiling, not the floor. **Start with just CLAUDE.md and 2--3 skills** (`/compile-latex`, `/proofread`, `/commit`). Add rules and agents as you discover what you need. The template is designed for progressive adoption: fork it, fill in the placeholders, and start working. Everything else is there when you're ready. ::: --- @@ -412,7 +412,7 @@ APPROVED NEEDS WORK | (Round 2) | +----+----------+ | - ... (up to 5 rounds) + ... (loop-until-dry: converge after 2 dry rounds; 5-round cap is a fallback) ``` **Why it works:** The critic can't fix files (read-only), so it has no incentive to downplay issues. The fixer can't approve itself (the critic re-audits). This prevents the common failure of Claude saying "looks good" about its own work. @@ -431,17 +431,17 @@ The critic-fixer loop above is text-only — it compares LaTeX source to Quarto Optional, not required: `/qa-quarto` and `/visual-audit` already catch most issues without it. Treat Computer Use as an extra rung when text-level audits aren't enough (e.g., suspecting a TikZ render glitch that the source code looks fine for). ::: -## The Orchestrator: A Pattern, Not a Daemon +## The Orchestrator: A Runtime, Not a Daemon Individual agents are specialists. Skills like `/slide-excellence` and `/qa-quarto` coordinate a few agents for specific tasks. In day-to-day work, you don't have to think about which agents to run — the right skill runs the right agents for you. -The **orchestrator protocol** (`.claude/rules/orchestrator-protocol.md`) defines the 6-step loop (IMPLEMENT → VERIFY → REVIEW → FIX → RE-VERIFY → SCORE) that skills implement internally. It is **not** a runtime daemon: plan approval does NOT auto-trigger this loop. You invoke a skill (e.g., `/create-lecture`, `/qa-quarto`, `/review-paper --adversarial`) and the skill runs the pattern within its own scope. +The **orchestrator protocol** (`.claude/rules/orchestrator-protocol.md`) is a **real runtime**, built on the `Task` subagent primitive every Claude Code session has: skills fan out to forked reviewers, reduce their typed findings through a deterministic gate, judge with a post-judge hallucination guard, and loop until dry. What is **not** automatic is the *trigger*: nothing launches this loop on its own, and there is no background daemon — plan approval does NOT auto-trigger it. You invoke a skill (e.g., `/create-lecture`, `/qa-quarto`, `/review-paper --adversarial`) and the skill runs the runtime within its own scope. The human is the auditor of the disagreements the loop surfaces; this is explicitly not an autonomous daemon. What's mechanically implemented today: `/commit` (verifier + quality_score), `/qa-quarto` (critic-fixer loop), `/review-paper --adversarial` (critic-fixer loop), `/slide-excellence` (multi-agent fanout), `/review-paper --peer` (editor + 2 referees + cross-artifact). See [Pattern 2](#pattern-2-contractor-mode-orchestrator) for the complete workflow. ## Quality Review: The 80/90/95 System {#sec-quality} -The [quality-gates rule](#sec-blocks) (`quality-gates.md`) defines scoring thresholds that `/commit` and review skills apply. Thresholds are **advisory** — enforced inside specific skills, not by a repo-wide git pre-commit hook. A direct `git commit` bypasses the review. Every file gets a quality score from 0 to 100: +The [quality-gates rule](#sec-blocks) (`quality-gates.md`) defines scoring thresholds that `/commit` and review skills apply. Thresholds are **advisory** — enforced inside specific skills **and**, once you run `./scripts/install-hooks.sh`, by a real git pre-commit hook (`.githooks/pre-commit`) that runs the surface-sync + quality (≥80) gates on every commit. Bypass it sparingly with `SKIP_QUALITY_GATE=1` or `--no-verify`. Every file gets a quality score from 0 to 100: | Score | Threshold | Meaning | Action | |-------|-----------|---------|--------| @@ -586,8 +586,9 @@ Rules are markdown files in `.claude/rules/` that Claude loads automatically. Th ``` .claude/rules/ ├── plan-first-workflow.md # ~83 lines — plan before you build -├── orchestrator-protocol.md # ~42 lines — contractor mode loop +├── orchestrator-protocol.md # ~93 lines — review-fix runtime (fan-out → reduce → judge → loop-until-dry) ├── session-logging.md # ~23 lines — three logging triggers +├── prompt-shaping.md # ~32 lines — shape informal asks (ambient; replaces /prompt) └── meta-governance.md # ~251 lines — template vs working project ``` @@ -601,10 +602,10 @@ Rules are markdown files in `.claude/rules/` that Claude loads automatically. Th ├── replication-protocol.md # paths: ["scripts/**/*.R"] — replicate first ├── exploration-folder-protocol.md # paths: ["explorations/**"] — sandbox rules ├── orchestrator-research.md # paths: ["scripts/**/*.R", "explorations/**"] — simple loop -└── ...20 path-scoped rules total +└── ...27 path-scoped rules total ``` -The first three always-on rules total ~148 lines of actionable instructions. `meta-governance` is a reference document for the template's dual nature (working project vs. public template) and loads passively. Path-scoped rules add rich, domain-specific guidance exactly when Claude needs it. +The always-on rules are deliberately compact — together they fit well within the budget Claude reads reliably each session. `meta-governance` is a reference document for the template's dual nature (working project vs. public template) and loads passively. Path-scoped rules add rich, domain-specific guidance exactly when Claude needs it. **Sync vs. translate:** The `beamer-quarto-sync` rule handles incremental edits --- fix a typo in Beamer, same fix goes to Quarto. The `/translate-to-quarto` skill is for full initial translation of a new lecture. Translate once, sync thereafter. @@ -695,7 +696,7 @@ argument-hint: "[filename without .tex extension]" ::: {.callout-note} ## Built-In Skills -Claude Code ships with built-in skills beyond this template's 38: `/batch` orchestrates parallel refactoring across your codebase (using git worktrees for isolation), `/simplify` runs 3-agent code review and applies fixes, and `/debug` helps troubleshoot sessions. These complement the academic skills above. +Claude Code ships with built-in skills beyond this template's 51: `/batch` orchestrates parallel refactoring across your codebase (using git worktrees for isolation), `/simplify` runs 3-agent code review and applies fixes, and `/debug` helps troubleshoot sessions. These complement the academic skills above. ::: ## Agents --- Specialized Reviewers {#agents---specialized-reviewers} @@ -764,22 +765,22 @@ Claude Code also offers experimental **Agent Teams** --- multiple independent se **Retirement notice:** Sonnet 4 and the original Opus 4 retire on **2026-06-15**. If your environment pins one of these (`ANTHROPIC_MODEL` env, hard-coded model IDs in CI, etc.), migrate before then. See [TROUBLESHOOTING.md](../TROUBLESHOOTING.md) for the migration checklist. ::: -Not all agents need the same model. Each agent file has a `model:` field in its YAML frontmatter. By default, all agents use `model: inherit` (they use whatever model your main session runs). But you can customize this to optimize cost: +Not all agents need the same model. Each agent file has a `model:` field in its YAML frontmatter. In v2.0 all 18 agents are **pinned** to an explicit model + effort tier (no longer `inherit`), following the 70/20/10 routing pattern below. You can re-tune any agent to optimize cost: | Task Type | Recommended Model | Why | Examples | |-----------|-------------------|-----|----------| | Complex translation | `model: opus` | Needs deep understanding of both formats | beamer-translator, quarto-critic | | Fast, constrained work | `model: sonnet` | Speed matters more than depth | r-reviewer, quarto-fixer | -| Default | `model: inherit` | Uses whatever the main session runs | proofreader, slide-auditor | +| Mechanical default | `model: haiku` | Bounded, fast work | proofreader, slide-auditor | -**The principle:** Use Opus for tasks that require holding two large documents in mind simultaneously (translation, adversarial comparison). Use Sonnet for tasks with clear, bounded scope (fix these 12 issues, check this R script). Let everything else inherit. +**The principle:** Use Opus for tasks that require holding two large documents in mind simultaneously (translation, adversarial comparison). Use Sonnet for tasks with clear, bounded scope (fix these 12 issues, check this R script). Use Haiku for mechanical work. In v2.0 every agent is pinned explicitly rather than inheriting — see `agent-fleet.md` for the full fleet + tiers. To change an agent's model, edit its YAML frontmatter: ```yaml --- name: quarto-critic -model: opus # was: inherit +model: opus # pinned per agent-fleet.md (v2.0: no agent inherits) --- ``` @@ -1085,11 +1086,11 @@ While MEMORY.md stores long-lived project facts, **plans** store task-specific s See Pattern 1 in [Workflow Patterns](#sec-patterns) for the full protocol. -### Session Logs --- Why-Not-Just-What History (with Automated Reminders) +### Session Logs --- Why-Not-Just-What History (auto-written) Git commits record what changed, but not *why*. Session logs fill this gap. Claude writes to `quality_reports/session_logs/` at three points: right after plan approval, incrementally during implementation (as decisions happen), and at session end. This means the log captures reasoning *as it happens*, before auto-compression can discard it. -Because relying on instructions alone is fragile (Claude forgets during long sessions), a **Stop hook** (`.claude/hooks/log-reminder.py`) fires after every response. It tracks how many responses have passed since the session log was last updated. After a threshold, it blocks Claude from stopping until the log is current. This turns a best practice into an enforced behavior. +Because relying on instructions alone is fragile (Claude forgets during long sessions), a **Stop hook** (`.claude/hooks/log-reminder.py`) fires after every response. In v2.0 it no longer merely nags — it **auto-writes the session log** for every meaningful change-set, so the why-history is captured without depending on Claude remembering to do it. New sessions can read these logs to understand not just the current state of the project, but the reasoning behind it. See Pattern 1 in [Workflow Patterns](#sec-patterns) for the full protocol. @@ -1111,7 +1112,7 @@ The first four layers are your safety net. Anything written to disk survives ind The session log reminder above is one example of a broader pattern: using **hooks** to enforce rules that Claude might otherwise forget during long sessions. Rules live in context and can be compressed away. Hooks live in `.claude/settings.json` and fire every time, regardless of context state. -The template includes hooks for logging, notifications, and context survival: +The template includes 7 hooks for logging, notifications, context survival, and git safety (plus a real git pre-commit hook, `.githooks/pre-commit`, installed via `./scripts/install-hooks.sh`): | Hook | Event | What It Does | |------|-------|-------------| @@ -1120,7 +1121,8 @@ The template includes hooks for logging, notifications, and context survival: | Context state capture | `PreCompact` | Saves plan state before auto-compaction | | Context restoration | `SessionStart[compact|resume]` | Restores context after compaction or resume | | Context monitor | `PostToolUse[Bash|Task]` | Progressive warnings at 40%/55%/65%/80%/90% context | -| Verification reminder | `PostToolUse[Write|Edit]` | Reminds to compile/render before marking done | +| Git guardrails | `PreToolUse[Bash]` | Blocks destructive git (`reset --hard`, `clean -f`, `push --force`, `add -A`) | +| Claim reconcile | `PostToolUse[Edit|Write]` | Flags stale numeric claims when analysis scripts change | Verification and Beamer-Quarto sync are enforced via auto-loaded rules, which are the right tool for nuanced judgment. Hooks are reserved for enforcement that *must* survive context compression. @@ -1336,9 +1338,10 @@ User: "Translate Lecture 5 to Quarto" Step 6: SCORE | Apply quality-gates rubric | - +-- Score >= 80? + +-- Converged? (a round adds 0 new Critical/Major) YES -> Present summary to user - NO -> Loop to Step 3 (max 5 rounds) + NO -> Loop to Step 3 in fresh context + (loop-until-dry; 5-round cap is a fallback only) ``` ### Agent Selection @@ -1363,7 +1366,7 @@ Sometimes you do not want to approve the final result --- you just want it done: > "Translate Lecture 5 to Quarto. Just do it." -In this mode, the orchestrator still runs the full verify-review-fix loop (quality is non-negotiable), but skips the final approval pause and auto-commits if the score is 80 or above. It still presents the summary so you can see what was done. +In this mode, the orchestrator still runs the full fan-out → reduce → judge → loop-until-dry runtime (quality is non-negotiable) and skips the final approval pause for the current skill. It does **not** auto-commit: "just do it" is never commit authorization — commits require an explicit `/commit` or unambiguous request. It still presents the summary so you can see what was done. ### Relationship to Existing Skills @@ -1373,7 +1376,7 @@ The orchestrator does NOT replace skills. It coordinates them: - `/slide-excellence` remains available for comprehensive multi-agent review - `/create-lecture` remains available as a guided creation workflow -The difference: when you invoke a skill directly, it runs its specific workflow. When the orchestrator is active, it decides which agents to invoke based on context. The orchestrator is the default; skills are for targeted use. +The difference: when you invoke a skill directly, it runs its specific workflow. The orchestrator runtime lives *inside* each skill — there is no repo-wide orchestrator that activates on its own or chains skills together. A skill is always user- or skill-initiated; nothing fires the runtime unattended. ::: {.callout-tip collapse="true"} ## When to Use Skills vs. the Orchestrator @@ -1432,7 +1435,7 @@ Translation preserves all content while adapting format, converting TikZ to SVG +-- Critic: finds issues +-- Fixer: applies fixes +-- Critic: re-audits - +-- ... (until APPROVED or 5 rounds) + +-- ... (loop-until-dry: converge after 2 dry rounds; 5-round cap is a fallback) ``` ## Pattern 5: Replication-First Coding {#pattern-5-replication-first-coding} @@ -2385,7 +2388,7 @@ The template includes matching LaTeX and Quarto palettes. To customize: ## Step 4: Creating Custom Skills {#sec-create-skills} -The guide includes 24 [skills](#skills---reusable-slash-commands) for common academic tasks. But if you have repetitive workflows specific to your domain, you can create your own. +The guide includes 51 [skills](#skills---reusable-slash-commands) for common academic tasks. But if you have repetitive workflows specific to your domain, you can create your own. ### When to Create a Skill @@ -2594,7 +2597,7 @@ Cross-references in-text citations against bibliography entries. Identifies miss ::: {.callout-tip} ## Real Example: /deep-audit Was Created with /learn -The `/deep-audit` skill was itself extracted from a repeating workflow using `/learn`. After running 7 rounds of manual consistency audits --- each time launching 4 parallel agents to check guide accuracy, hook code quality, skills/rules consistency, and cross-document counts --- the pattern was codified into a skill. Now `/deep-audit` launches those same 4 agents, triages findings, applies fixes, and loops until clean (max 5 rounds). It also encodes a table of known bug patterns from past audits so future rounds catch regressions faster. +The `/deep-audit` skill was itself extracted from a repeating workflow using `/learn`. After running 7 rounds of manual consistency audits --- each time launching 4 parallel agents to check guide accuracy, hook code quality, skills/rules consistency, and cross-document counts --- the pattern was codified into a skill. Now `/deep-audit` launches those same 4 agents, triages findings, applies fixes, and loops until clean (loop-until-dry: converges after 2 dry rounds, with a fallback cap). It also encodes a table of known bug patterns from past audits so future rounds catch regressions faster. This is the `/learn` lifecycle in action: discover a repeating workflow → extract it → never repeat the manual steps again. ::: @@ -2608,7 +2611,7 @@ This is the `/learn` lifecycle in action: discover a repeating workflow → extr 5. **Verify everything.** The verification rule exists for a reason. Never skip compilation or rendering checks. 6. **Session logs matter.** Document design decisions, not just what changed. Future-you will thank present-you. 7. **Devil's Advocate early.** Challenge slide structure before you've built 50 slides on a shaky foundation. -8. **Progressive disclosure.** Start with CLAUDE.md + 2--3 rules. Add more as your workflow matures. Newcomers should not face 24 rules on day one. +8. **Progressive disclosure.** Start with CLAUDE.md + 2--3 rules. Add more as your workflow matures. Newcomers should not face all 32 rules on day one. 9. **Use `CLAUDE.local.md` for personal overrides.** This file is automatically gitignored and loaded alongside `CLAUDE.md`. Put machine-specific paths, personal preferences, and local tool versions here --- they won't pollute the shared repo. ::: {.callout-note collapse="true"} @@ -2683,13 +2686,25 @@ Claude Code supports **plugins** --- bundled collections of skills, agents, hook | `/checkpoint` | `.claude/skills/checkpoint/` | Structured session-handoff snapshot — companion to narrative session logs (v1.8.0) | | `/preregister` | `.claude/skills/preregister/` | Generate a preregistration document (OSF / AsPredicted / AEA RCT Registry) from a research spec (v1.8.0) | | `/humanize` | `.claude/skills/humanize/` | Detect AI-voice tells in academic prose (read-only audit; no rewrite) (v1.9.0) | -| `/prompt` | `.claude/skills/prompt/` | Reformat informal input into a structured six-section prompt, then execute (v1.9.0; ported from Blattman v2.1 with stripping) | -| `/prompt-only` | `.claude/skills/prompt-only/` | Same formatting as `/prompt` but emits the prompt as a reusable artifact (no execution) (v1.9.0) | | `/compress-session` | `.claude/skills/compress-session/` | Distil the current session into a structured note (decisions, files, open questions, next actions, discarded-as-noise) before auto-compaction (v1.9.0) | | `/promote-memory` | `.claude/skills/promote-memory/` | Five-critic council that votes on whether candidate `[LEARN]` entries should be promoted from personal-memory.md to MEMORY.md (v1.9.0) | | `/stata-replication` | `.claude/skills/stata-replication/` | End-to-end Stata pipeline scaffold + execution via the `stata-mcp` MCP server (mirrors `/data-analysis` for R-first projects; v1.9.0) | | `/simulation-study` | `.claude/skills/simulation-study/` | Reproducible Monte Carlo study — DGP, estimator grid, seeded reps, bias/RMSE/coverage/size/power with Monte Carlo SEs (v1.10.0) | | `/r-package-check` | `.claude/skills/r-package-check/` | R package release gate — `devtools::document()` + tests + `R CMD check --as-cran` + CRAN-policy triage (v1.10.0) | +| `/replication-package` | `.claude/skills/replication-package/` | Assemble a submission-ready DCAS / openICPSR replication package; calls `/audit-reproducibility` and blocks on FAIL (v2.0) | +| `/capture-environment` | `.claude/skills/capture-environment/` | Snapshot the computational environment (renv.lock / sessionInfo / requirements.txt / uv.lock + optional Dockerfile + seeds) (v2.0) | +| `/did-event-study` | `.claude/skills/did-event-study/` | Staggered DiD / event-study wrapper over canonical packages (Callaway–Sant'Anna `did`, `fixest::sunab`, HonestDiD; Stata `csdid`/`eventstudyinteract`/`honestdid`) (v2.0) | +| `/power-analysis` | `.claude/skills/power-analysis/` | Power / required-N / MDE for study design (RCT, clustered/ICC, multi-arm, or simulation-based) + a methods paragraph for `/preregister` (v2.0) | +| `/disclosure-check` | `.claude/skills/disclosure-check/` | Pre-screen restricted-data outputs for SDL violations (small cells, dominance, PII); gates on any CRITICAL (v2.0) | +| `/grant-proposal` | `.claude/skills/grant-proposal/` | Scaffold a grant proposal (NSF/NIH/ERC/foundation) from an `/interview-me` spec; delegates DMP + facilities; aims↔methods↔budget coherence pass (v2.0) | +| `/data-management-plan` | `.claude/skills/data-management-plan/` | Draft a funder-compliant Data Management Plan (NSF/NIH 2023/ERC/Horizon) (v2.0) | +| `/coauthor-brief` | `.claude/skills/coauthor-brief/` | Generate a collaborator handoff brief: git delta, per-artifact state, open questions, reproduce-locally + restricted-data steps (v2.0) | +| `/triage-inbox` | `.claude/skills/triage-inbox/` | Triage academic email + calendar (Gmail/Calendar MCP) into a prioritized digest + referee-obligations tracker; human-gated actions only (v2.0) | +| `/syllabus` | `.claude/skills/syllabus/` | Build a course syllabus from a topic/reading list (schedule, objectives, assessment + rubric, per-week `/create-lecture` work-list) (v2.0) | +| `/teach-from-paper` | `.claude/skills/teach-from-paper/` | Turn a paper into a lecture outline, teachable results, slide skeleton (→ `/create-lecture`), discussion questions, exercise brief (v2.0) | +| `/respond-to-eval` | `.claude/skills/respond-to-eval/` | Turn student course evaluations into a classified teaching-improvement plan (Keep/Change/Investigate/Out-of-scope) (v2.0) | +| `/scaffold-exercises` | `.claude/skills/scaffold-exercises/` | Scaffold a graded problem set (analytical/empirical/coding) with worked solutions; clean student set + separate solution key (v2.0) | +| `/new-skill` | `.claude/skills/new-skill/` | Scaffold a convention-compliant skill (interview → write `SKILL.md` with gate-passing frontmatter) (v2.0) | ## All Rules @@ -2698,9 +2713,10 @@ Claude Code supports **plugins** --- bundled collections of skills, agents, hook | Rule | File | Purpose | |------|------|---------| | Plan-First Workflow | `plan-first-workflow.md` | Plan mode + context preservation | -| Orchestrator Protocol | `orchestrator-protocol.md` | Contractor mode loop | +| Orchestrator Protocol | `orchestrator-protocol.md` | Review runtime: fan-out → reduce → judge (+ hallucination gate) → loop-until-dry (goal-first / gate-enforced; augments contractor mode) | | Session Logging | `session-logging.md` | Three logging triggers | | Meta-Governance | `meta-governance.md` | Template vs working project distinctions | +| Prompt Shaping | `prompt-shaping.md` | Ambient prompt-shaping habit (replaces the retired `/prompt` and `/prompt-only` skills) | **Path-scoped** (load only when working on matching files): @@ -2730,17 +2746,19 @@ Claude Code supports **plugins** --- bundled collections of skills, agents, hook | Stata Code Conventions | `stata-code-conventions.md` | `**/*.do`, `scripts/stata/**` — header scaffold, numbered pipeline, esttab tables, clustering discipline, AEA compliance (v1.9.0) | | Simulation Conventions | `simulation-conventions.md` | `**/*simulation*.R`, `**/*_sim.R`, `explorations/**` — Monte Carlo discipline: DGP/estimand, L'Ecuyer seeding, Monte Carlo SE, coverage-vs-truth, raw-result storage (v1.10.0) | | R Package Conventions | `r-package-conventions.md` | `R/**`, `tests/**`, `DESCRIPTION`, `NAMESPACE`, `man/**` — package-source standards: no `library()` in `R/`, roxygen NAMESPACE, Imports/Suggests, testthat 3e, CRAN policy (v1.10.0) | +| Confidential Data | `confidential-data.md` | restricted/confidential data paths — no raw confidential data in git, disclosure-avoidance discipline, used by `/disclosure-check` / `/replication-package` / `/data-management-plan` (v2.0) | ## Hooks | Hook | Type | Configuration | |------|------|--------------| -| Session log reminder | Stop (command) | `.claude/hooks/log-reminder.py` | +| Session log auto-writer | Stop (command) | `.claude/hooks/log-reminder.py` — auto-writes the session log on every meaningful change-set (no longer merely a reminder) | | Desktop notification | Notification (command) | `.claude/hooks/notify.sh` | -| Context state capture | PreCompact (command) | `.claude/hooks/pre-compact.py` | +| Context state capture | PreCompact (command) | `.claude/hooks/pre-compact.py` — DRAFT-block default is ON | | Context restoration | SessionStart[compact\|resume] (command) | `.claude/hooks/post-compact-restore.py` | | Context monitor | PostToolUse[Bash\|Task] (command) | `.claude/hooks/context-monitor.py` | -| Verification reminder | PostToolUse[Write\|Edit] (command) | `.claude/hooks/verify-reminder.py` | +| Git guardrails | PreToolUse (command) | `.claude/hooks/git-guardrails.py` — blocks `reset --hard` / `clean -f` / `push --force` / `add -A` | +| Claim reconcile | PostToolUse (command) | `.claude/hooks/claim-reconcile.py` — flags stale numeric claims when scripts change | **Additional hook events** available in Claude Code (not used in this template but available for custom hooks): @@ -2780,7 +2798,7 @@ Claude Code supports **plugins** --- bundled collections of skills, agents, hook ### Hooks Not Firing -**Symptom:** No context warnings, no verification reminders. +**Symptom:** No context warnings, no auto-written session log, no git guardrails firing. **Fix:** 1. Check hooks are configured: `cat .claude/settings.json | grep hooks` diff --git a/quality_reports/audits/.gitkeep b/quality_reports/audits/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/quality_reports/diagnoses/.gitkeep b/quality_reports/diagnoses/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/quality_reports/did_validation.md b/quality_reports/did_validation.md new file mode 100644 index 000000000..1902dae6f --- /dev/null +++ b/quality_reports/did_validation.md @@ -0,0 +1,76 @@ +# `/did-event-study` validation — Card & Krueger (1994) + +**Date:** 2026-06-09 · **Model:** Opus 4.8 · **Standard:** `DiD_book` 1e-6 tolerance +**Source of truth:** Pedro Sant'Anna's `DiD_book/Card_Krueger_American_Economic_Review_1994` (his code + recorded outputs). + +## Result: PASS (principle) + 1 real bug found & fixed (prescription) + +| Check | Value | vs target | Status | +|---|---|---|---| +| Reproduce his canonical 2×2 DiD (`feols(fte ~ treated*post)`) | `2.913982357430426` | his `…376` | **1e-14** ✅ | +| Skill's estimator `DRDID::drdid` reproduces it (fixed: RC + row-id) | `2.913982357430…` | his 2×2 | **2.7e-14** ✅ | +| Skill's *original* prescription `DRDID(panel=TRUE, idname=id)` | **ERROR** | — | ✗ bug | + +## The bug (why this validation mattered) +Card–Krueger is an **unbalanced** panel (409 stores; only 389 in both waves) with **2 duplicate `(id,wave)` rows**. The skill prescribed `DRDID::drdid(panel = TRUE)` with the panel id, which **errors** ("idname must be unique by tname") on exactly this common real-world shape. + +**Fixes applied:** +- Skill Phase 3: a pre-flight balance/uniqueness check; full-sample 2×2 via `panel = FALSE` + a **row-unique id** (matches `feols` to ~1e-10); balancing → `panel = TRUE` is flagged as a **different estimand**. +- `did-conventions` rule: idname-unique-by-period + check-balance-before-panel-mode is now HARD. + +## Estimand note (the `EXPLAINED` pattern, live) +- Full-sample textbook 2×2 (his target): **ATT = 2.914** +- Balanced-panel DR (389 stores, 19 attriters dropped): **ATT = 2.972** + +Both are defensible — they answer different questions. The skill now records this as a named alternative rather than presenting one number as "the" answer. + +## Caveat +SE comparison is not 1e-6: `DRDID` RC SE (1.73) treats waves as independent; `feols` clustered SE (1.29) uses the panel. For a true panel, report the clustered/panel SE. Point-estimate equivalence is the validation test. + +## Staggered + sensitivity path — VALIDATED (did::mpdta, the canonical CS example) + +Installed `HonestDiD` 0.2.8 + `didFF` 0.1.0 (local source). Ran the full skill pipeline with his defaults: + +| Step | Result | Status | +|---|---|---| +| `att_gt` (notyettreated, `dr`, universal base, bootstrap+cband) | runs clean | ✅ | +| Overall ATT (his notyettreated default) | **−0.0323** (se 0.0115) | ✅ | +| Overall ATT (nevertreated, vignette ref) | **−0.0328** vs documented ≈−0.031 | ✅ | +| Event study `aggte(dynamic)` | pre ≈ 0 (`e=-1`=0), post −0.02→−0.14 (canonical) | ✅ | +| `ggdid` | plot produced | ✅ | +| HonestDiD relative-magnitudes (direct path) | robust CIs at Mbar 0/0.5/1 | ✅ | +| `didFF` functional-form test | p = 0.998 (can't reject insensitivity) | ✅ | + +**2nd real bug found & fixed:** the skill said `honest_did()` is "README glue, not an export." Precisely, it's a **non-exported internal S3 method** in `HonestDiD 0.2.8` — bare `honest_did()` errors. The skill now ships the **validated direct recipe** (`createSensitivityResults_relativeMagnitudes` with betahat + IF-based sigma from `aggte(dynamic)`), confirmed to run on `mpdta`. + +## Stata-matches-R check — PASS (R is the benchmark; his strict 1e-6 standard) + +**R (`did::att_gt`) is the canonical implementation / benchmark; Stata (`csdid … asinr` = "as in R") must reproduce it** (not the reverse). + +`csdid lemp lpop, ... method(dripw) notyet asinr` (Stata-MP) vs `did::att_gt(... est_method="dr", control_group="notyettreated")` (R) on `mpdta`: + +| Quantity | Max \|R − Stata\| | Tol | Status | +|---|---|---|---| +| ATT(g,t), group 2004 (point) | **4.65e-08** | 1e-6 | ✅ | +| ATT(g,t), group 2004 (analytic SE) | **4.11e-08** | 1e-6 | ✅ | +| Overall simple ATT | **0.0** (−0.0413516 both) | 1e-6 | ✅ | + +His dual-software discipline — same substantive result in R and Stata to 1e-6 — is met. + +## `contdid` continuous-treatment path — PASS (reproduces his README example) + +Installed `contdid` (needed `npiv`). Ran his exact README example (`simulate_contdid_data(seed=1234)` → `cont_did(target_parameter="slope", aggregation="dose", notyettreated, num_knots=1, degree=3)`): + +| Quantity | My run | His README | Status | +|---|---|---|---| +| Overall ACRT | **0.1341** | 0.1341 | ✅ exact | + +## Overall verdict — VALIDATED END-TO-END +The `/did-event-study` pipeline is validated across **every path it prescribes**, driving *his* packages against *his* reference outputs/standards: +- **2×2** (Card–Krueger) → 1e-14; +- **staggered** `att_gt`→`aggte`→`ggdid` (mpdta) → matches vignette; +- **HonestDiD + didFF** sensitivity → run, validated recipe shipped; +- **R↔Stata** dual-software → 4.65e-08 (his 1e-6 standard); +- **continuous treatment** (`contdid`) → reproduces his README exactly. + +**2 real bugs found by running it and fixed** (panel=TRUE on unbalanced data; honest_did non-export). This is no longer "drafted in 2 hours" — it is run, broken, hardened, and verified against Pedro's own materials. diff --git a/scripts/check-surface-sync.py b/scripts/check-surface-sync.py index 9e1663331..e5feb424f 100755 --- a/scripts/check-surface-sync.py +++ b/scripts/check-surface-sync.py @@ -6,6 +6,16 @@ (agent, rule, hook) updates `.claude/` but leaves stale counts in README, CLAUDE.md, the guide source, the rendered guide, or the landing page. +Two kinds of check: + 1. COUNT assertions — prose like "13 agents, 27 skills, 21 rules" must + match the on-disk inventory (the original check). + 2. TABLE-ROW assertions — an enumerative markdown table preceded by a + `` marker must have exactly one + data row per item of on disk. This catches the drift the + count check misses: e.g. the v1.5.0 peer-review trio that was added + to `.claude/` but left OUT of the README/CLAUDE.md skills tables for + three releases (the counts were right; the table rows were stale). + Run via `./scripts/check-surface-sync.sh` pre-commit, or `/commit` will invoke it automatically. @@ -85,12 +95,38 @@ # template (not attribution, not a generic count). Each must be a scaffold # specific enough that false positives are unlikely. SINGULAR_PHRASINGS: list[tuple[str, str]] = [ - # "this template's 27" (prose shortcut in Built-In Skills callout) - (r"this template's\s+(\d+)\b", "skills"), + # "this template's 27" (prose shortcut in Built-In Skills callout). + # Match BOTH the ASCII apostrophe and the typographic ’ (U+2019) that + # Quarto emits in rendered HTML — a straight-quote-only regex let + # docs/workflow-guide.html drift to a stale count past a green gate. + (r"this template['’]s\s+(\d+)\b", "skills"), # "(N skills for LaTeX..." (templates/skill-template.md trailing note) (r"\((\d+)\s+skills?\s+for\b", "skills"), + # "The guide includes N skills" — the guide's .qmd prose AND its rendered + # HTML (an tag sits between the number and "skills", so match the + # number right after "guide includes"). This prose form matched none of + # the count regexes at v2.0 and shipped a stale "50" to live users. + (r"guide includes\s+(\d+)\b", "skills"), ] +# Enumerative-table markers. A surface opts a markdown table into the +# row-count gate by placing this comment immediately before it: +# +# +# | Skill | What It Does | +# |-------|--------------| +# | `/compile-latex` | ... | <- one data row per skill on disk +# +# must be a key of GROUND_TRUTH. The data-row count (header and the +# `|---|` separator excluded) must equal the on-disk count for that kind. +# Only markdown sources should carry the marker — do NOT add it to the +# rendered .html surfaces (their tables are , not pipe rows). +TABLE_MARKER_RE = re.compile(r"") + + +def _is_table_row(line: str) -> bool: + return line.lstrip().startswith("|") + def scan_file(path: Path) -> list[tuple[int, str, int, str]]: """ @@ -121,6 +157,56 @@ def scan_file(path: Path) -> list[tuple[int, str, int, str]]: return hits +def scan_tables(path: Path) -> list[tuple[int, str, int | None, str]]: + """ + Find every `` marker and count the + data rows of the markdown table that immediately follows it. + + Returns [(marker_line_number, kind, data_row_count, marker_raw)]. + `data_row_count` is None when no well-formed table follows the marker. + """ + if not path.exists(): + return [] + lines = path.read_text(encoding="utf-8").splitlines() + n = len(lines) + hits: list[tuple[int, str, int | None, str]] = [] + i = 0 + while i < n: + m = TABLE_MARKER_RE.search(lines[i]) + if not m: + i += 1 + continue + kind = m.group(1) + marker_lineno = i + 1 + marker_raw = lines[i].strip() + + # Advance to the header row: the first pipe-line after the marker, + # skipping intervening blanks/prose (a heading often sits between). + j = i + 1 + while j < n and not _is_table_row(lines[j]) and not TABLE_MARKER_RE.search(lines[j]): + j += 1 + + # Need: header pipe-line, then a `|---|` separator, then data rows. + if ( + j >= n + or not _is_table_row(lines[j]) + or j + 1 >= n + or "---" not in lines[j + 1] + ): + hits.append((marker_lineno, kind, None, marker_raw)) + i = j + 1 + continue + + k = j + 2 # first data row + count = 0 + while k < n and _is_table_row(lines[k]): + count += 1 + k += 1 + hits.append((marker_lineno, kind, count, marker_raw)) + i = k + return hits + + def main() -> int: rel = lambda p: p.relative_to(REPO) drift: list[str] = [] @@ -152,6 +238,32 @@ def main() -> int: f"[matched: {raw!r}]" ) + # Enumerative-table row-count assertions (marker-driven). + table_hits = 0 + for path in SURFACES: + for lineno, kind, count, raw in scan_tables(path): + table_hits += 1 + if kind not in GROUND_TRUTH: + drift.append( + f" {rel(path)}:{lineno} unknown table kind {kind!r} " + f"(expected one of {', '.join(sorted(GROUND_TRUTH))}) " + f"[marker: {raw!r}]" + ) + continue + if count is None: + drift.append( + f" {rel(path)}:{lineno} marker {raw!r} is not " + f"immediately followed by a well-formed markdown table" + ) + continue + expected = GROUND_TRUTH[kind] + if count != expected: + drift.append( + f" {rel(path)}:{lineno} '{kind}' table has {count} " + f"data row(s) (actual {kind} on disk: {expected}) " + f"[marker: {raw!r}]" + ) + if drift: print("DRIFT DETECTED:", file=sys.stderr) for d in drift: @@ -165,8 +277,8 @@ def main() -> int: return 1 total_assertions = sum(len(v) for v in per_file.values()) - print(f"All {total_assertions} count assertions match ground truth across " - f"{len(SURFACES)} surfaces.") + print(f"All {total_assertions} count assertions + {table_hits} enumerative-" + f"table row counts match ground truth across {len(SURFACES)} surfaces.") return 0 diff --git a/scripts/install-hooks.sh b/scripts/install-hooks.sh new file mode 100755 index 000000000..c4da202cb --- /dev/null +++ b/scripts/install-hooks.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# +# Install the repo's version-controlled git hooks (run once per clone). +# +# Points git at `.githooks/` so `git commit` runs the pre-commit quality +# gate (surface-sync + quality score). The hook lives in version control, +# so it stays in sync across machines and forks — unlike `.git/hooks/`, +# which is local and never committed. +# +set -euo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel)" +cd "$REPO_ROOT" + +if [ ! -d .githooks ]; then + echo "install-hooks: .githooks/ not found at repo root" >&2 + exit 1 +fi + +chmod +x .githooks/* 2>/dev/null || true +git config core.hooksPath .githooks + +echo "✓ core.hooksPath → .githooks" +echo " Every 'git commit' now runs surface-sync + quality (>=80) gates." +echo " Bypass once: SKIP_QUALITY_GATE=1 git commit ... (quality only)" +echo " git commit --no-verify ... (all hooks)" +echo " Uninstall: git config --unset core.hooksPath" diff --git a/scripts/nightly-repro-check.sh b/scripts/nightly-repro-check.sh new file mode 100755 index 000000000..c7477907c --- /dev/null +++ b/scripts/nightly-repro-check.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# +# Nightly reproducibility-drift check (cron-able; no Claude/auth needed). +# +# A thin LOCAL equivalent of the "Reproducibility drift" Routine in +# .claude/references/scheduled-routines.md, for users who prefer a machine +# cron over a managed Routine. It does NOT re-run analysis — it flags +# passport claims whose source_file / output_file is newer than the claim's +# last_verified_on (i.e. the number on disk may have moved since it was last +# checked). Exits 1 if any claim is stale, so cron can email you. +# +# For the full re-audit, run /audit-reproducibility inside Claude Code (or +# prefer the managed Routine, which survives a closed laptop). +# +# Usage: ./scripts/nightly-repro-check.sh (run from the repo root or via cron) +# +set -uo pipefail +REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" +cd "$REPO_ROOT" || exit 2 + +python3 - "$REPO_ROOT" <<'PY' +import sys, re, pathlib, datetime +root = pathlib.Path(sys.argv[1]) +pdir = root / "quality_reports" / "passports" +if not pdir.is_dir(): + print("no quality_reports/passports/ — nothing to check"); sys.exit(0) + +def parse_iso(s): + s = s.strip().strip('"').strip("'") + date_only = ("T" not in s) and (":" not in s) # e.g. "2026-06-01" + try: + dt = datetime.datetime.fromisoformat(s.replace("Z", "+00:00")) + if date_only: # treat as END of that day so a + dt = dt.replace(hour=23, minute=59, second=59) # same-day edit isn't STALE + return dt.timestamp() + except Exception: + return None + +stale = [] +for pf in sorted(pdir.glob("*.yaml")): + lines = pf.read_text(errors="replace").splitlines() + cur = {} + def flush(c): + lv = c.get("last_verified_on") + for key in ("source_file", "output_file"): + f = c.get(key) + if f and lv is not None: + p = root / f + try: + if p.exists() and p.stat().st_mtime > lv: + stale.append(f"{pf.name}: {c.get('id','?')} — {f} newer than last_verified_on") + except Exception: + pass + for ln in lines: + m = re.match(r"\s*-\s*id:\s*(.+)", ln) + if m: + flush(cur); cur = {"id": m.group(1).strip()} + continue + m = re.match(r"\s*(source_file|output_file|last_verified_on):\s*(.+)", ln) + if m: + k, v = m.group(1), m.group(2).strip() + cur[k] = parse_iso(v) if k == "last_verified_on" else v + flush(cur) + +if stale: + print("STALE claims (re-run /audit-reproducibility):") + for s in stale: print(" " + s) + sys.exit(1) +print("All tracked claims fresh (no source/output newer than last_verified_on).") +PY diff --git a/scripts/validate-setup.sh b/scripts/validate-setup.sh index 1c05a831d..8be121940 100755 --- a/scripts/validate-setup.sh +++ b/scripts/validate-setup.sh @@ -97,6 +97,31 @@ else echo -e " ${YELLOW}⚠${RESET} .claude/hooks/ directory not found (are you in the project root?)" warn=$((warn + 1)) fi + +echo "" +echo -e "${BOLD}Git pre-commit gate (v2.0):${RESET}" +pchook="$(dirname "$0")/../.githooks/pre-commit" +if [ -f "$pchook" ]; then + if [ -x "$pchook" ]; then + echo -e " ${GREEN}✓${RESET} .githooks/pre-commit is executable" + pass=$((pass + 1)) + else + echo -e " ${YELLOW}⚠${RESET} .githooks/pre-commit is NOT executable — git silently skips it, disabling the gate" + echo -e " Fix: chmod +x .githooks/pre-commit (or re-run ./scripts/install-hooks.sh)" + warn=$((warn + 1)) + fi + if command -v git >/dev/null 2>&1; then + if [ "$(git config core.hooksPath 2>/dev/null || true)" = ".githooks" ]; then + echo -e " ${GREEN}✓${RESET} core.hooksPath → .githooks (gate active on every commit)" + else + echo -e " ${YELLOW}⚠${RESET} pre-commit gate not activated — run ./scripts/install-hooks.sh" + warn=$((warn + 1)) + fi + fi +else + echo -e " ${YELLOW}⚠${RESET} .githooks/pre-commit not found" + warn=$((warn + 1)) +fi echo "" echo -e "${BOLD}Palette sync (LaTeX ↔ SCSS):${RESET}" diff --git a/templates/archive-readme.md b/templates/archive-readme.md index e6faf2788..ffa6cc87c 100644 --- a/templates/archive-readme.md +++ b/templates/archive-readme.md @@ -1,3 +1,7 @@ + + # Abandoned: [Project Name] ([DATE RANGE]) ## Why Abandoned diff --git a/templates/passport-template.yaml b/templates/passport-template.yaml index bc3f20d54..8c34ca7b3 100644 --- a/templates/passport-template.yaml +++ b/templates/passport-template.yaml @@ -30,7 +30,7 @@ claims: n: exact # sample-size mismatch is never tolerated last_verified_on: 2026-05-20T00:00:00Z last_verified_by: "/audit-reproducibility" - status: UNVERIFIED # PASS | FAIL | STALE | UNVERIFIED + status: UNVERIFIED # PASS | FAIL | EXPLAINED | STALE | UNVERIFIED notes: | Optional notes. Use this field to record: - Known small discrepancies and why they're within tolerance @@ -39,6 +39,10 @@ claims: - Cross-references to other claims that depend on this one (e.g., "C5 derives a derived quantity from C1; both must update together"). + - To downgrade an out-of-tolerance FAIL to EXPLAINED, a CONCRETE + named alternative spec (e.g., "never-treated vs not-yet-treated + comparison group; published value matches under not-yet-treated"). + A blank or vague note ("unclear") never downgrades a FAIL. # Example claim 2 — a percentage figure from a robustness check - id: C2 diff --git a/templates/skill-template.md b/templates/skill-template.md index f3fef5899..8718833d3 100644 --- a/templates/skill-template.md +++ b/templates/skill-template.md @@ -437,4 +437,4 @@ When adapting this template to your domain: - **Purpose:** Starter for domain-specific skills - **Usage:** Copy to `.claude/skills/[name]/SKILL.md`, customize for your field -For existing skills examples, see `.claude/skills/` directory (38 skills for LaTeX, R, Quarto, and research workflows). +For existing skills examples, see `.claude/skills/` directory (51 skills for LaTeX, R, Quarto, and research workflows).