diff --git a/CLAUDE.md b/CLAUDE.md index 1a5473d8..52ff90f0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -70,7 +70,7 @@ claude/lightcone/ # Claude plugin source — force-included into the w ├── skills/ # lc-new, lc-from-code, lc-from-paper, │ # lc-feedback, ralph; │ # paper-reproduction bundle: lc-from-paper (entry), -│ # ralph (loop substrate), narrative, +│ # ralph (loop substrate), │ # paper-extraction, figure-comparison, │ # check-sentence-by-sentence │ # (see skills/README.md for the full bundle map) diff --git a/claude/lightcone/skills/README.md b/claude/lightcone/skills/README.md index 6d3a238d..5501fdda 100644 --- a/claude/lightcone/skills/README.md +++ b/claude/lightcone/skills/README.md @@ -18,7 +18,7 @@ Not direct entry points — Claude invokes these (or other skills invoke them) t | Skill | Role | |---|---| -| `astra` | Reference for the `astra.yaml` spec: structure, decisions, options, prior insights, findings, evidence, sub-analyses, narrative anchors, composition mechanics. | +| `astra` | Reference for the `astra.yaml` spec: structure, decisions, options, prior insights, findings, evidence, sub-analyses, composition mechanics. | | `lc-cli` | Reference for `lc` workflow: commands, the Spec-Code Invariant, status interpretation, failure diagnosis, multiverse runs, WRROC export. | ## Paper-reproduction bundle @@ -29,7 +29,6 @@ A self-contained toolkit for reproducing published papers in ASTRA. The bundle i |---|---| | [`lc-from-paper`](lc-from-paper/SKILL.md) | **Reproduction driver.** ORIENT-first; one pre-loop phase in the user's main session that asks for the paper, runs `/paper-extraction` inline, interviews the user (grounded in the paper), clones the reference code and runs `/lc-from-code` scan-only (when a repo exists), and drafts the per-paper `constitution.md` + `CLAUDE.md`. Then hands off to a ralph loop whose iterations carry the long middle: ARCHITECT → SPECIFY → LITERATURE → IMPLEMENT → RUN → COMPARE. When the loop closes (constitution `status: closed` after COMPARE returns `pass`), REVIEW runs back in the user's main session. Fidelity intent — captured as prose at ORIENT — is what every iteration reads when sizing its next move, and what COMPARE grades opportunities against. | | [`ralph`](ralph/SKILL.md) | The loop substrate. `lc-from-paper`'s ORIENT invokes `/ralph`'s Authoring mode to draft the per-paper constitution; the loop launcher hands off after ORIENT lands. Each iteration runs `/ralph`'s Loop protocol against the constitution. | -| [`narrative`](narrative/SKILL.md) | Author the `narrative:` prose and decision `rationale:` in `astra.yaml`. Invoked by `lc-from-paper`'s ARCHITECT (for the structural narrative) and SPECIFY (for anchored content narrative). | | [`paper-extraction`](paper-extraction/SKILL.md) | Turn an arXiv ID or DOI into a standardized `work/reference/` directory: structural index (figures, tables, outline, citations with resolved DOIs) plus a stub `astra.yaml` for the paper. Primary acquisition path for `lc-from-paper`'s ORIENT (Stage 2); also invoked per cited paper by LITERATURE. | | [`check-sentence-by-sentence`](check-sentence-by-sentence/SKILL.md) | Audit paper claims against code locations (`file:line` or `NOT FOUND`). Invoked from `lc-from-paper`'s REVIEW close-out (opt-in); also user-invokable directly. | | [`figure-comparison`](figure-comparison/SKILL.md) | Build a self-contained HTML side-by-side: original figures/tables/numerics vs replicated. Invoked from `lc-from-paper`'s REVIEW close-out (mandatory); also user-invokable directly. | diff --git a/claude/lightcone/skills/astra/SKILL.md b/claude/lightcone/skills/astra/SKILL.md index e46a044e..51430183 100644 --- a/claude/lightcone/skills/astra/SKILL.md +++ b/claude/lightcone/skills/astra/SKILL.md @@ -3,8 +3,8 @@ name: astra description: > Comprehensive reference for the `astra.yaml` specification — top-level structure, sub-analyses, inputs/outputs, decisions and options, prior - insights and findings, evidence and quote verification, narrative - anchors, and composition mechanics. Invoke whenever reading, writing, + insights and findings, evidence and quote verification, and + composition mechanics. Invoke whenever reading, writing, validating, or debugging an `astra.yaml` spec; whenever working with decisions, options, prior_insights, findings, or evidence; or whenever the user asks about ASTRA schema, spec syntax, or sub-analysis @@ -22,13 +22,13 @@ An `astra.yaml` spec captures this for a single unit of work. The structure is * ## astra.yaml Structure -Fields: `id`, `version`, `name`, `narrative`, `authors`, `tags`, `inputs`, `outputs`, `decisions`, `prior_insights`, `findings`, `analyses`, `container`. `narrative` is the analysis-level prose field -- see [Narrative](#narrative) (typically filled in later, once the structural pieces have settled). +Fields: `id`, `version`, `name`, `description`, `tags`, `inputs`, `outputs`, `decisions`, `prior_insights`, `findings`, `analyses`, `container`. `description` is the analysis-level free-prose field -- see [Description](#description) (the same optional field every element carries). -**Reserved IDs.** No analysis entity (input, output, decision, option, finding, prior insight, evidence, sub-analysis) may use any of these names as its `id` -- they collide with the narrative anchor grammar: +**Reserved IDs.** No analysis entity (input, output, decision, option, finding, prior insight, evidence, sub-analysis) may use any of these names as its `id` -- they collide with the tree-path reference grammar (used by `from:`, `when`, `requires`, and `incompatible_with`): ``` inputs outputs decisions findings prior_insights -analyses options content narrative +analyses options content ``` **`label` field.** Inputs, Outputs, Decisions, Options, and Insights all accept an optional `label:` -- a short human-readable name for compact rendering (margin glyphs, breadcrumbs, card titles). Tooling falls back to `id` when absent. `label` is required only on Options. @@ -37,7 +37,7 @@ analyses options content narrative # Simple analysis -- everything at top level version: "1.0" name: "My Analysis" -# narrative: { ... } # see Narrative section; typically added later +# description: "One-paragraph orientation (optional)" # see Description section inputs: - id: training_data type: data @@ -351,46 +351,21 @@ decisions: The **`universe:` field** in universe files selects which sub-analysis universe to load: `build_mocks: { universe: baseline }` loads `./analyses/build_mocks/universes/baseline.yaml`. -## Narrative +## Description -`narrative` is the analysis-level prose field on any Analysis (root or sub). It's structured as five Markdown sections: `summary`, `findings`, `methods`, `inputs`, `outputs`. The schema is closed (`additionalProperties: false`) -- no other keys are allowed. +`description` is a single optional free-prose field on any Analysis (root or sub) -- the same field every other element carries (`Input`, `Output`, `Option`, `Universe`). It holds a short human orientation to the analysis (a paragraph or two), nothing more. -**Recommendation:** fill `narrative` in *later*, once the structural pieces of the analysis (decisions, outputs, sub-analyses) have settled. Prose written too early goes stale fast and tends to describe what no longer exists. Per-element prose (what each Input, Output, Decision, Option, or Insight is and why) belongs on the elements themselves via `description`/`rationale`/`notes` -- those can be written from day one. - -**Conditional coverage.** All five sections are schema-optional, but `astra validate` enforces: - -| Section | Required when | -|---|---| -| `findings` | the analysis has entries under `findings:` | -| `methods` | the analysis has entries under `decisions:` or `analyses:` | -| `inputs` | the analysis has entries under `inputs:` | -| `outputs` | the analysis has entries under `outputs:` | -| `summary` | always optional | - -Authors narrate what they declare; stub analyses with only a summary stay clean. - -**Anchor references.** Inside any section, link to other elements with Markdown anchor links (`[text](#path.to.element)`) using the same tree-path grammar as `from:` -- `#decisions.scaling`, `#decisions.scaling.options.standard`, `#findings.best_model`, `#analyses.preprocessing` (whole sub-analysis), `#analyses.preprocessing.outputs.features` (element inside a sub-analysis), `#../decisions.method` to escape to a parent scope. - -**Inline images.** Standard Markdown image syntax inside any section -- `![alt](path/to/img.png)` for repo-relative paths or `![alt](https://...)` for URLs. Renderers like lightcone-ui pick them up the same way they pick up text. +Per-element prose (what each Input, Output, Decision, Option, or Insight is and why) belongs on the elements themselves via `description`/`rationale`/`notes` -- those can be written from day one. The analysis-level `description` can be filled in at any time and is safe to leave short. ```yaml -narrative: - summary: | - A two-stage pipeline for Iris classification that demonstrates - sub-analyses. - methods: | - The [feature_extraction sub-analysis](#analyses.feature_extraction) - produces encoded features, which feed - [classification](#analyses.classification). A - [test_split](#decisions.test_split) decision controls the holdout. - inputs: | - [iris_data](#inputs.iris_data) is Fisher's 150-sample, 4-feature, - 3-class dataset. - outputs: | - The top level exposes [accuracy](#outputs.accuracy) and a - [pipeline_summary](#outputs.pipeline_summary) report. +description: | + A two-stage pipeline for Iris classification that demonstrates + sub-analyses: a feature-extraction stage feeds a classification + stage. The top level exposes the classifier accuracy and a + pipeline-summary report. ``` + ## CLI Reference (astra) ```bash diff --git a/claude/lightcone/skills/check-sentence-by-sentence/SKILL.md b/claude/lightcone/skills/check-sentence-by-sentence/SKILL.md index 06c0211a..7e3e8fdc 100644 --- a/claude/lightcone/skills/check-sentence-by-sentence/SKILL.md +++ b/claude/lightcone/skills/check-sentence-by-sentence/SKILL.md @@ -149,9 +149,10 @@ PROCEDURE or generic framing -- skip it. 3. Before searching, **read `astra.yaml` once** -- it is a pre-built paper↔code map maintained by the project. Harvest specifically: - - `narrative.methods` — links paper methodology concepts to decision - IDs (e.g. paper prose "the chosen " → `#decisions.`) - - `narrative.findings` — links paper claims/values to result anchors + - `decisions` — each decision's `label` / `rationale` links paper + methodology concepts to a decision ID (e.g. paper prose "the chosen + " → `decisions.`) + - `findings` — links paper claims/values to result entries - `prior_insights` (if present) — extracted paper quotes already tied to decisions - per-decision `evidence` quotes and `description` fields diff --git a/claude/lightcone/skills/figure-comparison/SKILL.md b/claude/lightcone/skills/figure-comparison/SKILL.md index 0c5247ce..0025e4c4 100644 --- a/claude/lightcone/skills/figure-comparison/SKILL.md +++ b/claude/lightcone/skills/figure-comparison/SKILL.md @@ -93,8 +93,8 @@ Read, in this order: - If neither file exists, use the default paper-driven flow below and build a best-effort report from `astra.yaml` plus `work/reference/`. -2. **`astra.yaml`** -- specifically `narrative.summary`, `narrative.outputs`, - `narrative.findings`, `outputs:`, and `findings:` if present. Use it to +2. **`astra.yaml`** -- specifically the top-level `description`, `outputs:`, + and `findings:` if present. Use it to map scoped targets to output IDs and to harvest declared findings. Do not assume ASTRA outputs have a dedicated filename-hint field; result paths come from the output ID and the result resolver in Phase 2. @@ -200,7 +200,7 @@ paper asserts and the project tracks. Concretely, harvest from: reported sample size after a specific cut, every bin width or step used as a result-defining choice, every reported accuracy / score / metric. -- Any explicit reproduction targets in `astra.yaml`'s `narrative.findings`. +- Any explicit reproduction targets in `astra.yaml`'s `findings:`. It is fine to repeat one quantity in multiple manifest entries when the paper reports it under different conditions (preliminary vs. final, diff --git a/claude/lightcone/skills/lc-from-code/SKILL.md b/claude/lightcone/skills/lc-from-code/SKILL.md index b4da467a..f81ac86f 100644 --- a/claude/lightcone/skills/lc-from-code/SKILL.md +++ b/claude/lightcone/skills/lc-from-code/SKILL.md @@ -61,7 +61,7 @@ When the codebase is large enough that one Explore pass risks missing depth (a m Write the scan results to `CLAUDE.md` under `## Project Notes` (fresh migration) or to the path the invocation prompt specifies (scan-only — typically `work/reference/code-index.md`) as a script inventory, then in fresh migration mode draft or add to `astra.yaml` from the scan results following the spec structure documented in `/astra`. In scan-only mode, stop after the inventory file lands; do not touch `astra.yaml`. Use the decision criteria from `/astra` (Decisions section) to filter candidate decisions down to only true analytical choices — most hardcoded values are implementation details, not decisions. Use current hardcoded values as defaults. -In augment mode, preserve the existing paper-derived or user-derived `inputs`, `outputs`, `decisions`, `findings`, and `narrative` unless the code scan shows a real conflict. Attach code evidence to the nearest existing home first. Create new ASTRA structure only when the code reveals a real analysis object that has no suitable home in the current spec. +In augment mode, preserve the existing paper-derived or user-derived `inputs`, `outputs`, `decisions`, `findings`, and `description` unless the code scan shows a real conflict. Attach code evidence to the nearest existing home first. Create new ASTRA structure only when the code reveals a real analysis object that has no suitable home in the current spec. For each output, list the upstream artifacts it depends on under `Output.inputs: [...]` and the decisions it consumes under `Output.decisions: [...]`. Then add a `recipe.command` template that references each via `{inputs.}` / `{decisions.}` and writes to `{output}`. Example: diff --git a/claude/lightcone/skills/lc-from-paper/SKILL.md b/claude/lightcone/skills/lc-from-paper/SKILL.md index 78e79db8..b48c1c65 100644 --- a/claude/lightcone/skills/lc-from-paper/SKILL.md +++ b/claude/lightcone/skills/lc-from-paper/SKILL.md @@ -36,8 +36,8 @@ Eight phases (zero-indexed). ORIENT runs before the loop, in the user's main ses | # | Phase | Where it runs | Reference | Primary outputs | |---|---|---|---|---| | 0 | ORIENT | user's main session | [`references/orient.md`](references/orient.md) | per-paper `constitution.md` + `CLAUDE.md` + paper substrate at `work/reference/{paper.pdf, source/ or document.md, figures/, tables/, index.json, astra.yaml}` (from inline `/paper-extraction`) + code substrate at `work/reference/{code/, code-status.yaml, code-index.md}` (from inline `/lc-from-code` scan-only, when a repo exists) | -| 1 | ARCHITECT | ralph iteration | [`references/architect.md`](references/architect.md) | stub `astra.yaml` at project root (sub-analyses, inputs, outputs, narrative) | -| 2 | SPECIFY | ralph iteration | [`references/specify.md`](references/specify.md) | filled `astra.yaml` (`decisions:`, `findings:`, `prior_insights:` placeholders, anchored narrative); `targets/targets.md`; `implementation-notes.md`; `universes/baseline.yaml` | +| 1 | ARCHITECT | ralph iteration | [`references/architect.md`](references/architect.md) | stub `astra.yaml` at project root (sub-analyses, inputs, outputs, per-analysis `description`) | +| 2 | SPECIFY | ralph iteration | [`references/specify.md`](references/specify.md) | filled `astra.yaml` (`decisions:`, `findings:`, `prior_insights:` placeholders); `targets/targets.md`; `implementation-notes.md`; `universes/baseline.yaml` | | 3 | LITERATURE | ralph iteration | [`references/literature.md`](references/literature.md) | `astra.yaml`'s `prior_insights:` Evidence entries each carry resolved `quote:` + `location:` selectors; per-paper PDFs cached via `astra paper add` | | 4 | IMPLEMENT | ralph iteration | [`references/implement.md`](references/implement.md) | `scripts/`, `requirements.txt`, recipes in `astra.yaml` | | 5 | RUN | ralph iteration | [`references/run.md`](references/run.md) | `results///` | diff --git a/claude/lightcone/skills/lc-from-paper/references/architect.md b/claude/lightcone/skills/lc-from-paper/references/architect.md index a2556e77..c93e4b3b 100644 --- a/claude/lightcone/skills/lc-from-paper/references/architect.md +++ b/claude/lightcone/skills/lc-from-paper/references/architect.md @@ -1,6 +1,6 @@ # ARCHITECT — write the stub `astra.yaml` -ARCHITECT is the structural seam: decide the sub-analysis decomposition, wire the inputs and outputs at the sub-analysis level, and author high-level narrative prose for each analysis — all in one stub `astra.yaml`. SPECIFY then fills the stub with `decisions:`, `prior_insights:`, `findings:`, and `astra-anchor:` references. Splitting **structure** from **content** keeps each iteration's cognitive load manageable: ARCHITECT decides *what the analyses are*; SPECIFY decides *what's inside each one*. +ARCHITECT is the structural seam: decide the sub-analysis decomposition, wire the inputs and outputs at the sub-analysis level, and author a short `description:` for each analysis — all in one stub `astra.yaml`. SPECIFY then fills the stub with `decisions:`, `prior_insights:`, and `findings:`. Splitting **structure** from **content** keeps each iteration's cognitive load manageable: ARCHITECT decides *what the analyses are*; SPECIFY decides *what's inside each one*. ARCHITECT is what a ralph iteration does when the workdir signals "ORIENT substrate present + project-root `astra.yaml` absent (or empty stub)." The heavy work of *understanding* the paper and code happened in `/paper-extraction` and `/lc-from-code`'s scan-only branch — both invoked inline during ORIENT in the user's main session. Their on-disk substrate (the structural `index.json`, the paper-extraction `astra.yaml`, the `code-index.md`) is what you read on entry. No persistent expert sub-agents; targeted reads against the substrate carry the orientation. @@ -9,7 +9,7 @@ ARCHITECT is what a ralph iteration does when the workdir signals "ORIENT substr - `constitution.md` — Goal, Fidelity intent, Scope, Quality bar. Read first; the Goal's intended replication targets fence what `outputs:` belong in the stub. - `CLAUDE.md` — auto-loaded; Rules + accumulators (still empty at this point). - `work/reference/index.json` — paper-side structural index from `/paper-extraction` (figures, tables, section outline with line numbers, citations with resolved DOIs). -- `work/reference/astra.yaml` — paper-extraction's ASTRA-shape stub of the paper itself: id, name, `narrative.summary` (from abstract), optionally `findings:` (paper's claimed numerical results). +- `work/reference/astra.yaml` — paper-extraction's ASTRA-shape stub of the paper itself: id, name, `description` (from abstract), optionally `findings:` (paper's claimed numerical results). - `work/reference/code-index.md` — code-side inventory from `/lc-from-code`'s scan: script inventory, candidate decisions with `file:line` refs, module map, entry-points, external data dependencies, container hints. - `work/reference/source/` (Path A) or `work/reference/document.md` (Path B) — paper text. Grep into for specific facts; do not re-read whole. - `work/reference/code/` (when present) — the cloned reference code. Read targeted modules when `code-index.md` doesn't answer a structural question. @@ -17,7 +17,7 @@ ARCHITECT is what a ralph iteration does when the workdir signals "ORIENT substr ## Outputs -- `astra.yaml` at the project root — **stub form**: sub-analyses named, architecture wired (inputs / outputs declared at the sub-analysis level), high-level `narrative:` prose blocks per analysis. **No `decisions:`, `prior_insights:`, `findings:`, or `astra-anchor:` references yet** — those entries don't exist for the narrative to reference. +- `astra.yaml` at the project root — **stub form**: sub-analyses named, architecture wired (inputs / outputs declared at the sub-analysis level), a short `description:` per analysis. **No `decisions:`, `prior_insights:`, or `findings:` yet** — those entries are SPECIFY's. - `constitution.md` updates: Open dimensions, when something material surfaces that warrants user ratification at REVIEW. ## Step 1 — Read the substrate, then write the stub @@ -27,32 +27,29 @@ Read `constitution.md`, `CLAUDE.md`, `work/reference/index.json`, `work/referenc ### What to do 1. **Reconcile sub-analysis decompositions.** Read `code-index.md`'s natural-decomposition section and `index.json`'s section outline. Where paper and code agree on a stage, use that name (noun-phrase, e.g. `reconstruction`). Where they disagree, **code's structure is canonical for stage boundaries** — the paper compresses; the code reveals the actual decomposition. Where code is absent or thin, follow the paper alone. Where module boundaries are genuinely ambiguous, read the relevant modules under `work/reference/code/` to settle it. -2. **Choose: one analysis or sub-analyses?** If the paper has only one stage end-to-end (no clean intermediate handoffs), write a single analysis. If it has genuinely independent stages (each stage's output flows as the next's input), write sub-analyses. Sub-analysis IDs must be noun phrases: `reconstruction`, `clustering`, `bao_fit`. Avoid reserved names: `inputs`, `outputs`, `decisions`, `findings`, `prior_insights`, `analyses`, `options`, `content`, `narrative`. +2. **Choose: one analysis or sub-analyses?** If the paper has only one stage end-to-end (no clean intermediate handoffs), write a single analysis. If it has genuinely independent stages (each stage's output flows as the next's input), write sub-analyses. Sub-analysis IDs must be noun phrases: `reconstruction`, `clustering`, `bao_fit`. Avoid reserved names: `inputs`, `outputs`, `decisions`, `findings`, `prior_insights`, `analyses`, `options`, `content`. 3. **Wire inputs and outputs at the sub-analysis level.** For each sub-analysis: - Declare `inputs:` from `code-index.md`'s External-data-dependencies plus any paper-named external datasets. The depth (acquisition path, selection criteria) is SPECIFY's; ARCHITECT names the input and gives it a stable id. - Declare `outputs:` matching the result loci from `index.json` (figures + tables) plus any intermediate artifacts a downstream sub-analysis consumes. Tag each output's `priority:` from the paper's emphasis (primary / secondary). **The reproduction's targeted scope from `constitution.md`'s Scope takes precedence** — if the user only wants Figure 3 and Table 2, only those land as `outputs:`; the rest are out-of-scope and noted as such. -4. **Author the root and per-analysis narrative.** Invoke `/narrative` for prose authoring (it carries the discipline on reserved names, voice, the data-flow paragraph requirement). High-level prose only — **no `astra-anchor:` references yet**, because the entries those would point at don't exist. SPECIFY will weave in anchors as it authors `decisions:` / `prior_insights:` / `findings:` per sub-analysis. The root `narrative:` MUST include a top-down end-to-end data-flow paragraph (per the narrative skill's data-flow rules) when sub-analyses exist. -5. **Validate.** `astra validate astra.yaml` must return clean — even with empty `decisions:` / `prior_insights:` / `findings:` blocks, the structural fields and narrative prose must pass schema checks. +4. **Author the root and per-analysis `description`.** Write a short `description:` (one or two paragraphs) on the root analysis and on each sub-analysis — enough to orient a reader on what the analysis is and what it produces. The root `description:` should give a top-down, end-to-end sketch of how the sub-analyses' outputs flow into one another when sub-analyses exist. Keep it high-level; per-decision and per-finding prose lives on those entries' own `rationale:` / `notes:` fields, authored in SPECIFY. +5. **Validate.** `astra validate astra.yaml` must return clean — even with empty `decisions:` / `prior_insights:` / `findings:` blocks, the structural fields and descriptions must pass schema checks. ### Stub shape — what `astra.yaml` looks like after ARCHITECT ```yaml -# Stub: structure + narrative. SPECIFY fills decisions/findings/prior_insights and weaves astra-anchor references into the narrative. +# Stub: structure + descriptions. SPECIFY fills decisions/findings/prior_insights. id: title: "" doi: -narrative: - summary: | - - methods: | - +description: | + analyses: : - narrative: - summary: | - + description: | + inputs: : @@ -74,9 +71,9 @@ analyses: - **Stub, not snapshot.** Don't try to author content for `decisions:`, `prior_insights:`, `findings:`. Those go in SPECIFY. Your job is the structural skeleton. - **Reserved names.** Sub-analysis IDs are noun phrases; avoid the reserved set. Each ID must be unique across the spec. -- **Code-as-canonical for structure.** Where paper and code disagree on the decomposition, the code's structure is canonical (the paper compresses for narrative; the code reveals real seams). +- **Code-as-canonical for structure.** Where paper and code disagree on the decomposition, the code's structure is canonical (the paper compresses; the code reveals real seams). - **Targeted scope wins.** `constitution.md`'s Scope fences the reproduction. If the user only wants Figures 3–4 plus Table 2, only those land as `outputs:`. -- **Narrative prose, no anchors.** Author `narrative:` prose at root and per-sub-analysis levels. Do NOT add `astra-anchor:` references — the entries those would point at don't exist yet. +- **Short descriptions, high-level only.** Author a `description:` at root and per-sub-analysis levels. Keep it orienting; per-decision / per-finding prose lives on those entries in SPECIFY. - **Validate before exit.** `astra validate astra.yaml` must return clean. - **Targeted reads, not whole-paper absorption.** The indices give you most of what you need; reach into the source / document / code for specific items, not as a default. @@ -84,15 +81,15 @@ After the stub is written and validates, commit it (`architect: stub astra.yaml` ## Reviewing prior ARCHITECT work as part of survey -There is no separate review phase. Every iteration that enters and finds an ARCHITECT stub on disk reads it critically before doing anything else. If you see real issues — wrong sub-analysis decomposition, reserved-name collision, missing in-scope output, narrative gap — fix them inline, commit (`architect: fix `), and exit. Only when a fresh-context read finds nothing to fix does the iteration move on to SPECIFY work. The fresh-context property at iteration boundaries makes the next iteration the review; nothing else is needed. +There is no separate review phase. Every iteration that enters and finds an ARCHITECT stub on disk reads it critically before doing anything else. If you see real issues — wrong sub-analysis decomposition, reserved-name collision, missing in-scope output, a `description` gap — fix them inline, commit (`architect: fix `), and exit. Only when a fresh-context read finds nothing to fix does the iteration move on to SPECIFY work. The fresh-context property at iteration boundaries makes the next iteration the review; nothing else is needed. What to look at: 1. **Sub-analysis decomposition.** Right cuts? Consistent with `code-index.md`? Defensible against the paper where the paper compresses? -2. **Sub-analysis IDs.** Noun phrases. No reserved-name collisions (`inputs`, `outputs`, `decisions`, `findings`, `prior_insights`, `analyses`, `options`, `content`, `narrative`). +2. **Sub-analysis IDs.** Noun phrases. No reserved-name collisions (`inputs`, `outputs`, `decisions`, `findings`, `prior_insights`, `analyses`, `options`, `content`). 3. **Inputs at sub-analysis level.** Each input has a stable id; the data dependency is real (cross-check against `code-index.md`'s External-data-dependencies and the paper's data section). 4. **Outputs at sub-analysis level.** Each output corresponds to a result locus from `index.json` OR an intermediate artifact a downstream sub-analysis consumes. Targeted scope from `constitution.md`'s Scope is honored — no out-of-scope outputs sneaking in, no in-scope targets missed. -5. **Narrative coverage.** Root narrative includes a data-flow paragraph (when sub-analyses exist). Each sub-analysis's narrative accurately describes its role. No `astra-anchor:` references at this stage. +5. **Description coverage.** Root `description` sketches the end-to-end data flow (when sub-analyses exist). Each sub-analysis's `description` accurately describes its role. 6. **Validates.** `astra validate astra.yaml` returns clean. Don't flag empty `decisions:` / `prior_insights:` / `findings:` — that's SPECIFY's territory. Don't re-read the entire paper or code; use the indices and targeted reads. If you see the same artifact getting churned across many recent commits without convergence, log the situation to `open-questions.md` and advance the phase anyway. @@ -101,12 +98,12 @@ Don't flag empty `decisions:` / `prior_insights:` / `findings:` — that's SPECI - `work/reference/index.json` + `work/reference/astra.yaml` + `work/reference/code-index.md` (when code present) exist ⇒ ORIENT substrate is ready - `astra.yaml` at project root absent (or present-but-empty) ⇒ this iteration writes the stub -- `astra.yaml` exists with stub form (sub-analyses + inputs + outputs + narrative populated; `decisions:` / `prior_insights:` / `findings:` blocks present-and-empty) ⇒ ARCHITECT's output is on disk; read it critically. Fix anything wrong; otherwise the iteration moves on to SPECIFY. +- `astra.yaml` exists with stub form (sub-analyses + inputs + outputs + per-analysis `description` populated; `decisions:` / `prior_insights:` / `findings:` blocks present-and-empty) ⇒ ARCHITECT's output is on disk; read it critically. Fix anything wrong; otherwise the iteration moves on to SPECIFY. ## Notes - **No persistent expert sub-agents.** The on-disk substrate (`index.json`, `code-index.md`, the paper-extraction `astra.yaml`) carries the orientation iterations need; re-read what you need on entry. - **The stub's empty blocks are intentional.** `decisions: {}`, `prior_insights: {}`, `findings: {}` make it clear at a glance that ARCHITECT's job is structural and SPECIFY fills them. Don't try to half-author content — empty is honest. -- **Code-as-canonical for structure, paper-as-canonical for narrative voice.** The code reveals where the real stage boundaries are; the paper provides the words to describe them. The stub uses both. -- **The narrative skill is the prose author, not the structure author.** Invoke `/narrative` for the prose blocks; ARCHITECT's job is the structural skeleton plus invoking `/narrative` to fill the `narrative:` keys cleanly. +- **Code-as-canonical for structure, paper-as-canonical for wording.** The code reveals where the real stage boundaries are; the paper provides the words to describe them. The stub uses both. +- **Descriptions are orienting, not exhaustive.** ARCHITECT's `description:` blocks give a reader the shape of each analysis; the detailed prose (decision `rationale:`, finding `notes:`) lands on those entries in SPECIFY. - **Commit each artifact as it lands.** The stub commits when it lands; each subsequent fix pass commits separately. Small, descriptive commits keep `git log` legible to the next iteration. diff --git a/claude/lightcone/skills/lc-from-paper/references/implement.md b/claude/lightcone/skills/lc-from-paper/references/implement.md index 7822e346..e0e6c1df 100644 --- a/claude/lightcone/skills/lc-from-paper/references/implement.md +++ b/claude/lightcone/skills/lc-from-paper/references/implement.md @@ -6,7 +6,7 @@ IMPLEMENT is what a ralph iteration does when the workdir signals "SPECIFY done ## Inputs -- `astra.yaml` — the filled spec (sub-analyses, decisions, prior_insights, findings, narrative — all populated by SPECIFY) +- `astra.yaml` — the filled spec (sub-analyses, decisions, prior_insights, findings — all populated by SPECIFY; per-analysis `description` from ARCHITECT) - `implementation-notes.md` — tricky algorithms, numerical gotchas, data-format quirks - `work/reference/index.json` — paper-side structural index (figures, tables, outline, citations); useful when the spec compresses or you need to find where in the paper a behavior is described. - `work/reference/code-index.md` (when code present) — code inventory: module map, candidate decisions with file:line, entry-points, data dependencies, gotchas (the canonical map of where each sub-analysis's logic lives in `work/reference/code/`). @@ -54,7 +54,7 @@ The iteration merges scripts and recipes after the per-output sub-agents finish. 1. **One script per output** (or a shared script for tightly-coupled outputs). 2. **Parameterize by decisions.** Each decision is a CLI argument; scripts also receive `--universe `. See lightcone-cli's `CLAUDE.md` for the full convention. -3. **Add recipes** to each output in `astra.yaml` with `command:` and `inputs:` (dependencies). Recipe inputs use the same `.` form the narrative skill's data-flow rules require. +3. **Add recipes** to each output in `astra.yaml` with `command:` and `inputs:` (dependencies). Recipe inputs use the `.` tree-path form that wires outputs to downstream inputs. 4. **Create `requirements.txt`** with needed packages. Do not install them — the RUN phase manages environments. 5. **Do not execute scripts** — the RUN phase handles execution via `lc run`. 6. **Validate** with `astra validate astra.yaml` after adding recipes. diff --git a/claude/lightcone/skills/lc-from-paper/references/specify.md b/claude/lightcone/skills/lc-from-paper/references/specify.md index eba368dc..aa715e16 100644 --- a/claude/lightcone/skills/lc-from-paper/references/specify.md +++ b/claude/lightcone/skills/lc-from-paper/references/specify.md @@ -1,6 +1,6 @@ # SPECIFY — fill the stub `astra.yaml`, two passes per sub-analysis -Read the stub `astra.yaml` from ARCHITECT and fill in `decisions:`, `prior_insights:`, `findings:` per sub-analysis, weaving the existing narrative with `astra-anchor:` references as entries land. SPECIFY is the **first material-disagreement seam** — paper-vs-code conflicts surface here, and they're often the highest-value moments for the user to weigh in on at REVIEW. +Read the stub `astra.yaml` from ARCHITECT and fill in `decisions:`, `prior_insights:`, `findings:` per sub-analysis. SPECIFY is the **first material-disagreement seam** — paper-vs-code conflicts surface here, and they're often the highest-value moments for the user to weigh in on at REVIEW. SPECIFY is what a ralph iteration does when the workdir signals "stub `astra.yaml` present + sub-analyses' `decisions:` / `prior_insights:` / `findings:` blocks still empty." Iterations run detached in tmux; the user isn't reachable interactively, so the canonical-resolution default (code wins where paper and code disagree on a material choice) applies and disagreements are logged to CLAUDE.md's **Paper-vs-code disagreements** section plus `open-questions.md` for REVIEW close-out. @@ -10,7 +10,7 @@ Per-sub-analysis work is parallelizable when sub-analyses are independent. Each ## Inputs -- `astra.yaml` — the stub from ARCHITECT (sub-analyses, inputs, outputs, narrative; empty `decisions:` / `prior_insights:` / `findings:` blocks) +- `astra.yaml` — the stub from ARCHITECT (sub-analyses, inputs, outputs, per-analysis `description`; empty `decisions:` / `prior_insights:` / `findings:` blocks) - `constitution.md` — Goal (scope), Fidelity intent, Quality bar - `CLAUDE.md` — Rules; **Paper-vs-code disagreements** for prior-iteration entries - `work/reference/index.json` — paper-extraction's structural index: figures, tables, section outline, citations. The `citations:` block maps each cited paper's BibTeX key (Path A) or synthetic `_` key (Path B) to `{locations, citation, doi}`. SPECIFY uses this to write each `prior_insights:` placeholder's `doi:` so LITERATURE knows which paper to fetch. @@ -22,18 +22,18 @@ Per-sub-analysis work is parallelizable when sub-analyses are independent. Each ## Outputs -- `astra.yaml` — **filled form**: each sub-analysis's `decisions:` populated with decision-level `rationale:` prose plus options (the paper's choice is identified by `default:`); `findings:` populated as full `Insight` blocks with paper-anchored `evidence:` (the target paper's DOI + `quote: {exact, prefix, suffix}` + `location: {page: N}`); `prior_insights:` populated as citation **placeholders** — each a syntactically-complete `Insight` (`id`, `claim`, `created_at`, `evidence: [{id, doi}]`) whose placeholder Evidence carries the cited paper's DOI looked up from `work/reference/index.json#citations[].doi` **but no `quote:` selector yet** — LITERATURE fills those in. Each option that draws on a placeholder cites it via `Option.insights: [, ...]` (the back-reference that links options to prior_insights in the ASTRA grammar). `narrative:` keys updated to weave `astra-anchor:` references into prose as entries land. `astra validate astra.yaml` returns clean (Evidence with `doi:` and no `quote:` is structurally valid at this stage); `astra validate astra.yaml --verify-evidence` runs after LITERATURE has authored the quotes. +- `astra.yaml` — **filled form**: each sub-analysis's `decisions:` populated with decision-level `rationale:` prose plus options (the paper's choice is identified by `default:`); `findings:` populated as full `Insight` blocks with paper-anchored `evidence:` (the target paper's DOI + `quote: {exact, prefix, suffix}` + `location: {page: N}`); `prior_insights:` populated as citation **placeholders** — each a syntactically-complete `Insight` (`id`, `claim`, `created_at`, `evidence: [{id, doi}]`) whose placeholder Evidence carries the cited paper's DOI looked up from `work/reference/index.json#citations[].doi` **but no `quote:` selector yet** — LITERATURE fills those in. Each option that draws on a placeholder cites it via `Option.insights: [, ...]` (the back-reference that links options to prior_insights in the ASTRA grammar). `astra validate astra.yaml` returns clean (Evidence with `doi:` and no `quote:` is structurally valid at this stage); `astra validate astra.yaml --verify-evidence` runs after LITERATURE has authored the quotes. - `universes/baseline.yaml` — selects the paper's choices (where paper and code disagree per the canonical-resolution rule, see "Material conflicts" below) - `implementation-notes.md` — concise practical guidance for the IMPLEMENT phase: tricky algorithms, numerical gotchas, data-format quirks, things the spec can't capture. Bullets, not essays. - `targets/targets.md` — small target ledger COMPARE consumes: per output (already declared by ARCHITECT), a brief entry with type, priority, paper value, expected match criteria, and the path to the reference figure / table / metric (when applicable, copy the reference file into `targets/` so the directory is self-contained) - `CLAUDE.md` updates — append entries to **Paper-vs-code disagreements** for each material conflict surfaced - `constitution.md` updates — Open dimensions when something material warrants user ratification at REVIEW -## Substrate skills to invoke +## Prose discipline -- **`/narrative`** — narrative authoring (any of the five `narrative.{summary,inputs,methods,findings,outputs}` keys, plus decision `rationale:` fields) is owned by the narrative skill. Invoke it during the **paper pass** when authoring or extending narrative prose. The narrative skill teaches reserved entity names, the tree-path anchor grammar, the conditional-narrative requirement (which keys are required when), the five-key authoring order, paper-reproduction fidelity discipline, and the new downstream-consumer discipline (lightcone-cli#108). Do not duplicate that content. +Per-element prose lives directly on the entries SPECIFY authors: each decision's `rationale:` carries the paper's stated reasoning (or the code's, where canonical-resolution applies); findings and prior_insights carry their `claim:` and optional `notes:`. Keep the paper's hedges and qualifiers intact and don't add editorial commentary beyond what the paper supports. -Your responsibility in this phase is the **content**: build out the `decisions:` / `prior_insights:` / `findings:` for each sub-analysis (each with its own evidence shape — detailed below), and weave `astra-anchor:` references back into the narrative as entries land. ARCHITECT already settled the structure. +Your responsibility in this phase is the **content**: build out the `decisions:` / `prior_insights:` / `findings:` for each sub-analysis (each with its own evidence shape — detailed below). ARCHITECT already settled the structure and the orienting `description:` blocks. ## The two-pass-per-sub-analysis structure @@ -46,7 +46,7 @@ Read the paper's section(s) covering this sub-analysis. Author: 1. **`decisions:`** — every choice in this sub-analysis where a different defensible option could plausibly shift a numerical result: algorithmic methods, thresholds, statistical approaches, data selection criteria, calibration choices. Use `when`, `incompatible_with`, and `requires` constraints for non-independent decisions. For each decision, the paper-pass authors: - - **Decision-level fields:** `label:` (short human-readable name), `rationale:` (the paper's stated reasoning — use `/narrative` for the prose), `default:` (the option the paper actually selects), and `options:` (the map of option entries below). + - **Decision-level fields:** `label:` (short human-readable name), `rationale:` (the paper's stated reasoning, authored as prose), `default:` (the option the paper actually selects), and `options:` (the map of option entries below). - **Options:** the chosen option plus any sibling alternatives the paper discusses. Each option carries `label:` (required) and an optional `description:`. Per the 0.0.10 grammar, options do **not** carry their own `rationale:` or `evidence:` block — the decision's `rationale:` covers the reasoning; paper-text evidence flows through `findings:` (for the paper's own quantitative claims) or via `Option.insights` back-references into `prior_insights:` (for citation-backed support). - **Option ↔ prior_insights linkage:** when the option's support derives from cited literature, list the relevant `prior_insights:` ids in `Option.insights: [, ...]`. The placeholder block under `prior_insights:` (authored in step 2 below) is the back-end of this link — LITERATURE fills in the verbatim cited-paper quote later. **Scope rules** (astra-tools ≥ 0.2.9): bare ids resolve **node-locally only** — the prior_insight must be declared in the same sub-analysis as the option. For a citation declared at an ancestor scope, use explicit upward refs: `[../id]` for the parent, `[../../id]` for the grandparent, etc. (same `../` grammar as `Input.from` and `Decision.from`). The natural shape — declare each cited paper at the sub-analysis that uses it, reference with a bare id from same-scope options — keeps everything node-local and needs no `../`. @@ -56,7 +56,7 @@ Read the paper's section(s) covering this sub-analysis. Author: decisions: : label: "" - rationale: "" + rationale: "" default: options: : @@ -101,9 +101,7 @@ Read the paper's section(s) covering this sub-analysis. Author: location: { page: } ``` -4. **Weave `astra-anchor:` references into the existing narrative.** ARCHITECT wrote `narrative:` prose without anchors because the entries didn't exist. Now they do — extend the narrative to point at the new `decisions:` / `prior_insights:` / `findings:` entries via the tree-path anchor grammar. Use `/narrative` for this pass; it carries the discipline. - -5. **Verify finding quotes against the paper source by Grep.** For each `findings:` Evidence entry with a `quote:`, Grep the paper source to confirm the `exact:` text is verbatim and the `prefix:` / `suffix:` are real surrounding text. `astra validate --verify-evidence` will run the deterministic check across every quote later (after LITERATURE resolves the `prior_insights:` placeholders); a manual Grep now catches typos and paraphrases before the code pass. +4. **Verify finding quotes against the paper source by Grep.** For each `findings:` Evidence entry with a `quote:`, Grep the paper source to confirm the `exact:` text is verbatim and the `prefix:` / `suffix:` are real surrounding text. `astra validate --verify-evidence` will run the deterministic check across every quote later (after LITERATURE resolves the `prior_insights:` placeholders); a manual Grep now catches typos and paraphrases before the code pass. ### Pass B — code pass (when `work/reference/code/` exists) @@ -132,9 +130,8 @@ The cross-check questions on entry: are the decisions covering everything materi 3. **Evidence verification.** Every `findings:` Evidence entry uses `TextQuoteSelector` with a verbatim `exact:` quote, real surrounding-text `prefix:` / `suffix:`, and a `location: {page: N}` (1-indexed). Quotes that are paraphrased or whose `prefix:` / `suffix:` are editorial parentheticals will fail `--verify-evidence`. `prior_insights:` placeholders intentionally have `evidence: [{id, doi}]` without a `quote:` at this stage — LITERATURE authors the quotes — so do not flag a missing quote on placeholder entries. After LITERATURE resolves the placeholders, run `astra validate astra.yaml --verify-evidence`. 4. **Findings traceability.** Each `findings:` Insight's `evidence:` resolves either to a real paper claim (target-paper DOI + verbatim `quote:` + page) or to a real declared output via `artifact: ` (with optional `source_commit:` and `snapshot:`). 5. **Material-disagreement surfacing.** Where paper and code disagree on a material choice, the spec records both options under the relevant `decisions:` entry, `universes/baseline.yaml` selects the code's option (canonical-resolution default), and the conflict is appended to CLAUDE.md's *Paper-vs-code disagreements* section plus `open-questions.md` for the user to resolve at REVIEW close-out. Flag any material disagreement that got silently dropped, that didn't make it into the disagreements log, or where the baseline picked the paper without the canonical-resolution rule applying. -6. **Narrative anchors.** The sub-analysis's `narrative:` weaves `astra-anchor:` references to the new `decisions:` / `prior_insights:` / `findings:` entries — the tree-path grammar must be valid, and entries actually exist at the referenced paths. -7. **`narrative:` voice fidelity.** Hedges and qualifiers from the paper survive (per the narrative skill's discipline). Editorial commentary added beyond what the paper supports gets flagged. -8. **No synthetic data.** Unless the paper itself uses synthetic data, every input has a real acquisition source — no mock / synthetic substitutes anywhere in the sub-analysis's inputs, decisions, or implementation-notes. +6. **Prose voice fidelity.** Hedges and qualifiers from the paper survive in the decisions' `rationale:` and the findings' `claim:` / `notes:`. Editorial commentary added beyond what the paper supports gets flagged. +7. **No synthetic data.** Unless the paper itself uses synthetic data, every input has a real acquisition source — no mock / synthetic substitutes anywhere in the sub-analysis's inputs, decisions, or implementation-notes. Apply fixes inline as you find them — `astra.yaml`, `universes/baseline.yaml`, `implementation-notes.md`, the disagreements log in CLAUDE.md as needed. The diff against the prior commit is the record of what changed. After any change to `astra.yaml`: @@ -175,11 +172,11 @@ Out-of-scope targets stay in `targets/targets.md` with an explicit reason and sh - **Equation and section numbers must match the rendered paper / PDF**, not a naïve count of TeX blocks or markdown headings. When citing "eq. N" or "§N", find the equation or heading by content in the rendered paper and use the printed number. - **Validate** with `astra validate astra.yaml` after each pass. - **Targeted reads, not whole-paper absorption.** Use `work/reference/index.json` and `work/reference/code-index.md` for structural lookups; Grep into `work/reference/source/` (Path A) or `work/reference/document.md` (Path B) for specific verbatim quotes; read targeted code modules under `work/reference/code/` for canonical method details. Don't re-read the whole paper or whole code base. -- **The narrative skill is the prose author, not the structure author.** SPECIFY weaves anchors into the prose ARCHITECT wrote — the structural surface is fixed, the anchored references are SPECIFY's contribution. +- **Content correctness, not structure.** ARCHITECT settled the structure and the orienting `description:` blocks; SPECIFY's contribution is the `decisions:` / `prior_insights:` / `findings:` content and the prose on those entries. ## Survey signals (entry into SPECIFY) -- `astra.yaml` exists with stub form (sub-analyses + inputs + outputs + narrative; empty decisions / prior_insights / findings) ⇒ ready to specify +- `astra.yaml` exists with stub form (sub-analyses + inputs + outputs + per-analysis `description`; empty decisions / prior_insights / findings) ⇒ ready to specify - For each sub-analysis: `decisions:` populated with decision-level `rationale:` + options (paper's choice at `default:`); `findings:` populated as full Insight blocks with paper-anchored Evidence (DOI + `quote: {exact, prefix, suffix}` + `location: {page}`); `prior_insights:` populated as citation placeholders (`id`, `claim`, `created_at`, `evidence: [{id, doi}]` with `quote:` omitted — LITERATURE fills the quotes next); `Option.insights` back-references wired up where options draw on placeholders ⇒ paper pass done - For each sub-analysis: when `work/reference/code/` exists, code-pass material-disagreement entries land in `decisions:` (with both options) and `universes/baseline.yaml` selects the canonical-resolution choice; `implementation-notes.md` carries non-material gotchas ⇒ code pass done - For each sub-analysis: a fresh-context iteration reads the slice and finds nothing to fix ⇒ that sub-analysis is done; the next iteration moves on @@ -191,7 +188,7 @@ Out-of-scope targets stay in `targets/targets.md` with an explicit reason and sh ## Notes - **Material disagreements** are appended to CLAUDE.md's **Paper-vs-code disagreements** section AND `open-questions.md`. CLAUDE.md is the at-a-glance summary every iteration sees; `open-questions.md` is the user-resolution accumulator. Both lead to the same place: the user resolves at REVIEW close-out. -- **The narrative skill is the prose author, not the structure author.** SPECIFY's job is content correctness; `/narrative` invocation comes during the paper pass when authoring or extending the narrative prose to weave in anchor references. +- **SPECIFY owns content, not structure.** Its job is content correctness — the `decisions:` / `prior_insights:` / `findings:` and the `rationale:` / `claim:` / `notes:` prose on them. The orienting `description:` blocks were ARCHITECT's; the structural surface is fixed. - **The target ledger is a derivation, not a separate phase's output.** Treat `targets/targets.md` as a small index produced alongside the filled `astra.yaml`, not a heavyweight artifact. The depth lives in `astra.yaml`'s `outputs:` / `findings:` / `decisions:`. - **Two-pass discipline is the cross-check.** Skipping the code pass (when code exists) loses the canonical-resolution surface and lets paper-vs-code material disagreements slip through. The fresh-context review can recover *some* of these but not all — the disciplined sequence (paper → code → review) catches more. - **Per-sub-analysis parallelism is opt-in.** When sub-analyses are independent (no shared decision blocks, no cross-sub-analysis findings), the iteration can fan out one-level-deep sub-agents (one per sub-analysis from inside its main session) to run their passes in parallel. When they share material decisions or findings (rare), serialize across iterations. diff --git a/claude/lightcone/skills/lc-new/SKILL.md b/claude/lightcone/skills/lc-new/SKILL.md index c2db4c63..4cac53cc 100644 --- a/claude/lightcone/skills/lc-new/SKILL.md +++ b/claude/lightcone/skills/lc-new/SKILL.md @@ -25,7 +25,7 @@ Then sharpen: - "What would a clear answer look like?" (sharpens the description) - "Why does this matter?" (context for decisions) -**Update astra.yaml** — set `name`. (`astra init` scaffolded a placeholder example decision/input/output plus a TODO `narrative` skeleton; the placeholder structure is replaced in Phases 2–3, and narrative prose is filled in Finalize once structure has settled — written too early it goes stale.) +**Update astra.yaml** — set `name`. (`astra init` scaffolded a placeholder example decision/input/output plus a TODO `description`; the placeholder structure is replaced in Phases 2–3, and the `description` is filled in Finalize once structure has settled — written too early it goes stale.) --- @@ -115,9 +115,9 @@ astra universe generate -n baseline Generate only `baseline` unless the user explicitly asks for additional universes. -### Populate Narrative +### Populate Description -Replace the TODO entries in `astra.yaml`'s `narrative:` block now that structure is stable: `summary` (one-paragraph framing), `methods` (decisions and sub-analyses), `inputs`, `outputs`. Use `#path.to.element` anchors for cross-references. Leave `findings` as TODO until results exist. +Replace the TODO `description:` in `astra.yaml` with a short one-or-two-paragraph orientation now that structure is stable — what the analysis is and how its pieces fit together. Keep it brief; per-element prose lives on each Input/Output/Decision/Option via `description`/`rationale`. ### Populate CLAUDE.md diff --git a/claude/lightcone/skills/narrative/SKILL.md b/claude/lightcone/skills/narrative/SKILL.md deleted file mode 100644 index 4cb3c5d7..00000000 --- a/claude/lightcone/skills/narrative/SKILL.md +++ /dev/null @@ -1,228 +0,0 @@ ---- -name: narrative -description: > - Authors prose throughout an `astra.yaml` — analysis-level - `narrative:` blocks (five fixed keys: `summary`, `findings`, - `methods`, `inputs`, `outputs`), decision `rationale:` fields, and - shorter `description:` / `notes:` prose on individual entities. The - five-key narrative is the most substantive case; the same - architectural and syntactical frame applies wherever prose appears - in the spec. - Always written against an existing `astra.yaml`; what differs - between modes is the second source paired with the spec — an - authoritative text (paper reproduction), project artifacts - (retrofit), or dialogue with the user (co-drafting). Triggers on - "narrative", "draft the narrative", "narrate this analysis", - "rationale for this decision", "write the summary", "describe this - input", or any request for reader-facing prose keyed off an - `astra.yaml`. ---- - -# narrative - -This skill covers prose authoring across an `astra.yaml`. The prose surfaces are: - -- **Analysis `narrative:` blocks** — five keys (`summary`, `inputs`, `methods`, `findings`, `outputs`) on each analysis and sub-analysis. -- **Decision `rationale:` fields** — one paragraph per decision. -- **Per-entity prose** — shorter `description:` / `notes:` on individual inputs, outputs, options, insights. - -ASTRA's structural content surfaces alongside the prose in renderers like lightcone-ui. **Prose does not duplicate the structure** — it cites into it. An anchor is a citation; a sentence pointing to a decision is a small argument; prose is the layer where decisions, sub-analyses, findings, and outputs become a connected story. - -## Modes - -Prose cites the spec's structure (decisions, findings, outputs, sub-analyses) by anchor, so the structure must exist when the prose lands: write the spec first, write both concurrently, or revise narrative after spec changes settle. - -There are three modes, distinguished by what's available beyond the spec itself. Every mode draws on the under-construction `astra.yaml`; what differs is the **second source** paired with it. - -| Mode | Second source | Status | Reference | -|---|---|---|---| -| **Paper reproduction** | An authoritative text source (paper, thesis, technical report, …) | Ready | [`references/paper-reproduction.md`](references/paper-reproduction.md) | -| **Retrofit** | Project artifacts — code, notebooks, fibers, commit history | Stub | [`references/existing-analysis.md`](references/existing-analysis.md) | -| **Co-drafting** | The user, in conversation | Stub | [`references/co-drafting.md`](references/co-drafting.md) | - -If the second source isn't obvious from context, ask: is there an authoritative text (paper, thesis, technical report) to draw from? If not, are we harvesting from existing artifacts, or working from the user's own framing? Hybrid is allowed — a reproduction with co-drafted extensions, a retrofit with co-drafted gap-filling. - -The rest of this file is the mode-independent substrate every reference relies on. Read it through, then open the matching reference. - ---- - -## The five keys - -| Key | What it carries | Required when | -|---|---|---| -| `summary` | Question, scope, headline shape — the only key without a structural peer. | optional in the schema, but should always exist | -| `inputs` | Provenance — the data the analysis rests on. | `Analysis.inputs` is non-empty | -| `methods` | Pipeline walk; cite each decision and sub-analysis by anchor. | `Analysis.decisions` or `Analysis.analyses` is non-empty | -| `findings` | Synthesis of declared findings; each cited by anchor. | `Analysis.findings` is non-empty | -| `outputs` | Which artifacts were promoted, and where they go downstream. | `Analysis.outputs` is non-empty | - -`astra validate` enforces the right column. **Narrate what you declare:** if `findings:` is empty, `narrative.findings` should not appear. A stub analysis with only `summary` is valid. - -A decision's `rationale:` is its own one-paragraph slot — what was decided, the insight that motivated it (cite by anchor), and what the load-bearing alternative was and why it lost. The alternatives themselves live in the options structure. - -## Length - -1–3 paragraphs per key, at any level (root, sub-analysis, decision). - -Length is the mechanism that keeps analyses modular, not a style preference. **If references don't fit in three paragraphs, the analysis is too big — split it.** The narrative is a compressor; if it won't compress, split the thing being compressed. - -## Anchors - -Markdown link syntax with `#`-target, **tree-path-first** — same grammar as decision `from:` references. - -| Target | Anchor | -|---|---| -| Input | `#inputs.` | -| Output | `#outputs.` | -| Decision | `#decisions.` | -| Option within a decision | `#decisions..options.` | -| Finding | `#findings.` | -| Prior insight | `#prior_insights.` | -| Sub-analysis (whole node) | `#analyses.` | -| Element inside sub-analysis | `#..` | -| Parent scope (from a sub-analysis) | `#../decisions.` | - -The sub-analysis form is **sub-analysis first, then category**: `#reconstruction.decisions.algorithm`, not `#decisions.reconstruction.algorithm`. References resolve relative to the hosting analysis; use `../` to escape to parent scope. - -Rules: - -- Anchor text is **authored prose**, not the raw id. -- Inline references do the work of a citation; don't footnote or parenthesize. -- One reference per idea. Stacking three on a sentence means the sentence carries too much. -- Prior insights motivate decision options via `decisions..options..insights:`. Findings cannot appear there (validator-enforced); if a finding motivates a decision, cite it from the decision's `rationale:` prose. - -### Reserved IDs - -These names cannot be used as entity IDs (they collide with the anchor grammar): `inputs`, `outputs`, `decisions`, `findings`, `prior_insights`, `analyses`, `options`, `content`, `narrative`. The validator rejects them. - -## Data flow - -Make the data-flow linkage navigable in the prose itself. Anchors are the trail — a reader follows the flow inline, without leaving the narrative. - -1. **`narrative.outputs` says where each output goes next.** A sub-analysis's outputs are usually consumed by other sub-analyses or roll up into root findings. When you write the `outputs` prose, name those downstream destinations by anchor. Example, in the `reconstruction` sub-analysis's `outputs` key: - - > *"`xi_post_recon_lrg1` feeds [the post-reconstruction BAO fit](#analyses.bao_fit.outputs.bao_fit_post_iso_ap_lrg1) and supports the [headline detection finding](#findings.bao_detection_chi2_lrg1)."* - - Anchor downstream consumers where you can. When no anchor is reachable from the current scope (typically a sibling sub-analysis), bare `.` text is acceptable. - -2. **The root narrative is the end-to-end view.** When the project has sub-analyses, the root analysis's `methods` (or `summary`) traces the pipeline from raw inputs to final outputs — as much overview as fits in a few paragraphs. The root is the place a reader can land cold and get the shape of the work; details telescope into the sub-analyses. A condensed example: - - > *"raw catalogs → [reconstruction](#analyses.reconstruction) → [clustering](#analyses.clustering) → root [BAO fit](#outputs.bao_fit_post_iso_ap_lrg1)."* - -## Validation - -```sh -astra validate astra.yaml -``` - -- **Broken references** → error. Anchor doesn't resolve to a real id. -- **Uncited declared elements** → warning. Every declared finding, decision, output, and sub-analysis must be cited somewhere in the narrative tree. If an element genuinely isn't worth a prose mention, consider whether it should be declared at all. -- **Conditional coverage** → error. The required-when rule above. - -## User presence - -Multi-turn back-and-forth → user present; use `AskUserQuestion` to clarify mode, scale, and any mode-specific framing before drafting. Single-shot or pipeline invocation → autonomous; make the reasonable default inference and note it inline on the narrative. Ambiguous → err on present and ask. - ---- - -## Craft - -- **Economy.** Every sentence introduces a new idea or sharpens an existing one. Release real verbs: `conducted cross-correlation` → `cross-correlated`. -- **Anchor text is prose, not an id.** `[the post-reconstruction catalogs](#analyses.reconstruction)`, not `[reconstruction](#analyses.reconstruction)`. -- **One reference per idea.** Three anchors on one sentence means the sentence carries too much; split it or drop one. -- **Specificity.** Names, numbers, references over generic claims. -- **Arrive through content.** No "in this analysis we will describe…"; the content is the opening. - -### Real subjects, real verbs - -"We measure the BAO peak with the LRG sample" reads as agency. "The measurements of the BAO peak reveal a 7σ detection" reads as zombie-noun abstraction. The test: can you picture someone or something physically doing the verb? If not, rewrite. - -Valid subjects: - -- **We** — for decisions and actions ("we chose the Gaussian damping prior") -- **The thing itself** — for states and properties ("the covariance is dominated by shot noise") -- **Passive voice** — when the actor is obvious ("a redshift cut is applied") -- **Results / data as epistemic subjects** — for what the data shows ("the measurement shows a 7σ peak"; "Figure 2 reveals…") -- **Physics doing physics** — for physical processes ("lensing distorts shapes"; "higher-order effects produce B-modes") - -Anthropomorphized abstractions fail the test: "the methodology validates," "this analysis demonstrates," "the catalogue evolution follows." Rewrite to a real subject doing a real verb. - -## Anti-patterns (mode-independent) - -- **Wiki-style what-is framing.** "BAO is the baryon acoustic oscillation feature." A wiki summarizes; an ASTRA narrative points into reasoning. Replace with the load-bearing statement and an anchor: "we chose the Gaussian BAO damping prior over flat because flat admitted spurious minima — see [the prior comparison](#decisions.bao_damping_prior)." -- **Decision-list paragraph.** "We made the following decisions: A, B, C." Cite each decision where it shapes the pipeline, not as recitation. Too many to weave coherently → the spec wants more sub-analyses. -- **`summary` as primer.** Teaching what the field is. Readers arrive with context. -- **Drafting `findings` on a sub-analysis with no declared findings.** Skip the key. -- **Narrative-per-element.** Writing `narrative:` on findings, inputs, outputs, or insights. The five-key analysis narrative is the only home; per-element prose is `description` / `rationale` / `notes`. - -Mode-specific anti-patterns live in each mode's reference. - ---- - -## Self-contained example - -A minimal (not necessarily valid) sketch showing how the blocks fit together. The point is the *shape*. - -```yaml -id: example_analysis -version: "0.1.0" -name: "Example analysis" - -narrative: - summary: | - We measure in . The feature is - [detected at high significance](#findings.headline_detection) and - [exceeds prior precision by 1.2×](#findings.precision_improvement), - with [an anomalous feature at ](#findings.anomaly) - motivating follow-up. - - inputs: | - Primary data are [the ](#inputs.primary_data); validation - uses [](#inputs.validation_mocks). - - methods: | - The pipeline runs in two stages. [Preparation](#analyses.preparation) - ingests the raw catalog and produces [cleaned two-point statistics - ](#preparation.outputs.clean_stats). [Fitting](#analyses.fitting) - consumes those statistics and fits model parameters. Both stages - inherit the parent's [fiducial cosmology](#decisions.fiducial_cosmology) - so the distance-redshift relation is used end-to-end. - - findings: | - Three findings constitute the result: a - [headline detection](#findings.headline_detection), a - [precision comparison with prior work](#findings.precision_improvement), - and [an anomalous feature](#findings.anomaly). The anomaly is the - most-discussed qualitative feature. - - outputs: | - Two artifacts are promoted to the top level: - [the final measurement table](#outputs.final_table) and - [the headline figure](#outputs.headline_figure), both produced by - [fitting](#analyses.fitting). - -decisions: - fiducial_cosmology: - label: "Fiducial cosmology" - rationale: | - Planck 2018-ΛCDM is the community reference; distance-redshift - conversion is downstream of this choice, and fixing it lets - results be compared directly to prior measurements. Inherited by - [fitting](#analyses.fitting) so the end-to-end chain uses one - distance scale. - default: planck2018 - options: - planck2018: - label: "Planck 2018-ΛCDM" - wmap9: - label: "WMAP9" - excluded_reason: "Superseded; no longer the community reference." -``` - -For a canonical reproduction narrative in context, see `Reproductions/DESI/desi-dr1-bao/astra.yaml` in the [LightconeResearch/Reproductions](https://github.com/LightconeResearch/Reproductions) repo. - ---- - -## Now read the mode reference - -Open the reference file that matches the user's situation. Each carries the mode's draft order, mode-specific moves, critique pass, and mode-specific anti-patterns. diff --git a/claude/lightcone/skills/narrative/references/co-drafting.md b/claude/lightcone/skills/narrative/references/co-drafting.md deleted file mode 100644 index 1deee5b6..00000000 --- a/claude/lightcone/skills/narrative/references/co-drafting.md +++ /dev/null @@ -1,79 +0,0 @@ -# Co-drafting mode (stub) - -> **Status: under development.** Use paper reproduction (the default flow when a paper exists) when applicable. This file names what's distinct about co-drafting and the open questions; it isn't yet production guidance. - -The narrative is being drafted in dialogue with the user, against an existing-shape `astra.yaml`. There's no paper to harvest from and no body of code or fibers to mine; the spec's structure is the only artifact, and the user is the source for everything the structure doesn't already carry. - -This mode covers a spectrum: - -- **Fresh scoping.** `astra.yaml` was scaffolded by `/lc-new` (or by hand); decisions and outputs are sketched but the analysis hasn't run. Narrative drafted against intent, not results. -- **Live in-flight research.** Work is happening; data is coming in, decisions are settling, results are landing. Spec moves between conversations, narrative moves with it. -- **Newly-stable analysis.** Work has finished or paused; the user wants to write a narrative for what they did. No paper, no fibers — they remember it, and that memory is the source. - -Pure greenfield (no `astra.yaml` at all) isn't a coherent narrative-skill task — there's nothing to cite into. If a user is at that stage, route them to `/lc-new` to scaffold structure first. - -## What's distinct from paper reproduction - -- **Source is conversation, not prose.** The paper-reproduction harvest move (paraphrase from a written source) doesn't apply. Draft moves come from dialogue with the user — `AskUserQuestion` when several framing questions land together, prose follow-ups when one question opens the next. -- **Voice depends on stage.** Reproduction is always declarative ("The pipeline runs in…"). Co-drafting voice tracks where the work is: present-tense for live work, past tense for completed steps, provisional markers when content is volatile. -- **Spec and narrative move together.** In reproduction the spec is fixed (or close to it) and the narrative reconstructs the paper. In co-drafting the spec may shift between drafts; expect to revisit narrative when a decision lands or a sub-analysis splits. - -## The ask-first discipline - -Co-drafting is the one mode where authoring without asking produces fiction. The user is available; ask. Surface the load-bearing reads before drafting — `AskUserQuestion` when several land together, single questions or prose follow-ups when the conversation wants its own rhythm: - -- **Research question.** What are you trying to learn? One sentence. -- **Current headline finding** (if any). What's been established so far? One sentence; a gesture is fine. -- **Movement so far.** What pivots, abandoned options, surprises belong in the record? -- **Implications.** What would you claim today about what this means? Premature strong claims aren't required; honest gestures are. - -The user's framing is the substrate. Don't draft around a guess at it. - -## Provisional voice - -When content is moving, make incompleteness visible. Three moves: - -**Phrasing carries confidence.** Not "we constrain X to 3%"; rather "our current best constraint on X is 3%, pending validation of the covariance in [reconstruction](#analyses.reconstruction)." Hedge what's uncertain; claim what's settled. - -**Explicit markers.** At the top of `summary` (or any volatile key), an italic note: - -```yaml -summary: | - _(Provisional — revisit after bao_fitting. Last updated 2026-04-23.)_ - We are measuring the BAO scale... -``` - -The `_(Provisional)_` prefix is a convention, not a spec field. It reads as expected-to-change without breaking the narrative shape. - -**Decision rationales can be open.** "We are currently running with option X, pending validation of Y. See [[fiber-or-sub-analysis]]." A `rationale:` doesn't have to be retrospective. - -When work stabilizes (a paper draft lands, results publish), revise into reproduction voice — past tense, declarative, scope clear. Co-drafting was scaffolding; the final narrative reads as a stable artifact. - -## Open questions before this is production-ready - -- **Provisional markers — convention or schema?** Today they're prose conventions (`_(Provisional)_`); whether they belong as structured metadata is open. -- **What's a `tempered`-style flag for narrative?** `tempered: true` on fibers signals "solid enough to build on." A narrative-level analog could let renderers display freshness state. -- **Anchor coverage for elements that don't exist yet.** "Once [reconstruction](#analyses.reconstruction) is run, we expect X." The validator currently requires anchors to resolve — co-drafting may need a "planned" sub-analysis form, or the prose may need to avoid forward-anchoring entirely. -- **Boundary with `/lc-new`.** `/lc-new` does conversational scoping but defers narrative ("filled in later, once structural pieces have settled"). When does the user finish `/lc-new` and switch to `/narrative` for the prose pass? Unclear today. -- **Boundary with retrofit.** A user co-drafting a narrative for completed work is reaching for the same artifacts retrofit mines. The line between "harvest from your own memory" (co-drafting) and "harvest from artifacts you produced" (retrofit) is fuzzy when the user is the artifacts' author. - -## Pointers when authoring today - -The substrate from SKILL.md applies in full: five keys, length cap, anchor grammar, reserved IDs, data flow, validation, craft. What changes is the *source* of content (dialogue) and the *voice* (provisional where moving). - -- Use first-person plural and present tense for live work; past tense for completed steps. -- Hedge when uncertain; claim when confident. Over-hedging is its own failure mode. -- Mark sub-analyses that don't exist yet with provisional language rather than fake anchors. -- Inverted draft order can help: write `summary` first as a stub (to fix intent), then draft the rest, then return to `summary` last to revise. This is the opposite of reproduction's compress-last because the substrate is moving. - -## Anti-patterns (co-drafting-specific) - -- **Solo drafting.** The user is available; ask before guessing motivation, headline finding, or implications. -- **False completeness.** Writing in reproduction voice ("we measure," "we constrain") when the measurement is in flight. Use "we are measuring" / "our current constraint is X, pending Y." -- **Provisional everywhere.** If every sentence is hedged, the narrative reads as afraid of itself. Hedge the genuinely uncertain claims; state the settled ones plainly. -- **Stale markers.** A "revisit after X" comment left in place after X has landed is worse than no marker at all. Revise on each touch. -- **Over-committing to implications.** Promising what results will mean before they land. A gesture is honest; a claim before evidence is not. - -## Report friction - -If you hit co-drafting cases this stub doesn't cover, file a fiber or GitHub issue against `lightcone-cli` with `narrative` in the title so the next pass can firm this up. diff --git a/claude/lightcone/skills/narrative/references/existing-analysis.md b/claude/lightcone/skills/narrative/references/existing-analysis.md deleted file mode 100644 index a28cba29..00000000 --- a/claude/lightcone/skills/narrative/references/existing-analysis.md +++ /dev/null @@ -1,50 +0,0 @@ -# Existing-analysis retrofit mode (stub) - -> **Status: under development.** Use paper reproduction (the default flow when a paper exists) when applicable. This file names what's distinct about retrofit and the open questions; it isn't yet production guidance. - -A project has been running — code, results, partial spec, no published paper — and is being imported into ASTRA. The `astra.yaml` has been built (or is being built); the narrative is reconstructed from the artifacts that produced the work, not from a written source. Retrofit is **harvest from artifacts**; co-drafting is **harvest from conversation**; reproduction is **harvest from a paper**. - -## What's distinct from paper reproduction - -- **No source narrative.** The five-key shape has to be assembled from - what the artifacts carry: README, CLAUDE.md, fibers, notebook cells, - code comments, commit messages, meeting notes, old proposals, issues, - closed PRs. -- **Triage comes first.** Sub-analyses and decisions classify as live / - superseded / abandoned / unclear. The narrative speaks for live content - by default; abandoned and superseded only appear if the user wants - history surfaced. -- **Gaps are explicit.** When a decision's original rationale isn't - recoverable from artifacts and the user can't reconstruct it, the - honest move is to say so — `_(Reconstructed YYYY-MM: original rationale - not recorded.)_` — not to fabricate a plausible justification. -- **Past tense for what happened.** Present tense only for living - structure ("the pipeline runs three stages"). - -## Open questions before this is production-ready - -- **What's the canonical artifact harvest?** README, fibers, notebooks, - commits, PR threads — order, depth, when to stop. Real retrofit cases - will vary widely; the skill needs a default ordering and the criteria - for going deeper. -- **How aggressive is `AskUserQuestion`?** A retrofit on a year-old - project may have a researcher who remembers some decisions but not - others. Where's the line between asking and reconstructing? -- **History sections.** When abandoned options are load-bearing - ("we tried X for six months, switched to Y"), they belong in - movement-of-learning. Routing: new sub-analysis with `excluded:` / - `lifecycle: abandoned`? Inline marker in `methods`? No firm answer. -- **Voice for reconstructed content.** `_(Reconstructed)_` works - inline. Whether reconstructed-vs-original needs structural distinction - in the spec, or stays a prose convention, is open. - -## When retrofit shifts modes - -- **Becomes reproduction.** If the project is reproducing an unacknowledged paper, switch to the default flow for the parts that map. Hybrid is fine. -- **Becomes co-drafting.** If retrofit surfaces that core decisions are still open and the user wants to revisit them now, switch to co-drafting mode for those sections (provisional voice, revisit after decisions land). - -## Report friction - -If you hit retrofit cases this stub doesn't cover, file a fiber or -GitHub issue against `lightcone-cli` with `narrative` in the title so -the next pass can firm this up. diff --git a/claude/lightcone/skills/narrative/references/paper-reproduction.md b/claude/lightcone/skills/narrative/references/paper-reproduction.md deleted file mode 100644 index a5fa9d1f..00000000 --- a/claude/lightcone/skills/narrative/references/paper-reproduction.md +++ /dev/null @@ -1,118 +0,0 @@ -# Paper reproduction mode - -An authoritative text source exists — most often a published paper, but also a thesis, technical report, posted preprint, or other canonical account of the work. Reconstruct its narrative into ASTRA's five-key shape, drawing on **the text and the under-construction `astra.yaml` as paired sources**: the text carries the claims and the confidence register; the spec carries the structural decomposition (which decisions are nodes, which findings are nodes, where sub-analyses sit). Neither is sufficient alone. - -The spec may be stable, in flux, or both — paper-reproduction often runs concurrently with spec refinement. The narrative tracks both: when a decision is added, write its `rationale:`; when a sub-analysis splits, draft its five keys; when a finding is declared, fold it into the parent's `findings` synthesis. - -Read the main SKILL.md first. This file adds what's specific to reproduction. - -## Where the source text lives - -The skill expects `work/reference/` to exist — the standardized output of [`/paper-extraction`](../../paper-extraction/SKILL.md). If it doesn't, run `/paper-extraction` first. The predictable shape: - -- `work/reference/paper.tex` (Path A — symlink to main `.tex`) **or** `work/reference/document.md` (Path B — Docling output) -- `work/reference/index.json` — section outline with line numbers, figures, tables, citation locations -- `work/reference/astra.yaml` — the paper as an ASTRA artifact (claimed findings as ASTRA findings) -- `work/reference/figures/`, `work/reference/tables/`, `work/reference/source/` (Path A only) - -If no authoritative text is accessible at all, this isn't reproduction — fall back to `references/existing-analysis.md` or `references/co-drafting.md`. - -## Paper-to-ASTRA mapping - -Write this down before drafting a sentence. - -| Paper element | ASTRA home | -|---|---| -| Abstract | `summary` | -| Introduction (motivation, related work) | `summary` + `findings` intro | -| Methods section N | the matching sub-analysis's `narrative.methods` | -| Results | structural `findings.` claims; narrative intro in `findings` | -| Discussion | `findings` narrative + `summary` implications | -| Conclusions | reinforces `summary` | -| Figures / tables | `outputs.` — referenced in `findings` via anchors | -| "We chose X because Y" sentences | the relevant decision's `rationale:` | - -Not every text maps cleanly section-to-sub-analysis. When it doesn't, the sub-analysis DAG in `astra.yaml` is authoritative: narrate according to the DAG, harvesting the source text's prose for content. If the spec deliberately reorganized relative to the text, say so briefly in `methods`. - -## Workflow - -### 1 · Orient - -Read both sources before drafting. The spec carries the structural decomposition; the text carries the claims. - -1. **`astra.yaml` at the project root** — whole file. Note `inputs`, `outputs`, `decisions`, `findings`, `analyses`, existing `narrative:`. Notice which of the five keys are present vs. empty. -2. **Each sub-analysis `astra.yaml`** — skim decisions (inherited vs. local), findings, outputs, existing narrative. -3. **The source text** — abstract, intro open/close, methods section headers, discussion, conclusions. Read full sections when drafting the corresponding ASTRA piece. Use `work/reference/index.json` to navigate; the parsed `paper.tex` (Path A) or `document.md` (Path B) is the primary source. -4. **Project `CLAUDE.md` and any working notes** — paper-specific conventions, gotchas, scope decisions. - -If the user is present, surface the orienting questions — `AskUserQuestion` is useful when several land together; one question at a time is fine when only one is open: - -- **Scale:** top-level, a specific sub-analysis, or a decision's `rationale:`? -- **Pure reproduction, or with reproducer extensions** (e.g., the reproduction's covariance differs from the posted table)? -- **Approach:** start with a specific question first — a methods subsection, a particular figure's choices, a discussion claim worth tracing into the decisions — or one-shot the whole narrative? Sets the session shape. - -### 2 · Draft order - -Not `summary` first. `summary` compresses the rest; draft it last. - -1. **`inputs`** — shortest. Name the data and its provenance. One short paragraph. Let the inputs structure carry the dataset detail. -2. **`methods`** — walk the pipeline in DAG order. Cite each sub-analysis and decision by anchor as part of the argument, not as an enumeration. If too many to weave coherently, the analysis wants more sub-analyses. Inheritance that propagates across sub-analyses gets called out explicitly because it's load-bearing end-to-end. A pivot the paper narrates ("we initially tried X, but…") is cheap to preserve because of telescoping. -3. **`findings`** — only if findings are declared structurally. Synthesize how they relate; each cited by anchor, not enumerated. -4. **`outputs`** — thin. Which artifacts were promoted and why; cite the sub-analysis that produced them; name downstream consumers (see Data flow in SKILL.md). -5. **`summary`** — last. 1–2 paragraphs. Open with the question and the headline finding; thread motivation, method, and implications. No primer material. - -For each decision, write a one-paragraph `rationale:`: what was decided, the prior insight that motivated it (cite by anchor), what the load-bearing alternative was and why it lost. - -For sub-analyses, same order, same length target. - -**Conditional keys.** Only include keys whose structural counterpart is non-empty. A reconstruction sub-analysis with no findings gets `summary`, `methods`, `inputs`, `outputs` — no `findings`. - -### 3 · Reproduction-specific moves - -- **Tell the author's story by default.** The narrative reproduces what the paper says, restated within the ASTRA structure — anchored to what's referable in the spec (decisions, findings, prior insights). Decision rationales come from the paper's "we chose X because Y" sentences, not invented post-hoc. -- **Paraphrase, don't lift.** Restate the paper's claims in your own structuring rather than copying sentences verbatim — verbatim quotation calls authorship into question. Preserve meaning and confidence register; don't sharpen or soften (if the paper says "we detect," don't write "we strongly detect"; if it hedges, preserve the hedge). -- **Two sources, paired.** The authoritative text carries claims, confidence register, and sequence. The under-construction `astra.yaml` carries the structural decomposition. Draft against both; let the spec's structure shape what each key covers, and let the text shape what's said. -- **When the reproduction's results differ, adapt — and flag.** Where the reproduction landed on different findings (a covariance that diverges from the posted table, a coefficient with different precision, a null where the paper claimed detection), the narrative needs to report what was actually found, not what was claimed. This wants human input on phrasing; surface the divergence to the user rather than papering over it. -- **Voice seams.** When reproducer-specific content enters the narrative, mark the transition. *"During reproduction we found the published covariance differs from the posted table"* is a seam; the sentence before it can speak in the paper's voice, the sentences after it speak in the reproducer's. A sentence that silently mixes them confuses both. -- **Walk the paper's sequence in `methods`.** Traverse sub-analyses in DAG order — and the DAG order should match the paper's section order. If the spec deliberately reorganized (split one section into two sub-analyses, or merged two sections into one), name the deviation briefly in `methods`. Don't reorder silently. -- **Published = done.** Reproduction narrative is declarative, present-tense matching the paper's voice ("The analysis is organised as…", "The pipeline runs in…"). Not "we are measuring." -- **Scope-limited reproductions.** Real-world reproductions often cover a subset of the paper (e.g., DESI BAO reproducing only LRG1+LRG2). Name the scope in `summary` so a reader knows what's in and out. - -### 4 · Critique pass - -Run all four audits before declaring the narrative done. - -**Fidelity audit.** - -- Claims match the paper, **except where reproduction results actually differ.** If the reproduction landed on different findings, the narrative reports what was found — and the divergence has been surfaced to the user for phrasing input, not silently softened or sharpened. -- Voice seams marked where reproducer content enters. -- Rationales traceable to the paper's justifications or to a prior insight in the spec. -- No invented citations. Every anchor resolves to a real spec id. -- Scope (what's reproduced, what isn't) stated in `summary` if narrower than the paper. - -**Sequence audit.** - -- `methods` walks sub-analyses in DAG order; DAG order matches the paper's narrative sequence (or the deviation is named in prose). -- `summary` opens with the question, not a field primer. - -**Anchor coverage audit.** - -- `astra validate` warns on any declared finding / decision / output / sub-analysis not cited in the narrative. Review the warnings; either cite the element or consider whether it should be declared. - -**Structural-peer-redundancy audit.** - -- Citations woven into argument, not recited as a list. -- `findings` narrative synthesizes relationships between findings; `inputs` narrative names provenance. Neither catalogs fields. - -## Anti-patterns (reproduction-specific) - -- **Lifting verbatim.** Copy-pasting abstract sentences into `summary`. Paraphrase — otherwise the narrative reads as a citation of itself. -- **Adding implications the paper didn't make.** Fidelity cuts both ways. -- **Eliding the reproducer's voice entirely.** If the reproduction caught something the paper missed, name it with the seam. -- **Treating paper sections as sub-analyses.** A paper's Section 3.2 isn't automatically a sub-analysis; the DAG is the authority. -- **Listing instead of weaving.** Narrate each decision where it shapes the pipeline. Too many to weave coherently → the spec wants more sub-analyses. - -## When reproduction shifts modes - -- **Hybrid with co-drafting.** If the reproduction adds a sub-analysis the paper didn't have (a reproducer-specific extension), that sub-analysis's narrative is co-drafted, not reproduced. Use the seams. -- **Hybrid with retrofit.** If the reproduction inherits code or fibers from a prior iteration, those carry rationale that didn't make it into the paper — harvest from artifacts as in retrofit mode for those sections. diff --git a/claude/lightcone/skills/paper-extraction/SKILL.md b/claude/lightcone/skills/paper-extraction/SKILL.md index dd4dce32..65819686 100644 --- a/claude/lightcone/skills/paper-extraction/SKILL.md +++ b/claude/lightcone/skills/paper-extraction/SKILL.md @@ -98,7 +98,7 @@ The skill produces only the paper's own reading materials. Anything not containe The `citations:` block maps each cited paper's BibTeX key (Path A) or synthetic `_` key (Path B) to `{locations, citation, doi}`. Downstream consumers (e.g. lc-from-paper's SPECIFY when authoring `prior_insights:` placeholders, LITERATURE when discovering which DOIs to fetch) read the DOI directly from `citations[key].doi`. Unresolvable entries keep `citation: null` and/or `doi: null` and are flagged in `extraction_warnings`. -**`astra.yaml` is semantic and ASTRA-validating.** Treats the paper as an ASTRA artifact: `id`, `version`, `name`, `narrative.summary`, and `findings:` carrying the paper's claimed numerical results in ASTRA's Insight + Evidence shape. Read this when you want to know "what does this paper claim, with quote evidence anchored to the source." The script writes a stub (id, version, name, narrative.summary from abstract, empty findings); Step 5 fills in `findings:`. +**`astra.yaml` is semantic and ASTRA-validating.** Treats the paper as an ASTRA artifact: `id`, `version`, `name`, a top-level `description:` (from the abstract), and `findings:` carrying the paper's claimed numerical results in ASTRA's Insight + Evidence shape. Read this when you want to know "what does this paper claim, with quote evidence anchored to the source." The script writes a stub (id, version, name, `description` from abstract, empty findings); Step 5 fills in `findings:`. Why both: the structural index is queryable by any consumer (`grep`, `jq`, agent code) without needing to know about ASTRA. The ASTRA file composes directly into reproductions, MySTRA, and any other ASTRA-aware tool — and the verbosity of the Insight + Evidence shape *is* the back-pressure against hallucinated numerical claims (the agent has to find and quote the actual text). @@ -113,7 +113,7 @@ Always start with `ls work/reference/` and read `index.json` if present. Skip th | `source/` (Path A) or `document.md` (Path B) + `paper.pdf` | Substrate acquired (Step 2) | | `index.json` with non-empty figures/tables/outline | Structural extraction done (Step 3) | | `astra.yaml` exists | Stub written; never overwritten on re-run (preserves agent edits) | -| `astra.yaml` has non-empty `findings:` and `narrative.findings:` populated | Findings step done (Step 5, optional) | +| `astra.yaml` has non-empty `findings:` populated | Findings step done (Step 5, optional) | If nothing is present, run the full workflow. @@ -142,7 +142,7 @@ The script detects the path automatically and produces: - `tables/.tex` — one file per `\begin{table}` block (Path A only) - `bibliography-source.{bib,bbl}` if present in the source tarball (Path A only) - `index.json` — the unified structural index, including the enriched `citations:` block (each cited key carries `{locations, citation, doi}`; DOI resolution covers ~96% of typical-paper bibliographies) -- `astra.yaml` — stub ASTRA representation: id, version, name (from `\title{}`), narrative.summary (from abstract), empty `findings: {}` for Step 5 +- `astra.yaml` — stub ASTRA representation: id, version, name (from `\title{}`), `description` (from abstract), empty `findings: {}` for Step 5 - `.doi-cache.json` — Crossref/ADS lookup cache; re-runs skip the network for already-seen entries The `--arxiv-id` / `--doi` argument populates the `id` and the evidence `doi:` field in `astra.yaml`. If neither is provided, the script writes placeholder text the agent can fix. @@ -160,7 +160,7 @@ The script is purely deterministic. It walks the structural surface but does not - **`citation : cited in source but no matching entry in bibliography-source.{bib,bbl}`** — a `\cite{}` invocation has no corresponding bib record. Usually a typo in the LaTeX source; flag it and move on. The entry stays in `citations:` with `citation: null, doi: null`, locations preserved. - **Path B caveat** — outline extraction is not yet implemented for the Docling fallback. Bibliography resolution works on Path B by parsing the references section at the tail of `document.md` and synthesizing keys (`_`), but citation *invocations* from rendered prose aren't yet extracted — Path B citations carry empty `locations: []`. The warnings list flags this. -Also eyeball `astra.yaml`'s `name:` and `narrative.summary:`. The title or abstract may contain unresolved custom `\newcommand` macros (defined elsewhere in the source); the script doesn't expand macros, so they pass through verbatim. Clean them up if you need pretty rendering downstream — none of this blocks validation. +Also eyeball `astra.yaml`'s `name:` and `description:`. The title or abstract may contain unresolved custom `\newcommand` macros (defined elsewhere in the source); the script doesn't expand macros, so they pass through verbatim. Clean them up if you need pretty rendering downstream — none of this blocks validation. ### Step 5 — *(Optional)* Walk the paper for findings, append to `astra.yaml` @@ -182,10 +182,6 @@ findings: exact: "we find $S_8 = 0.795 \\pm 0.014$" ``` -When `findings:` is non-empty, `narrative.findings:` must reference at least one finding — e.g. `narrative: { findings: "The fiducial analysis yields the [S_8 constraint](#findings.s8_constraint)." }`. - -See `examples/unions-bmodes-astra.yaml` for a fully populated `astra.yaml` (six findings, narrative, evidence anchored to the published version). - **Discipline:** - **Read the abstract and conclusions first.** Most central findings can be quoted from one of those two surfaces. @@ -208,7 +204,7 @@ The slash-command form is `/paper-extraction `. **Script (`extract-paper-substrate.py`):** walks LaTeX (Path A) or Docling output (Path B) and emits two things: 1. `index.json` — figures (with copied files + line numbers + multi-graphic panels), tables (one `.tex` per block, including AAS `deluxetable`), section outline (with line numbers, in paper-reading order), citation keys (with every file+line they appear on, including biblatex commands, *plus the cited paper's full citation text and resolved DOI*), abstract, title, paths. -2. `astra.yaml` — a stub ASTRA artifact: `id` (derived from arxiv-id/DOI), `version`, `name` (from `\title{}`), `narrative.summary` (from abstract), empty `inputs:`/`outputs:`/`findings:`. Validates as-is. +2. `astra.yaml` — a stub ASTRA artifact: `id` (derived from arxiv-id/DOI), `version`, `name` (from `\title{}`), `description` (from abstract), empty `inputs:`/`outputs:`/`findings:`. Validates as-is. The script handles a few realities of LaTeX papers automatically: diff --git a/claude/lightcone/skills/paper-extraction/examples/unions-bmodes-astra.yaml b/claude/lightcone/skills/paper-extraction/examples/unions-bmodes-astra.yaml deleted file mode 100644 index 8fa35ceb..00000000 --- a/claude/lightcone/skills/paper-extraction/examples/unions-bmodes-astra.yaml +++ /dev/null @@ -1,106 +0,0 @@ -# Worked Step 5 example for paper-extraction. -# -# Generated from arXiv:2604.03227, then filled with 6 quote-anchored -# findings. Verified with: -# -# astra paper add 10.48550/arXiv.2604.03227 --version 1 --pdf paper.pdf -# astra validate astra.yaml --verify-evidence - -id: arxiv_2604_03227 -version: "0.0.7" -name: "UNIONS-3500 Weak Lensing: II. B-mode validation for cosmic shear" - -narrative: - summary: | - At Stage-III sensitivities, cosmic shear $B$ modes unambiguously indicate systematic contamination and are often used to inform data selection and scale cuts for cosmological inference. - We validate $B$ modes for the Ultraviolet Near-Infrared Optical Northern Survey (UNIONS)-3500 (\SI{2894}{\square\deg}, $n_\mathrm{eff} \approx \SI{5.0}{arcmin\tothe{-2}}$) using three $E$/$B$-separable statistics: pure-mode correlation functions $\xi_\pm^{\mathrm{B}}(\theta)$, Complete Orthogonal Sets of $E$/$B$-mode Integrals (COSEBI) $B$-mode amplitudes $B_n$, and harmonic-space power spectra $C_\ell^{BB}$. - For each statistic, we compute probability-to-exceed (PTE) values over a two-dimensional grid of scale-cut boundaries; our adopted cuts lie in broad stable regions of acceptable PTE. - $B$-mode detections and PTE failures on initial catalog versions led us to investigate galaxy size cuts and stellar halo masking. - After cuts, all three statistics pass the null test (minimum PTE $= \num{0.18}$). - Before scale cuts, we measure an oscillatory COSEBI $B$-mode pattern consistent with repeating additive shear bias, a detector-level effect seen across multiple Stage-III surveys including CFHTLenS, which used the same MegaCam camera; scale cuts that exclude the charge-coupled device (CCD) angular scale suppress it. - Although these statistics probe the same two-point shear field, scale cuts in one do not map exactly onto cuts in another, because their respective filter functions weight angular scales differently. - The most conservative validation therefore requires scale and sample selections that pass null tests across all frameworks simultaneously, an approach that applies directly to Stage-IV surveys where systematic errors dominate. - findings: | - The paper validates the UNIONS-3500 weak-lensing B-mode surface across three estimators: the [adopted cuts pass all three null tests](#findings.adopted_cuts_pass_all_statistics), [the conclusion restates consistency with zero at those cuts](#findings.bmodes_consistent_with_zero_at_adopted_cuts), and [the cuts remain acceptable under a two-parameter scale-cut accounting](#findings.scale_cut_degrees_of_freedom_still_pass). The central systematic is an [oscillatory full-range COSEBI pattern consistent with repeating additive shear bias](#findings.full_range_cosebi_repeating_additive_pattern). The paper also shows that [only the fiducial catalog passes in every representation](#findings.fiducial_catalog_only_full_pass) and that [COSEBI versus harmonic-space disagreement is driven by filter-function sensitivity](#findings.filter_functions_drive_representation_disagreement). - -inputs: [] -outputs: [] - -findings: - adopted_cuts_pass_all_statistics: - id: adopted_cuts_pass_all_statistics - label: "Adopted cuts pass all three B-mode null tests" - claim: | - After galaxy-size cuts and stellar-halo masking choices, the adopted UNIONS-3500 scale cuts pass pure-mode correlation-function, COSEBI, and harmonic-space B-mode null tests, with a minimum PTE of 0.18. - created_at: "2026-05-08T03:32:00+02:00" - evidence: - - id: abstract_minimum_pte - doi: "10.48550/arXiv.2604.03227" - version: 1 - quote: - exact: "After cuts, all three statistics pass the null test (minimum PTE = 0.18)." - - bmodes_consistent_with_zero_at_adopted_cuts: - id: bmodes_consistent_with_zero_at_adopted_cuts - label: "B modes are consistent with zero at the adopted scale cuts" - claim: | - In the paper's conclusion, the adopted scale cuts leave B modes consistent with zero across pure-mode correlation functions, COSEBIs, and harmonic-space power spectra. - created_at: "2026-05-08T03:32:00+02:00" - evidence: - - id: conclusion_consistency - doi: "10.48550/arXiv.2604.03227" - version: 1 - quote: - exact: "UNIONS-3500 weak-lensing B modes are consistent with zero at the adopted scale cuts across pure-mode correlation functions, COSEBIs, and harmonic-space power spectra." - - scale_cut_degrees_of_freedom_still_pass: - id: scale_cut_degrees_of_freedom_still_pass - label: "Scale-cut degree-of-freedom accounting still passes" - claim: | - Treating the selected scale-cut boundaries as two fitted parameters lowers the minimum PTE from 0.18 to 0.09, which remains above the 0.05 failure threshold. - created_at: "2026-05-08T03:32:00+02:00" - evidence: - - id: pte_two_parameter_accounting - doi: "10.48550/arXiv.2604.03227" - version: 1 - quote: - exact: "Doing so lowers the minimum PTE across all statistics from 0.18 to 0.09, still above the 0.05 threshold." - - full_range_cosebi_repeating_additive_pattern: - id: full_range_cosebi_repeating_additive_pattern - label: "Full-range COSEBIs show repeating-additive pattern" - claim: | - Before the adopted cuts, all catalog versions show an oscillatory full-range COSEBI B-mode pattern consistent with repeating additive shear bias at CCD angular scales. - created_at: "2026-05-08T03:32:00+02:00" - evidence: - - id: conclusion_full_range_pattern - doi: "10.48550/arXiv.2604.03227" - version: 1 - quote: - exact: "On the full angular range, all catalog versions show an oscillatory COSEBI B-mode pattern consistent with repeating additive shear bias at CCD angular scales" - - fiducial_catalog_only_full_pass: - id: fiducial_catalog_only_full_pass - label: "Only the fiducial catalog passes every representation" - claim: | - Of the four catalog variants tested, only the fiducial size-cut catalog passes the full set of B-mode validation representations at the adopted cuts. - created_at: "2026-05-08T03:32:00+02:00" - evidence: - - id: conclusion_fiducial_only - doi: "10.48550/arXiv.2604.03227" - version: 1 - quote: - exact: "Of the four catalog variants tested, only the fiducial passes in every representation." - - filter_functions_drive_representation_disagreement: - id: filter_functions_drive_representation_disagreement - label: "Filter functions explain representation disagreement" - claim: | - The paper argues that COSEBI versus harmonic-space disagreement is not a real-space versus harmonic-space basis effect; COSEBI filter functions concentrate sensitivity on contaminated angular scales. - created_at: "2026-05-08T03:32:00+02:00" - evidence: - - id: discussion_harmonic_cosebi_comparison - doi: "10.48550/arXiv.2604.03227" - version: 1 - quote: - exact: "By computing COSEBIs from the harmonic-space bandpowers, we confirm that the disagreement is not a matter of harmonic versus real space" diff --git a/claude/lightcone/skills/paper-extraction/scripts/extract-paper-substrate.py b/claude/lightcone/skills/paper-extraction/scripts/extract-paper-substrate.py index ce2309ee..5b6b4513 100755 --- a/claude/lightcone/skills/paper-extraction/scripts/extract-paper-substrate.py +++ b/claude/lightcone/skills/paper-extraction/scripts/extract-paper-substrate.py @@ -545,10 +545,10 @@ def write_astra_yaml_stub( ) -> str: """Emit a stub `work/reference/astra.yaml` that the agent fills in. - The script populates: id, version, name, narrative.summary (from abstract), + The script populates: id, version, name, description (from abstract), inputs/outputs as empty lists, and an empty findings map. The agent's job (Step 5 in SKILL.md) is to walk the paper and append findings entries with - quote evidence, plus a `narrative.findings:` cross-link. Once that's in, + quote evidence. Once that's in, `astra validate work/reference/astra.yaml` should pass. If the file already exists, leave it alone — it may have agent edits. @@ -562,21 +562,19 @@ def write_astra_yaml_stub( summary_str = abstract or "TODO: one-paragraph summary of the paper (no abstract extracted)" # Indent the summary as a block scalar so multi-line abstracts round-trip - summary_indented = "\n".join(" " + line for line in summary_str.splitlines()) + summary_indented = "\n".join(" " + line for line in summary_str.splitlines()) content = f"""# Stub ASTRA representation of the source paper. # -# Populated by paper-extraction's script: id, version, name, narrative.summary. +# Populated by paper-extraction's script: id, version, name, description. # The agent (paper-extraction Step 5) fills in `findings:` with the paper's -# claimed numerical results plus a `narrative.findings:` cross-link, then runs -# `astra validate astra.yaml` to confirm. +# claimed numerical results, then runs `astra validate astra.yaml` to confirm. id: {astra_id} version: "{ASTRA_SCHEMA_VERSION}" name: {json.dumps(title_str)} -narrative: - summary: | +description: | {summary_indented} inputs: [] diff --git a/docs/skills/figure-comparison.md b/docs/skills/figure-comparison.md index 3b2ad9df..c5b65fd6 100644 --- a/docs/skills/figure-comparison.md +++ b/docs/skills/figure-comparison.md @@ -47,8 +47,8 @@ The skill picks its target set in priority order: 2. **`targets/targets.md`** — the SPECIFY-phase scope ledger, used when COMPARE hasn't run yet. 3. **Default paper-driven flow** — when neither scope file exists, - builds a best-effort report from `astra.yaml`'s narrative and - findings plus `work/reference/`. + builds a best-effort report from `astra.yaml`'s `description` and + `findings:` plus `work/reference/`. ## Output diff --git a/docs/skills/index.md b/docs/skills/index.md index e1c352b7..6e367fba 100644 --- a/docs/skills/index.md +++ b/docs/skills/index.md @@ -12,8 +12,8 @@ This page is for maintainers. ## Available skills The `/lc-from-*` family is parallel in what you start from: a question, -code, or a paper. `/lc-from-paper` is the entry point of a six-skill -paper-reproduction bundle; the five siblings stand alone and are +code, or a paper. `/lc-from-paper` is the entry point of a five-skill +paper-reproduction bundle; the four siblings stand alone and are user-invokable directly. ### Project lifecycle @@ -36,7 +36,6 @@ dispatches them by role during the reproduction. |-------|---------|---------| | [ralph](ralph.md) | `/ralph` | Loop substrate. `lc-from-paper`'s ORIENT invokes ralph's Authoring mode to draft the per-paper constitution; the loop launcher hands off after ORIENT lands; each iteration runs ralph's Loop protocol. Also user-invokable standalone (see the Project lifecycle row above). | | [paper-extraction](paper-extraction.md) | `/paper-extraction` | Turn an arXiv ID or DOI into a standardized `work/reference/` directory: substrate, figures, tables, citations (with resolved DOIs), and a stub `astra.yaml`. | -| [narrative](narrative.md) | `/narrative` | Author the `narrative:` prose and decision `rationale:` against an existing `astra.yaml`, in paper-reproduction, retrofit, or co-drafting mode. | | [figure-comparison](figure-comparison.md) | `/figure-comparison` | Build a self-contained HTML side-by-side: paper figures, tables, and numerics vs reproduced artifacts. | | [check-sentence-by-sentence](check-sentence-by-sentence.md) | `/check-sentence-by-sentence` | Static audit of paper claims against code locations (`file:line` or `NOT FOUND`). | @@ -48,7 +47,7 @@ Not entry points. Other skills invoke them — or Claude does, when a deeper ref | Skill | Command | Purpose | |-------|---------|---------| -| `astra` | `/astra` | Reference for the `astra.yaml` spec: structure, decisions, options, prior insights, findings, evidence, sub-analyses, narrative anchors, composition mechanics. | +| `astra` | `/astra` | Reference for the `astra.yaml` spec: structure, decisions, options, prior insights, findings, evidence, sub-analyses, composition mechanics. | | `lc-cli` | `/lc-cli` | Reference for `lc` workflow: commands, the Spec-Code Invariant, status interpretation, failure diagnosis, multiverse runs, publishing via WRROC. | These intentionally stay out of the top-level README. Researchers use the project-lifecycle skills directly; the reference skills are infrastructure. @@ -85,7 +84,6 @@ claude/lightcone/ │ ├── lc-feedback/SKILL.md │ ├── ralph/{SKILL.md, references/*.md, scripts/ralph} │ ├── paper-extraction/{SKILL.md, scripts/*.py} -│ ├── narrative/{SKILL.md, references/*.md} │ ├── figure-comparison/{SKILL.md, scripts/*.py} │ ├── check-sentence-by-sentence/SKILL.md │ ├── astra/SKILL.md # reference: astra.yaml spec diff --git a/docs/skills/lc-from-paper.md b/docs/skills/lc-from-paper.md index 89f3c53f..aaa1f4ae 100644 --- a/docs/skills/lc-from-paper.md +++ b/docs/skills/lc-from-paper.md @@ -12,7 +12,7 @@ the loop closes. `/lc-from-paper` is the entry point of the paper-reproduction bundle. Sibling skills ([`ralph`](https://github.com/LightconeResearch/lightcone-cli/blob/main/claude/lightcone/skills/ralph/SKILL.md) for the loop, [`paper-extraction`](paper-extraction.md), -[`narrative`](narrative.md), [`figure-comparison`](figure-comparison.md), +[`figure-comparison`](figure-comparison.md), [`check-sentence-by-sentence`](check-sentence-by-sentence.md)) live in the same plugin and are invoked by role across the phases. @@ -51,8 +51,8 @@ session; phases 1–6 run as ralph iterations. | # | Phase | Where | Primary outputs | |---|-------|-------|------------------| | 0 | ORIENT | user's main session | per-paper `constitution.md` + `CLAUDE.md` + paper substrate at `work/reference/{paper.pdf, source/ or document.md, figures/, tables/, index.json, astra.yaml}` (from inline `/paper-extraction`) + code substrate at `work/reference/{code/, code-status.yaml, code-index.md}` (from inline `/lc-from-code` scan-only, when a repo exists) | -| 1 | ARCHITECT | ralph iteration | stub `astra.yaml` (sub-analyses, inputs, outputs, narrative) | -| 2 | SPECIFY | ralph iteration | filled `astra.yaml` (`decisions:`, `findings:`, `prior_insights:` placeholders, anchored narrative); `targets/targets.md`; `implementation-notes.md`; `universes/baseline.yaml` | +| 1 | ARCHITECT | ralph iteration | stub `astra.yaml` (sub-analyses, inputs, outputs, per-analysis `description`) | +| 2 | SPECIFY | ralph iteration | filled `astra.yaml` (`decisions:`, `findings:`, `prior_insights:` placeholders); `targets/targets.md`; `implementation-notes.md`; `universes/baseline.yaml` | | 3 | LITERATURE | ralph iteration | `prior_insights:` Evidence entries each carry resolved `quote:` + `location:` selectors; per-paper PDFs cached via `astra paper add` | | 4 | IMPLEMENT | ralph iteration | `scripts/`, `requirements.txt`, recipes in `astra.yaml` | | 5 | RUN | ralph iteration | `results///` | @@ -165,8 +165,6 @@ Pointers, not snapshots. — the loop substrate (authoring + launching + iterating). - [`/paper-extraction`](paper-extraction.md) — ORIENT Stage 2's acquisition path; also invoked per cited paper by LITERATURE. -- [`/narrative`](narrative.md) — ARCHITECT's structural narrative and - SPECIFY's anchored content narrative. - [`/figure-comparison`](figure-comparison.md) — REVIEW (mandatory) and also user-invokable. - [`/check-sentence-by-sentence`](check-sentence-by-sentence.md) — diff --git a/docs/skills/lc-new.md b/docs/skills/lc-new.md index 038caace..4196f3cc 100644 --- a/docs/skills/lc-new.md +++ b/docs/skills/lc-new.md @@ -37,10 +37,10 @@ files. The `lc-extractor` subagent is dispatched via `Agent`. literature together. 4. **Finalize.** `astra validate astra.yaml`; `astra validate --verify-evidence` if quotes exist; `astra universe generate -n - baseline`. Populate the `narrative:` block (`summary`, `methods`, - `inputs`, `outputs` — `findings` stays TODO until results exist), - then fill the `## Working Notes` section of `CLAUDE.md` with - conversational context the spec doesn't carry. + baseline`. Author a short `description:` on the root analysis (one or + two paragraphs orienting a reader), then fill the `## Working Notes` + section of `CLAUDE.md` with conversational context the spec doesn't + carry. Writes happen at the end of each phase, not in bulk — the user always has something visible to review. diff --git a/docs/skills/narrative.md b/docs/skills/narrative.md deleted file mode 100644 index 5a13e69f..00000000 --- a/docs/skills/narrative.md +++ /dev/null @@ -1,107 +0,0 @@ -# /narrative - -Author the reader-facing prose in an `astra.yaml`: analysis-level -`narrative:` blocks (`summary`, `inputs`, `methods`, `findings`, -`outputs`), decision `rationale:` fields, and shorter `description:` / -`notes:` on individual entities. Always written against an existing -spec — the structure must exist when the prose lands. - -Source: [`claude/lightcone/skills/narrative/SKILL.md`](https://github.com/LightconeResearch/lightcone-cli/blob/main/claude/lightcone/skills/narrative/SKILL.md). - -## Modes - -The skill draws on the spec plus a **second source**. Three modes, -distinguished by what that second source is: - -| Mode | Second source | Status | -|---|---|---| -| **Paper reproduction** | An authoritative text (paper, thesis, technical report) | Ready | -| **Retrofit** | Project artifacts — code, notebooks, fibers, commit history | Stub | -| **Co-drafting** | The user, in conversation | Stub | - -If the second source isn't obvious, the skill asks. Hybrid is allowed -(reproduction with co-drafted extensions; retrofit with co-drafted -gap-filling). - -`/lc-from-paper` invokes `/narrative` during SPECIFY (paper-reproduction -mode); users can invoke it directly in any mode. - -## Allowed surfaces - -The five-key analysis narrative: - -| Key | What it carries | Required when | -|---|---|---| -| `summary` | Question, scope, headline shape | optional, but should always exist | -| `inputs` | Provenance — the data the analysis rests on | `Analysis.inputs` non-empty | -| `methods` | Pipeline walk; cite each decision and sub-analysis by anchor | `Analysis.decisions` or `Analysis.analyses` non-empty | -| `findings` | Synthesis of declared findings, each cited by anchor | `Analysis.findings` non-empty | -| `outputs` | Which artifacts were promoted, and where they go downstream | `Analysis.outputs` non-empty | - -A decision's `rationale:` is its own one-paragraph slot: what was -decided, the insight that motivated it (cite by anchor), and the -load-bearing alternative and why it lost. Per-entity prose -(`description`, `notes`) is shorter and lives on individual entries. - -## Anchors - -Markdown link syntax with `#`-target, **tree-path-first** — same -grammar as decision `from:` references. - -| Target | Anchor | -|---|---| -| Input | `#inputs.` | -| Output | `#outputs.` | -| Decision | `#decisions.` | -| Option | `#decisions..options.` | -| Finding | `#findings.` | -| Prior insight | `#prior_insights.` | -| Sub-analysis | `#analyses.` | -| Element inside a sub-analysis | `#..` | -| Parent scope from a sub-analysis | `#../decisions.` | - -Anchor text is **authored prose**, never the raw id. One reference per -idea — stacking three on a sentence means the sentence carries too -much. - -## Length and modularity - -1–3 paragraphs per key, at any level. Length is the mechanism that -keeps analyses modular: **if references don't fit in three paragraphs, -the analysis is too big — split it.** The narrative is a compressor; -if it won't compress, split the thing being compressed. - -## Validation - -```sh -astra validate astra.yaml -``` - -- **Broken references** → error. -- **Uncited declared elements** → warning. Every declared finding, - decision, output, and sub-analysis must be cited somewhere in the - narrative tree. -- **Conditional coverage** (required-when rules above) → error. - -## Anti-patterns - -- **Wiki-style what-is framing.** A wiki summarizes; an ASTRA narrative - points into reasoning. -- **Decision-list paragraph.** "We made the following decisions: A, B, - C." Cite each where it shapes the pipeline. -- **`summary` as primer.** Teaching what the field is. Readers arrive - with context. -- **Drafting `findings` on a sub-analysis with no declared findings.** - Skip the key. -- **Narrative-per-element.** The five-key analysis narrative is the - only home; per-element prose is `description` / `rationale` / - `notes`. - -Mode-specific anti-patterns live in each mode's reference under -`claude/lightcone/skills/narrative/references/`. - -## Related - -- [`/lc-from-paper`](lc-from-paper.md) — invokes `/narrative` during - SPECIFY in paper-reproduction mode. -- [`/astra`](index.md#reference-skills-auto-primed-via-session-start) — full schema reference. diff --git a/docs/skills/paper-extraction.md b/docs/skills/paper-extraction.md index 7c7cc05f..92235b38 100644 --- a/docs/skills/paper-extraction.md +++ b/docs/skills/paper-extraction.md @@ -56,7 +56,8 @@ this when you want "what's in this paper, where do I find it." DOI resolution covers ~96% of typical-paper bibliographies. **`astra.yaml` is semantic and ASTRA-validating.** Treats the paper as -an ASTRA artifact: `id`, `name`, `narrative.summary`, and `findings:` +an ASTRA artifact: `id`, `name`, a `description:` (from the abstract), +and `findings:` carrying the paper's claimed numerical results in the Insight + Evidence shape. The verbosity of the shape *is* the back-pressure against hallucinated claims — the agent has to find and quote actual diff --git a/docs/user/agent-workflow.md b/docs/user/agent-workflow.md index 9ebf6b63..25d30a07 100644 --- a/docs/user/agent-workflow.md +++ b/docs/user/agent-workflow.md @@ -41,11 +41,10 @@ The skill walks you through four phases: (methodological choices that could shift results). `→ astra.yaml`. 4. **Finalize.** `astra validate astra.yaml` to make sure the spec is valid; `astra universe generate -n baseline` to seed a baseline - universe; the `narrative:` block in `astra.yaml` gets filled in - (`summary`, `methods`, `inputs`, `outputs` — `findings` stays TODO - until results exist); the `## Working Notes` section of `CLAUDE.md` - gets the conversational context that wouldn't otherwise survive a - `/clear`. + universe; a short `description:` gets authored on the root analysis + (one or two paragraphs orienting a reader); the `## Working Notes` + section of `CLAUDE.md` gets the conversational context that wouldn't + otherwise survive a `/clear`. You don't write any code or YAML during `/lc-new`. By the time it finishes, you have a precise specification. The agent enforces @@ -122,7 +121,7 @@ CLAUDE.md's *Open opportunities* list as the trajectory of what could be tightened on a return visit. The bundle composes sibling skills: `ralph` (the loop substrate), -`paper-extraction`, `narrative`, `figure-comparison`, and +`paper-extraction`, `figure-comparison`, and `check-sentence-by-sentence`. See [`claude/lightcone/skills/README.md`](https://github.com/LightconeResearch/lightcone-cli/blob/main/claude/lightcone/skills/README.md) for the full bundle map. diff --git a/docs/user/getting-started.md b/docs/user/getting-started.md index abbbd253..cfe2085b 100644 --- a/docs/user/getting-started.md +++ b/docs/user/getting-started.md @@ -142,7 +142,8 @@ container: Containerfile ``` Phase 4 (**FINALIZE**) runs `astra validate astra.yaml`, writes -`universes/baseline.yaml`, and fills in the `narrative:` block. You're handed +`universes/baseline.yaml`, and authors a short `description:` on the root +analysis. You're handed back a short summary table — two outputs, one decision, zero prior insights. The agent may suggest `/clear` to free up context. Take its advice. diff --git a/pyproject.toml b/pyproject.toml index a48f4e25..f7e5d297 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -26,7 +26,7 @@ classifiers = [ ] dependencies = [ - "astra-tools>=0.2.5", + "astra-tools>=0.2.10", "click>=8.0", "pyyaml>=6.0", "rich>=13.0", diff --git a/zensical.toml b/zensical.toml index 31acfefd..5206196a 100644 --- a/zensical.toml +++ b/zensical.toml @@ -52,7 +52,6 @@ nav = [ {"lc-feedback" = "skills/lc-feedback.md"}, {"ralph" = "skills/ralph.md"}, {"paper-extraction" = "skills/paper-extraction.md"}, - {"narrative" = "skills/narrative.md"}, {"figure-comparison" = "skills/figure-comparison.md"}, {"check-sentence-by-sentence" = "skills/check-sentence-by-sentence.md"}, {"Authoring Skills" = "skills/authoring.md"},