|
1 | 1 | # lightcone-cli |
2 | 2 |
|
| 3 | +[](https://opensource.org/licenses/BSD-3-Clause) |
| 4 | +[](https://pypi.org/project/lightcone-cli/) |
3 | 5 | [](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml) |
4 | | -[](https://www.python.org/downloads/) |
5 | | -[](https://opensource.org/licenses/BSD-3-Clause) |
6 | | -[](https://github.com/astral-sh/ruff) |
| 6 | +[](https://pypi.org/project/lightcone-cli/) |
7 | 7 |
|
8 | | -**lightcone-cli** (`lc`) is the agentic layer for [ASTRA](https://github.com/LightconeResearch/ASTRA) (Agentic Schema for Transparent Research Analysis). You interact with lightcone-cli through Claude Code: describe what you want, and the agent handles the implementation. |
| 8 | +<!-- [](https://github.com/astral-sh/ruff) --> |
9 | 9 |
|
10 | | -> **Note on the `lc` executable.** `lc` is not a standard Unix tool, but some users have a personal shell alias `lc='ls --color'`. If that's you, installing lightcone-cli will shadow that alias — rebind it if needed. |
| 10 | +**lightcone-cli** (`lc`) is the agentic execution layer for |
| 11 | +[ASTRA](https://astra-spec.org/latest/) (Agentic Schema for Transparent |
| 12 | +Research Analysis). Describe your analysis to an AI agent and `lc` takes |
| 13 | +care of the rest — specification, execution, and provenance. |
11 | 14 |
|
12 | 15 | ## Quick Start |
13 | 16 |
|
14 | 17 | ```bash |
15 | | -pip install lightcone-cli |
| 18 | +uv tool install lightcone-cli |
16 | 19 | lc init my-analysis |
17 | 20 | cd my-analysis |
18 | 21 | claude |
19 | 22 | ``` |
20 | 23 |
|
21 | | -Then tell the agent what you have to start from — a research question (`/lc-new`), existing code (`/lc-from-code`), or a paper to reproduce (`/lc-from-paper`). After the spec exists, work with the agent however suits you; the substrate (`astra.yaml`, `lc run`, `lc status`, `lc verify`) keeps things in sync. |
| 24 | +Then tell the agent what you have to start from — a research question |
| 25 | +(`/lc-new`), existing code (`/lc-from-code`), or a paper to reproduce |
| 26 | +(`/lc-from-paper`). |
22 | 27 |
|
23 | | -## Skills |
24 | | - |
25 | | -The `/lc-from-*` family is parallel by what you start from: a question, code, or a paper. |
26 | | - |
27 | | -### `/lc-new` — Scope and specify an analysis |
28 | | - |
29 | | -Guides you from a research question to a complete `astra.yaml` specification through interactive conversation. The agent will: |
30 | | - |
31 | | -- Help you identify the key decisions (methodological choices) in your analysis |
32 | | -- Search for and read relevant papers, extracting prior insights with exact verified quotes |
33 | | -- Structure decisions with options, defaults, and constraints between them |
34 | | -- Build universe files representing defensible alternative analysis paths |
35 | | -- Link literature evidence (prior insights) to the decision options it supports |
36 | | - |
37 | | -You don't write any code or YAML during this phase — the agent produces the full specification. |
38 | | - |
39 | | -### `/lc-from-code` — Bring an existing project into ASTRA |
40 | | - |
41 | | -Scans an existing codebase, drafts an `astra.yaml` that captures its inputs, outputs, and analytical decisions, parameterizes the code so decisions can vary across universes, and runs the analysis through `lc` until every output materializes. Existing logic is left intact — changes are confined to parameter plumbing. |
42 | | - |
43 | | -### `/lc-from-paper` — Reproduce a published paper |
44 | | - |
45 | | -ORIENT-first driver for reproducing a published paper in ASTRA. ORIENT runs in the user's main session in seven stages — asks for the paper, runs `/paper-extraction` inline to acquire it, interviews the user (grounded in the paper), clones the reference code and runs `/lc-from-code` scan-only (if a repo exists), optionally follows up, then drafts a per-paper `constitution.md` (the ralph loop's driving document) + `CLAUDE.md` (auto-loading rules + accumulators) from the full paper-plus-code context for user review. Then the rest of the reproduction hands off to a **ralph loop** whose iterations carry the long middle: ARCHITECT → SPECIFY → LITERATURE → IMPLEMENT → RUN → COMPARE. Each iteration runs in a fresh tmux session against the constitution; the fresh-context property between iterations is what makes per-phase review work. When the loop closes (constitution `status: closed` after COMPARE returns `pass`), REVIEW runs back in the user's main session. Composes a bundle of sibling skills (`ralph`, `paper-extraction`, `narrative`, `figure-comparison`, `check-sentence-by-sentence`). See [`claude/lightcone/skills/README.md`](claude/lightcone/skills/README.md) for the full bundle map. |
46 | | - |
47 | | -### `/lc-feedback` — Report a bug |
48 | | - |
49 | | -Files a GitHub issue against the right repo (ASTRA or lightcone-cli) with version info and error context auto-collected from your session. |
50 | | - |
51 | | -### Building and verifying |
52 | | - |
53 | | -Once `astra.yaml` exists, you (or the agent) build it however suits you. The typical flow is `lc run` to materialize outputs, `lc status` to track progress, `astra validate astra.yaml` for spec validity, and `lc verify` for provenance integrity — agent-driven, ralph-looped, or hand-written, the `lc` substrate stays in sync. |
54 | | - |
55 | | -## CLI Reference |
56 | | - |
57 | | -### Global configuration |
58 | | - |
59 | | -The first `lc` invocation auto-creates `~/.lightcone/config.yaml` with `container.runtime: auto`. To pin a runtime or change other settings, edit the file directly. |
60 | | - |
61 | | -### Project scaffolding |
62 | | - |
63 | | -```bash |
64 | | -lc init my-analysis # full scaffolding with Claude Code config |
65 | | -lc init my-analysis --no-git --no-venv # skip git/venv creation |
66 | | -lc init my-analysis --permissions yolo # Claude Code permission tier (yolo|recommended|minimal) |
67 | | -lc init my-analysis --scratch '$SCRATCH/lc' # override scratch root for snakemake state and dask spill |
68 | | -``` |
69 | | - |
70 | | -### Execution and monitoring |
| 28 | +→ [Full getting-started guide](https://docs.lightconeresearch.org/user/getting-started/) |
71 | 29 |
|
72 | | -The agent runs these as it builds out an analysis, but you can also run them directly: |
| 30 | +## Skills |
73 | 31 |
|
74 | | -```bash |
75 | | -lc run # materialize all outputs for all universes |
76 | | -lc run accuracy # materialize a specific output |
77 | | -lc run --universe baseline # materialize for a specific universe |
78 | | -lc run --jobs 8 # bound parallel job dispatch |
79 | | -lc run --force # force re-materialization |
80 | | -lc run --rerun-triggers code,input # override Snakemake rerun-triggers (default: code,input,mtime,params) |
81 | | -lc status # show materialization status (ok / stale / missing / alias) |
82 | | -lc status --universe baseline # status for a specific universe |
83 | | -lc status --json # machine-readable output |
84 | | -lc verify # recompute hashes and walk the provenance chain |
85 | | -lc build # pre-build container images from Containerfiles |
86 | | -lc build --force --runtime podman # force rebuild and pin runtime |
87 | | -lc export wrroc # export a Workflow Run RO-Crate bundle for publication |
88 | | -lc export wrroc --zip -o run.zip # zip the bundle for upload to Zenodo / WorkflowHub |
89 | | -``` |
| 32 | +| Skill | What it does | |
| 33 | +|---|---| |
| 34 | +| [`/lc-new`](https://docs.lightconeresearch.org/skills/lc-new/) | Scope a new analysis from a research question into a full `astra.yaml` spec | |
| 35 | +| [`/lc-from-code`](https://docs.lightconeresearch.org/skills/lc-from-code/) | Bring an existing codebase into ASTRA | |
| 36 | +| [`/lc-from-paper`](https://docs.lightconeresearch.org/skills/lc-from-paper/) | Reproduce a published paper end-to-end | |
| 37 | +| [`/lc-feedback`](https://docs.lightconeresearch.org/skills/lc-feedback/) | File a bug report with version and error context auto-collected | |
90 | 38 |
|
91 | 39 | ## Capabilities |
92 | 40 |
|
93 | | -### Multiverse analysis |
94 | | - |
95 | | -Define decisions (methodological choices) with multiple options. Each universe file selects one option per decision, representing a complete defensible analysis path. The agent can generate and run across all universes automatically. |
96 | | - |
97 | | -### Decision constraints |
98 | | - |
99 | | -Decisions can be mutually exclusive (`incompatible_with`) or co-required (`requires`). Options can also be marked as `excluded` with a reason, documenting alternatives that were considered and rejected. |
100 | | - |
101 | | -### Literature integration |
102 | | - |
103 | | -The agent can search for papers, download PDFs by DOI, and extract prior insights with exact quotes. Quotes are machine-verified against the source PDFs using fuzzy matching with Unicode normalization. Prior insights are linked to the decision options they support, creating a traceable evidence chain. After the analysis runs, findings capture conclusions backed by the analysis outputs. |
104 | | - |
105 | | -### Sub-analyses |
106 | | - |
107 | | -Complex analyses can be decomposed into nested stages, each with their own inputs, outputs, decisions, and recipes. Sub-analyses use the same schema as the top level, and can reference each other's outputs. |
108 | | - |
109 | | -### Execution backend |
110 | | - |
111 | | -`lc run` generates a Snakefile from `astra.yaml` and shells out to Snakemake. Snakemake owns DAG construction, staleness detection, parallel dispatch, and per-rule resource translation; `lc` owns the integrity layer (per-output content-addressed manifests written next to each result). |
112 | | - |
113 | | -Jobs always dispatch through a Dask cluster: a `LocalCluster` on a workstation, srun-launched workers inside a SLURM allocation, or an existing scheduler if `DASK_SCHEDULER_ADDRESS` is set. Recipes execute inside their declared container via Docker, podman, or podman-hpc — set `container.runtime` in `~/.lightcone/config.yaml` (default `auto`) to pin one. Per-recipe `resources:` (cpus, gpus, memory, time limit) flow through to the cluster. |
114 | | - |
115 | | -### Provenance integrity |
116 | | - |
117 | | -Every materialized output gets a `.lightcone-manifest.json` capturing `code_version` (sha256 of recipe + container + decisions), `data_version` (sha256 of the output directory contents), input manifest versions, git SHA, lc version, and host. `lc verify` recomputes hashes and walks the chain — failures surface as `tampered_data`, `broken_chain`, or `missing_manifest`. `lc status` reads only manifests, so it works offline. |
118 | | - |
119 | | -### Publishing analyses |
| 41 | +- **Multiverse analysis** — define methodological decisions with multiple options; `lc` runs your analysis across all defensible paths automatically |
| 42 | +- **Provenance integrity** — every output gets a content-addressed manifest; `lc verify` detects tampering or broken chains |
| 43 | +- **HPC-ready execution** — Snakemake-backed DAG dispatch with SLURM and container support (Docker, Podman, Apptainer) out of the box |
| 44 | +- **Reproducible publishing** — `lc export wrroc` emits a [Workflow Run RO-Crate](https://www.researchobject.org/workflow-run-crate/) bundle ready for Zenodo or WorkflowHub |
120 | 45 |
|
121 | | -`lc export wrroc` walks your manifests and emits a [Workflow Run RO-Crate](https://www.researchobject.org/workflow-run-crate/) bundle — a JSON-LD package readable by WorkflowHub, Zenodo's RO-Crate plugin, and any RO-Crate-aware archive. Each materialization becomes a `CreateAction` with `object` (inputs, including upstream datasets via stable `@id` references) and `result` (the output dataset); decisions become `PropertyValue` entities; the workflow is captured as a `ComputationalWorkflow`. The lightcone manifest format on disk is unchanged — WRROC is the publication view, generated on demand. Use `--metadata-only` to ship only the provenance graph (useful when data files are huge), `--zip` to package the bundle for upload, or `-u <universe>` to restrict to specific universes. |
| 46 | +→ [Full documentation](https://docs.lightconeresearch.org) |
122 | 47 |
|
123 | 48 | ## License |
124 | 49 |
|
125 | | -BSD 3-Clause |
| 50 | +BSD 3-Clause — see [LICENSE](LICENSE) for details. |
0 commit comments