Skip to content

Commit c76322d

Browse files
wmaynerclaude
andcommitted
Document provenance writers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEAxNzhDCaTrntX3o1JqMV
1 parent 0b2e7ad commit c76322d

3 files changed

Lines changed: 46 additions & 3 deletions

File tree

ROADMAP.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -633,9 +633,13 @@ extending B1's bound-certificate assertions.
633633
is gone. Spec: `docs/superpowers/specs/2026-07-19-result-serialization-design.md`.
634634
- **Thread-backend progress (M13).** The thread backend discards the progress policy on entry
635635
(an in-code deferral note); hook the progress bar into its futures loop.
636-
- **Script-facing provenance writer (M14).** `provenance.save_json`/`save_npz` with git SHA,
637-
parameters encoded in the filename, and no-clobber versioning — consolidating the pattern
638-
that experiment scripts repeatedly re-implement.
636+
- **Script-facing provenance writer (M14).** *Landed 2026-07-20:*
637+
`provenance.save_json`/`save_npz`/`save_dataframe` (parquet, per the DataFrame-output
638+
convention) with `format_stem` filename encoding, `unique_path` no-clobber versioning, an
639+
embedded `Provenance` record, and `read_metadata` to read it back; the three benchmark
640+
scripts with hand-rolled copies now import the shared helpers (the cross-generation
641+
harness keeps a fallback for the pre-refactor side). Spec:
642+
`docs/superpowers/specs/2026-07-20-provenance-writers-design.md`.
639643

640644
Already tracked elsewhere (verified during the triage): the Wave 7 ii-gate (M4), the
641645
intervention/lesion surface (N14 ← M8), the Φ-structure distance surface (N16 ← M6), the
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added script-facing provenance writers: `pyphi.provenance.save_json`, `save_npz`, and `save_dataframe` (parquet) encode parameters in the filename, never overwrite existing files (`_v2`/`_v3` versioning), and embed a full `Provenance` record (git SHA, seed, versions) in every output; `read_metadata` reads it back from any of the three formats.

docs/howto/save-load.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,44 @@ except TypeError as error:
133133
print(error)
134134
```
135135

136+
## Experiment provenance writers
137+
138+
Experiment scripts need two things beyond `pyphi.save`: output files that
139+
never overwrite earlier runs, and a record of how each file was produced.
140+
The writers in `pyphi.provenance` provide both. Parameters are encoded
141+
into the filename, a repeated save lands in a `_v2` file instead of
142+
clobbering the first, and every file embeds a full provenance record —
143+
pyphi version, git commit, timestamp, and seed.
144+
145+
```{code-cell} python
146+
from pyphi import provenance
147+
148+
path = provenance.save_json(
149+
{"phi": 0.133873},
150+
out,
151+
"sweep_study",
152+
params={"seed": 42, "trials": 60},
153+
)
154+
path.name
155+
```
156+
157+
```{code-cell} python
158+
provenance.save_json(
159+
{"phi": 0.5}, out, "sweep_study", params={"seed": 42, "trials": 60}
160+
).name
161+
```
162+
163+
`save_npz` does the same for arrays of raw per-trial data, and
164+
`save_dataframe` writes a DataFrame as parquet — the format used for
165+
DataFrame outputs throughout PyPhi — with the metadata embedded in the
166+
parquet schema. `read_metadata` recovers the provenance and parameters
167+
from any of the three formats:
168+
169+
```{code-cell} python
170+
metadata = provenance.read_metadata(path)
171+
{key: metadata["provenance"][key] for key in ("seed", "pyphi_version")}
172+
```
173+
136174
## Compatibility note
137175

138176
This serializer is a deliberate format break from the `jsonify` layer used in

0 commit comments

Comments
 (0)