Skip to content

Commit 4e02048

Browse files
MaxGhenisclaude
andcommitted
Update overnight session summary with G1 unblocker + follow-on additions
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent e46eb49 commit 4e02048

1 file changed

Lines changed: 42 additions & 3 deletions

File tree

docs/overnight-session-2026-04-16.md

Lines changed: 42 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ In order:
2727
| `06367fa` | `__main__.py` entry point + incremental-JSONL test |
2828
| `e750dc4` | Stage-1 results at 40k × 50 × 3 methods (key finding) |
2929
| `d0fa450` | Stage-1 at full 77k; cap PRDC samples to avoid OOM |
30+
| `6763237` | Apples-to-apples 40k with capped PRDC; overnight summary |
31+
| `225eb36` | Per-column zero-rate breakdown + embedding-PRDC validation script |
32+
| `31bae2a` | **Wire MicrocalibrateAdapter into us.py pipeline — G1 unblocker** |
33+
| `e46eb49` | Test zero_rate_per_column populated on every result |
3034

3135
Plus one commit on `main` archive: `archive/semantic-guards-wip-20260416` on microplex (core). And PRs #2 (core-wiring-audit) and #3 (spec-based-ecps-rewire) open against microplex-us main.
3236

@@ -89,9 +93,44 @@ Branch is in good shape for review. No outstanding tasks block merge.
8993

9094
## What I did not do
9195

92-
- **No changes to main production pipelines.** `pe_us_data_rebuild_checkpoint.py` / `us.py` are untouched. The rewire lives on its branch as docs + harness + adapter, ready to wire in.
93-
- **No v7 run.** With the stage-1 evidence now in hand, the next production run should use the rewired path (CPS scaffold + microcalibrate), not another v4/v5/v6-style invocation of the current pipeline.
94-
- **No rerun on GPU.** ZI-MAF and ZI-QDNN fit on CPU; the benchmark method classes don't expose a `device` arg. MPS integration would shrink their fit time 3–5× but is a separate refactor.
96+
- **No v7 run.** With the stage-1 evidence now in hand and
97+
`--calibration-backend microcalibrate` wired, the next production run
98+
should use that flag against the current pipeline. Expected outcome:
99+
the v4/v6 OOM is gone.
100+
- **No rerun on GPU.** ZI-MAF and ZI-QDNN fit on CPU; the benchmark
101+
method classes don't expose a `device` arg. MPS integration would
102+
shrink their fit time 3–5× but is a separate refactor.
103+
104+
## Second-half work (after initial summary)
105+
106+
After the stage-1 evidence landed, I continued with the open items:
107+
108+
1. **Microcalibrate wiring into `us.py`** (commit `31bae2a`) — 20-line
109+
change plus dispatch test. `calibration_backend="microcalibrate"` is
110+
now a valid configuration that routes to `MicrocalibrateAdapter`.
111+
The existing `_apply_policyengine_constraint_stage` call site at
112+
`us.py:2931` needed zero changes because the adapter matches the
113+
legacy `Calibrator.fit_transform` / `.validate` contract exactly.
114+
`docs/microcalibrate-wiring-plan.md` captures rollout steps and
115+
risk register.
116+
2. **Per-column zero-rate breakdown** (commits `225eb36`, `e46eb49`) —
117+
`ScaleUpResult.zero_rate_per_column` now reports `{real, synth,
118+
abs_diff}` per column. Lets the pilot/stage-1 findings identify
119+
which specific columns drive each method's overall zero-rate error.
120+
The stage-1 finding "all methods drive disabled_ssdi to 0" can be
121+
audited in finer detail on the next run.
122+
3. **Embedding-PRDC validation script**
123+
(`scripts/embedding_prdc_compare.py`, commit `225eb36`) — standalone
124+
CLI that fits a 16-dim autoencoder on the holdout, encodes real and
125+
synthetic, and reports PRDC both in raw 50-dim space and in the
126+
learned 16-dim latent space. Settles whether the stage-1 ordering
127+
is metric-driven or method-driven. Not yet executed.
128+
4. **ZI-MAF hyperparameter tuning run in progress** — four configs
129+
(default, wide, long, wide+long). Running at 40k × 50. Job started
130+
07:16 ET and is still progressing; will land in a separate doc
131+
update once complete.
132+
133+
Updated PR #3 count: **15 commits**, all green tests, all pushed.
95134

96135
## How to run stage 1 yourself
97136

0 commit comments

Comments
 (0)