Skip to content

Commit a2aeb6a

Browse files
feat(chapel): per-prover cwd + filenameOverride hooks (closes #158, closes #159) (#164)
## Summary - `ProverInfo` gains two optional fields (`cwd`, `filenameOverride`) with empty-string defaults; preserves prior contract for the 28 unaffected provers. - Agda + Idris2 registry entries set both, paired with a new `agda_trivial.agda` fixture and a refreshed `idris2_trivial.idr` (already correctly shaped). - `tryProver` shell-wraps the spawn when `cwd` is set (`sh -c \"cd <cwd> && exec ...\"`) for coforall-safety; `filenameOverride` swaps the generic basename when set. - Justfile `bench-chapel-mrr` derives `IDRIS2_PREFIX` from `which idris2` — the recipe is now self-contained, no parent-shell ceremony. ## Result `just bench-chapel-mrr` (single-run, full table in updated baseline doc): | fixture | strategy | wallclock | winning prover | |---|---|---|---| | idris2_trivial | sequential | 1.970 | Idris2 | | idris2_trivial | parallel_bestof | 1.037 | Idris2 | | idris2_trivial | parallel_speculative | 1.032 | Idris2 | | agda_trivial | sequential | 0.101 | Agda | | agda_trivial | parallel_bestof | 0.751 | Agda | | agda_trivial | parallel_speculative | 0.450 | Agda | Both acceptance criteria (Idris2 success in ≥1 strategy, Agda fixture round-trip) satisfied across all three strategies. ## Wave-1 baseline caveats resolved - Caveat 1 (Idris2 prelude resolution): closed via cwd hook + IDRIS2_PREFIX derivation - Caveat 2 (Agda module-name lexer): closed via filenameOverride + fixture pairing - Caveat 3 (sub-second wall-clock jitter): out of scope (#161 will use 10-30s prover invocations where jitter is negligible) ## Test plan - [x] `just chapel-build` — green (only known main()-in-library-mode warning) - [x] `just chapel-smoke` — `parallel-CAS winner: 3 (expected 3)` - [x] `just bench-chapel-mrr` — Idris2 + Agda → true across all 3 strategies - [x] 28 non-overriding provers unchanged (default args fill new fields) - [x] Commit GPG-signed ADR: `docs/decisions/2026-05-31-chapel-prover-spawn-hooks.md` covers mechanism + alternatives considered (env-extra list, regex extraction, function-pointer callback, native Chapel cwd param). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 20bb6ba commit a2aeb6a

7 files changed

Lines changed: 349 additions & 10 deletions

File tree

Justfile

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -530,9 +530,24 @@ chapel-test: build-chapel-ffi
530530
# parallelProofSearch / parallelProofSearchSpeculative against the
531531
# fixture corpus in tests/chapel_fixtures/. Emits CSV to stdout. Doc:
532532
# docs/bench/2026-05-30-chapel-mrr-baseline.md.
533+
#
534+
# IDRIS2_PREFIX is derived from `which idris2` because Idris2's prelude
535+
# resolution requires the env var to be set when invoking `--check`;
536+
# the per-prover cwd hook (#158) sets the subprocess working dir but
537+
# doesn't help if the parent never had the env var (see
538+
# src/chapel/parallel_proof_search.chpl ProverInfo[4]). The derived
539+
# prefix is `dirname(dirname(realpath(which idris2)))` — works for
540+
# both source builds and asdf-shimmed installs.
533541
bench-chapel-mrr:
534-
cd src/chapel && chpl -o bench_mrr bench_mrr.chpl && \
535-
./bench_mrr --verbose=false --timeout=10
542+
#!/usr/bin/env bash
543+
set -euo pipefail
544+
if command -v idris2 >/dev/null 2>&1 && [ -z "${IDRIS2_PREFIX:-}" ]; then
545+
export IDRIS2_PREFIX="$(dirname "$(dirname "$(readlink -f "$(command -v idris2)")")")"
546+
echo "# IDRIS2_PREFIX=$IDRIS2_PREFIX (derived from which idris2)" >&2
547+
fi
548+
cd src/chapel
549+
chpl -o bench_mrr bench_mrr.chpl
550+
./bench_mrr --verbose=false --timeout=10
536551
537552
# Rebuild Chapel 2.8.0 from source with CHPL_LIB_PIC=pic so that
538553
# `chpl --library --dynamic` can produce a shared-library form of the

docs/bench/2026-05-30-chapel-mrr-baseline.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,3 +106,34 @@ hand-off and the Wave-2 Idris2/Agda fixes above.
106106
## Raw data
107107

108108
5-run readings preserved at `docs/bench/2026-05-30-chapel-mrr-baseline.csv`.
109+
110+
## Update 2026-05-31 — Caveats 1 & 2 resolved (#158 + #159)
111+
112+
`ProverInfo` gained two optional spawn hooks: `cwd` (subprocess
113+
working directory, shell-wrapped to be coforall-safe) and
114+
`filenameOverride` (literal basename for provers that enforce
115+
module-name = filename). Registry entries for Idris2 and Agda set
116+
both; the bench fixture set gained `agda_trivial.agda`. The
117+
`bench-chapel-mrr` Justfile recipe now derives `IDRIS2_PREFIX` from
118+
`which idris2` so the parent shell doesn't need to export it.
119+
120+
Single-run readings (replace with 5-run medians in a follow-up):
121+
122+
| fixture | strategy | wallclock | winning prover |
123+
|---|---|---|---|
124+
| idris2_trivial | sequential | 1.970 | Idris2 |
125+
| idris2_trivial | parallel_bestof | 1.037 | Idris2 |
126+
| idris2_trivial | parallel_speculative| 1.032 | Idris2 |
127+
| agda_trivial | sequential | 0.101 | Agda |
128+
| agda_trivial | parallel_bestof | 0.751 | Agda |
129+
| agda_trivial | parallel_speculative| 0.450 | Agda |
130+
131+
Note the new Agda regime: `sequential` is fastest (0.101 s) because
132+
Agda sits at registry position 0 — the first prover tried — and
133+
succeeds immediately, so the parallel strategies pay coforall spawn
134+
overhead with no benefit. The trivial-goal regime observation from
135+
the original bench applies symmetrically here.
136+
137+
Wave-3 follow-ups still open: real-corpus speedup bench (#161,
138+
10-30 s prover invocations) and per-prover preempted/timeout/success
139+
telemetry (#162).

docs/decisions/2026-05-30-chapel-rehabilitation.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,13 @@ Wave 3+ (longer term):
128128
tactics consumed by the Chapel side); gated on a producer for
129129
the serialised record format.
130130

131+
Wave-3 follow-ups landed:
132+
- Per-prover `cwd` + `filenameOverride` hooks on `ProverInfo`
133+
(closes #158 / #159). Resolves caveats 1 + 2 of the Wave-1 MRR
134+
baseline — Idris2 and Agda fixtures now succeed end-to-end
135+
through `bench-chapel-mrr`. Mechanism + alternatives:
136+
[2026-05-31-chapel-prover-spawn-hooks.md](./2026-05-31-chapel-prover-spawn-hooks.md).
137+
131138
## Consequences
132139

133140
**Positive.**
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
<!-- SPDX-License-Identifier: MPL-2.0 -->
2+
3+
# 2026-05-31 — Per-prover spawn hooks: cwd + filenameOverride
4+
5+
ADR-style record of the mechanism added to `ProverInfo` and
6+
`tryProver` (`src/chapel/parallel_proof_search.chpl`) to handle
7+
provers whose CLI invocation requires per-prover state at spawn
8+
time. Closes issues #158 (Idris2 env hook) and #159 (Agda
9+
filename hook), both surfaced by the Wave-1 MRR baseline
10+
(`docs/bench/2026-05-30-chapel-mrr-baseline.md`).
11+
12+
## Status
13+
14+
Accepted. Implemented in the PR that introduces this ADR.
15+
16+
## Context
17+
18+
The L2.2 rehabilitation arc (#133 → PR #146) shipped the Chapel
19+
metalayer with a single uniform `tryProver(info, goal, …)`
20+
contract: every prover gets a temp file named
21+
`goal_<ProverName>_<nodeId>.<fileExt>`, spawned from the parent's
22+
CWD with the parent's environment. That contract held for ~22 of
23+
the 30 registered backends (Coq, Lean, the SMT row, ATPs, etc.)
24+
but two of the 30 — Idris2 and Agda — broke at spawn time:
25+
26+
- **Idris2** (`idris2 --check`) resolves its prelude relative to
27+
`IDRIS2_PREFIX` (or the install root) AND requires the source
28+
file to live inside the configured source directory. The
29+
Wave-1 invocation `idris2 --check /tmp/echidna-chapel/goal_Idris2_0.idr`
30+
failed with `Module Prelude not found` even when the parent
31+
shell had `IDRIS2_PREFIX` set, because the source file wasn't
32+
in the cwd Idris2 used as its source dir (the parent's cwd,
33+
not the temp dir).
34+
- **Agda** (`agda --safe`) requires the source-file basename
35+
to be a valid Agda identifier AND the module declaration
36+
inside the file to match the basename. `goal_Agda_0.agda`
37+
is rejected by the lexer (`in the name 0, the part 0 is not
38+
valid because it is a literal`); even after fixing the
39+
lexer issue, Agda then enforces module-name = file-name and
40+
rejects mismatched module declarations.
41+
42+
The Wave-1 baseline (`docs/bench/2026-05-30-chapel-mrr-baseline.md`)
43+
explicitly tracked these as caveats 1 + 2, deferred as Wave-2
44+
follow-up issues (`#158`/`#159`).
45+
46+
## Decision
47+
48+
**Two new optional fields on `ProverInfo`** with empty-string
49+
defaults that preserve the prior contract for the 28 provers
50+
that don't need per-prover spawn state:
51+
52+
```chapel
53+
record ProverInfo {
54+
var id: int;
55+
var name: string;
56+
// … existing fields …
57+
var cwd: string; // "" = inherit parent CWD
58+
var filenameOverride: string; // "" = goal_<name>_<nodeId>.<ext>
59+
}
60+
```
61+
62+
A custom `proc init(…, cwd = "", filenameOverride = "")` keeps
63+
all 30 existing registry call-sites compiling unchanged
64+
(default arguments fill the new fields). A zero-arg `proc init()`
65+
is also defined so `var provers: [0..29] ProverInfo` continues
66+
to default-construct each slot — without it the custom positional
67+
init would shadow Chapel's auto-generated zero-arg init and the
68+
array declaration would fail to compile.
69+
70+
`tryProver` consumes both fields:
71+
72+
- **`filenameOverride`** swaps the generic
73+
`goal_<name>_<nodeId>.<fileExt>` for a literal basename (with
74+
extension) when set. The default form remains locale-id-suffixed
75+
so non-overriding provers stay collision-free across locales in
76+
multi-locale runs.
77+
- **`cwd`** is implemented as a shell wrapper around the spawn:
78+
`sh -c "cd <cwd> && exec <executable> <args>"`. This is the
79+
thread-safe equivalent of `chdir()` — process-global `chdir`
80+
would race against other parallel spawn calls inside a
81+
`coforall`, but a per-subprocess `sh -c` keeps the directory
82+
change local. The parent's full environment (including
83+
`IDRIS2_PREFIX`, `IDRIS2_DATA_DIR`) is inherited regardless,
84+
per POSIX spawn defaults.
85+
86+
Registry entries updated:
87+
88+
- `provers[0]` (Agda) sets `cwd = "/tmp/echidna-chapel"` and
89+
`filenameOverride = "Trivial.agda"`. The fixture
90+
`tests/chapel_fixtures/agda_trivial.agda` declares
91+
`module Trivial where` to match.
92+
- `provers[4]` (Idris2) sets `cwd = "/tmp/echidna-chapel"` and
93+
`filenameOverride = "Trivial.idr"`. The fixture
94+
`tests/chapel_fixtures/idris2_trivial.idr` already declared
95+
`module Trivial`, so its content is unchanged.
96+
97+
The Justfile `bench-chapel-mrr` recipe gained a derived
98+
`IDRIS2_PREFIX` step: if the env var isn't set and `which idris2`
99+
resolves, the recipe sets it to `dirname(dirname(realpath(which idris2)))`.
100+
This means a user with idris2 on PATH no longer needs to remember
101+
to export the prefix manually for `just bench-chapel-mrr` to
102+
work end-to-end.
103+
104+
## Alternatives considered
105+
106+
1. **Per-prover env-extra list (key=value pairs)** — rejected as
107+
over-engineered. Chapel `spawn` inherits parent env by default;
108+
the only env hole was Idris2's `IDRIS2_PREFIX`, which the
109+
Justfile recipe now derives and exports parent-side. No prover
110+
in the current registry needs a per-spawn env override that
111+
isn't already in the parent.
112+
2. **Auto-extract module name from goal content** (regex
113+
`^module\s+(\w+)`) — rejected as fragile. Works for clean
114+
fixtures but breaks on goals containing comments-before-module,
115+
pragmas, or multi-module files. The explicit
116+
`filenameOverride` field keeps the registry honest about which
117+
provers have basename constraints.
118+
3. **Per-prover spawn callback (function pointer in the record)**
119+
rejected for now: Chapel records with function fields are
120+
awkward to construct in a literal-initialiser-heavy registry,
121+
and the two-field mechanism handles every concrete need
122+
surfaced by Wave-1. Revisit if a third spawn-time wart appears.
123+
4. **`POSIX_spawn`-style explicit cwd in Chapel `spawn`** — Chapel
124+
2.x `Subprocess.spawn` does not expose a `cwd` parameter at
125+
the time of writing (verified against the apt-shipped 2.3.0 and
126+
the source-built 2.8.0). The shell-wrapper workaround sidesteps
127+
the gap with no Chapel-version dependency.
128+
129+
## Consequences
130+
131+
**Positive.**
132+
133+
- `just bench-chapel-mrr` now shows `idris2_trivial → true` and
134+
`agda_trivial → true` across all three strategies, satisfying
135+
the explicit acceptance criteria for both #158 and #159.
136+
- The hook pattern is general — future provers with similar
137+
module-name or working-directory requirements (Lean 4 already
138+
works out-of-the-box but a future fixture might trigger its
139+
`Lake` workspace check; Isabelle has session-name resolution)
140+
can be wired by adding two field values to their registry
141+
entry, no `tryProver` changes.
142+
- The Wave-1 baseline document's caveats 1 + 2 (formerly tracked
143+
as Wave-2 follow-ups) are closed, leaving only caveat 3
144+
(sub-second wall-clock jitter, which is a measurement-method
145+
issue, not a metalayer defect).
146+
147+
**Negative.**
148+
149+
- Shell-wrapping every `cwd`-using prover costs one `fork+exec`
150+
for the `sh` itself per invocation. For 0.1-second prover runs
151+
this is ~5% overhead; for the 10-30 s real-corpus benchmark
152+
target (#161), it's negligible. If this becomes a hot path,
153+
switching to a direct-spawn cwd implementation (when Chapel
154+
exposes it) would remove the overhead.
155+
- `filenameOverride` couples the registry entry to fixture
156+
content (the fixture's `module X where` declaration must match
157+
the override). This is acceptable because the fixture corpus
158+
is small and version-controlled together with the registry;
159+
a real-corpus runner would need a different convention (or
160+
the auto-extract approach from "Alternatives" above).
161+
162+
**Net.** Two scoped one-line additions to `ProverInfo` close
163+
both acceptance-criterion-driven Wave-3 issues with no churn
164+
to the 28 unaffected provers; the shell-wrapper approach buys
165+
portability across Chapel 2.x runtimes; the Justfile derivation
166+
of `IDRIS2_PREFIX` removes the last user-side env-export
167+
ceremony.
168+
169+
## Toolchain version pinning
170+
171+
- `chpl 2.3.0` (CI) and `chpl 2.8.0` (local). The custom
172+
`proc init` syntax + `init this` invariant used here is
173+
stable across both.
174+
- `idris2 0.8.0` (local). `IDRIS2_PREFIX`-based prelude
175+
resolution is the documented mechanism; the source-directory
176+
check is `idris2 --check`-specific (a `package`-driven build
177+
doesn't have the same constraint).
178+
- `agda 2.6.3`. The module-name = filename rule is permanent
179+
Agda design (`Issue #1062` upstream).
180+
181+
These pins are re-stated in `chapel-ci.yml`, `Justfile`, and
182+
`docs/decisions/2026-05-30-chapel-rehabilitation.md`.

src/chapel/bench_mrr.chpl

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,10 +81,11 @@ proc benchOne(goal: string, fixtureName: string, allProvers: [] ProverInfo,
8181
proc main(): int {
8282
var allProvers = buildProverRegistry();
8383

84-
const fixtures: [0..2] Fixture = [
84+
const fixtures: [0..3] Fixture = [
8585
new Fixture("coq_trivial", fixtureDir + "/coq_trivial.v"),
8686
new Fixture("lean_trivial", fixtureDir + "/lean_trivial.lean"),
87-
new Fixture("idris2_trivial", fixtureDir + "/idris2_trivial.idr")
87+
new Fixture("idris2_trivial", fixtureDir + "/idris2_trivial.idr"),
88+
new Fixture("agda_trivial", fixtureDir + "/agda_trivial.agda")
8889
];
8990

9091
writeln("fixture,strategy,wallclock_s,success,winning_prover");

0 commit comments

Comments
 (0)