Skip to content

Commit f679172

Browse files
feat(chapel): Wave-3 rehab — chpl 2.8.0 compat + decoupled FFI smoke (closes #29) (#31)
* feat(chapel): Wave-3 rehab — chpl 2.8.0 compat + decoupled FFI smoke (closes #29) Wave 3 of the echidna#146 Chapel rehabilitation arc. Chapel 2.8.0 compatibility: - DocudactylHPC.chpl: FileSystem.stat removed; switch to OS.POSIX.stat + struct_stat (3 sites). forall task-private intent made ndjsonWriter const-shadow; add `with (ref ndjsonWriter)` clause (matches existing begin-with-ref-timer pattern at line 201). - NdjsonManifest.chpl: string[range] and string.createCopyingBuffer became throws; wrap with try/catch preserving the existing "return empty on error" contract (4 sites). Decoupled FFI smoke (per echidna pattern): - src/chapel/smoke.chpl (46 lines) exercises ddac_version / ddac_crypto_sha256_name / ddac_init / ddac_free without pulling in the DocudactylHPC metalayer. - Justfile: check-smoke / build-smoke / run-smoke recipes; check-chapel uses --main-module DocudactylHPC to disambiguate the two proc main(). - hpc-ci.yml: new strict `build-smoke` job (no continue-on-error); CHAPEL_VERSION bumped 2.3.0 → 2.8.0. ADR docs/decisions/2026-05-30-docudactyl-chapel-rehab.md documents the scope adjustment: docudactyl's parallel-dispatch surface (forall idx in dynamic(...)) is embarrassingly-parallel with deterministic per-locale aggregation, NOT speculative like echidna's parallelProofSearchSpeculative. Echidna's ParallelSoundness.agda theorems do not apply; docudactyl's invariants (at-most-once / aggregation-commutativity / cache-coherence / checkpoint-resumability) are documented informally and deferred to a follow-up if formal mechanisation becomes load-bearing. check-abi job retains continue-on-error: true — Idris2 source-bootstrap on ubuntu-24.04 is environmental fragility, separate from docudactyl correctness. build-smoke is the strict gate the pattern actually wants. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): add missing runtime deps unblocking the HPC pipeline The pre-existing apt install step omitted `liblmdb-dev` even though `ffi/zig/build.zig:145` links against lmdb (and `libarchive` + `libcurl` similarly). All three most recent main runs of "HPC Build & Test" had failed at the Zig FFI build step with: error: unable to find dynamic system library 'lmdb' using strategy 'paths_first' Add the missing -dev (build) and runtime (libNNNt64) packages so the Zig FFI builds, the resulting .so loads in build-chapel + build-smoke, and the smoke gate this PR introduces actually runs (it was being SKIPPED via `needs: build-ffi`). Fix-at-source for the pipeline; no .zig source changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(zig): pre-existing Zig 0.15.2 compile errors (surfaced by smoke gate) Three pre-existing Zig 0.15.2 errors in ffi/zig/src/* that were masked by the missing liblmdb-dev apt package (fixed in the prior commit). With lmdb resolving, the actual compile runs and 3 errors fire: 1. financial_extract.zig:252 — `var j = i + 4;` never mutated. Fix: `const j = i + 4;`. Zig 0.15 enforces var-vs-const strictly. 2. gpu_ocr.zig:221 — `fn freeTesseract(_handle: ?*anyopaque) void` triggers "unused function parameter" in 0.15 even with `_` prefix. Fix: rename to `handle` and `_ = handle;` in the body. The body is intentionally empty (cleanup is delegated to ddac_free in the main FFI) — the explicit discard is the Zig 0.15 idiom. 3. hw_crypto.zig:117 — inline-asm output syntax changed in 0.15. Old: `: ({eax}) = "={eax}" -> u32,` (lvalue + arrow type) New: `: [_] "={eax}" (eax),` (named binding + lvalue) The local `eax` / `ebx` / `ecx` / `edx` vars are written back directly; same observable semantics, modernised syntax. These match [[reference_zig_015_breaking_changes]] (estate-wide Zig 0.14 → 0.15 migration debt). ast-check passes locally on all three files. Full build requires liblmdb-dev (now installed in CI). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent e0a9e9f commit f679172

9 files changed

Lines changed: 264 additions & 29 deletions

File tree

.github/workflows/hpc-ci.yml

Lines changed: 69 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,9 @@ permissions:
3232

3333
env:
3434
ZIG_VERSION: '0.15.2'
35-
CHAPEL_VERSION: '2.3.0'
35+
# Chapel 2.8.0 is required by docudactyl#29 — FileSystem.stat was
36+
# removed; code now uses OS.POSIX.stat. echidna#146 set the same floor.
37+
CHAPEL_VERSION: '2.8.0'
3638
IDRIS2_VERSION: '0.8.0'
3739

3840
jobs:
@@ -58,7 +60,8 @@ jobs:
5860
libpoppler-glib-dev libglib2.0-dev \
5961
libtesseract-dev libleptonica-dev \
6062
libavformat-dev libavcodec-dev libavutil-dev \
61-
libxml2-dev libgdal-dev libvips-dev
63+
libxml2-dev libgdal-dev libvips-dev \
64+
liblmdb-dev libarchive-dev libcurl4-openssl-dev
6265
6366
- name: Build FFI (ReleaseFast)
6467
run: cd ffi/zig && zig build -Doptimize=ReleaseFast
@@ -80,6 +83,69 @@ jobs:
8083
ffi/zig/zig-out/lib/libdocudactyl_ffi.a
8184
retention-days: 1
8285

86+
# ═══════════════════════════════════════════════════════════════════════════
87+
# Job 2a: FFI Smoke — decoupled from the metalayer (docudactyl#29 / echidna#146)
88+
# ─────────────────────────────────────────────────────────────────────────────
89+
# Builds and runs `src/chapel/smoke.chpl` linked against the Zig FFI library
90+
# WITHOUT pulling in the full DocudactylHPC metalayer. Strict gate: if the
91+
# FFI ABI surface or the chapel/Zig boundary regresses, this fails fast even
92+
# when the metalayer build is broken. (No continue-on-error.)
93+
# ═══════════════════════════════════════════════════════════════════════════
94+
build-smoke:
95+
name: Chapel FFI Smoke
96+
needs: build-ffi
97+
runs-on: ubuntu-24.04
98+
steps:
99+
- name: Checkout
100+
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
101+
102+
- name: Download FFI artifacts
103+
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
104+
with:
105+
name: ffi-libs
106+
path: ffi/zig/zig-out/lib/
107+
108+
- name: Install Chapel
109+
run: |
110+
CHPL_TARBALL="chapel-${CHAPEL_VERSION}-linux64-x86_64.tar.gz"
111+
CHPL_URL="https://github.com/chapel-lang/chapel/releases/download/${CHAPEL_VERSION}/${CHPL_TARBALL}"
112+
wget -q "${CHPL_URL}" -O "/tmp/${CHPL_TARBALL}"
113+
tar xzf "/tmp/${CHPL_TARBALL}" -C /opt/
114+
echo "CHPL_HOME=/opt/chapel-${CHAPEL_VERSION}" >> "$GITHUB_ENV"
115+
echo "/opt/chapel-${CHAPEL_VERSION}/bin/linux64-x86_64" >> "$GITHUB_PATH"
116+
117+
- name: Install C runtime libraries (FFI .so transitively pulls these)
118+
run: |
119+
sudo apt-get update -qq
120+
sudo apt-get install -y --no-install-recommends \
121+
libpoppler-glib8t64 libglib2.0-0t64 \
122+
libtesseract5 libleptonica-dev \
123+
libavformat60 libavcodec60 libavutil58 \
124+
libxml2 libgdal34t64 libvips42t64 \
125+
liblmdb0 libarchive13t64 libcurl4t64 \
126+
tesseract-ocr-eng
127+
128+
- name: Verify Chapel installation
129+
run: chpl --version
130+
131+
- name: Build FFI smoke binary
132+
run: |
133+
mkdir -p bin
134+
ABSPATH=$(cd ffi/zig/zig-out/lib && pwd)
135+
chmod +x ffi/zig/zig-out/lib/libdocudactyl_ffi.so* || true
136+
chpl src/chapel/smoke.chpl \
137+
src/chapel/FFIBridge.chpl \
138+
-o bin/docudactyl-smoke \
139+
-Lffi/zig/zig-out/lib -ldocudactyl_ffi \
140+
--ldflags="-Wl,-rpath,$ABSPATH"
141+
142+
- name: Run FFI smoke (init/version/free; no manifest required)
143+
run: |
144+
export LD_LIBRARY_PATH="ffi/zig/zig-out/lib:$LD_LIBRARY_PATH"
145+
OUT=$(bin/docudactyl-smoke)
146+
echo "$OUT"
147+
echo "$OUT" | grep -qE '^\[smoke\] PASS$'
148+
83149
# ═══════════════════════════════════════════════════════════════════════════
84150
# Job 2: Chapel HPC — Build + Smoke Test
85151
# ═══════════════════════════════════════════════════════════════════════════
@@ -115,6 +181,7 @@ jobs:
115181
libtesseract5 libleptonica-dev \
116182
libavformat60 libavcodec60 libavutil58 \
117183
libxml2 libgdal34t64 libvips42t64 \
184+
liblmdb0 libarchive13t64 libcurl4t64 \
118185
tesseract-ocr-eng
119186
120187
- name: Verify Chapel installation

Justfile

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -582,10 +582,36 @@ test-ffi:
582582
@echo "Running Zig FFI tests..."
583583
toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && cd {{zig_ffi}} && zig build test'
584584

585-
# Check Chapel parse validity
585+
# Check Chapel parse validity (metalayer + smoke; --main-module
586+
# disambiguates the two `proc main()` files)
586587
check-chapel:
587588
@echo "Checking Chapel syntax..."
588-
toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && chpl --parse-only {{chapel_src}}/DocudactylHPC.chpl {{chapel_src}}/*.chpl'
589+
toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && chpl --parse-only --main-module DocudactylHPC {{chapel_src}}/DocudactylHPC.chpl {{chapel_src}}/*.chpl'
590+
591+
# Decoupled FFI smoke (per docudactyl#29 / echidna#146 pattern).
592+
# Compiles just smoke.chpl + FFIBridge.chpl — does not pull in the
593+
# DocudactylHPC metalayer, so this gate stays green even if the
594+
# metalayer breaks. Build oracle for the FFI ABI alone.
595+
check-smoke:
596+
@echo "Parse-checking FFI smoke..."
597+
toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && chpl --no-codegen {{chapel_src}}/smoke.chpl {{chapel_src}}/FFIBridge.chpl'
598+
599+
# Build the FFI smoke binary (links against Zig FFI library).
600+
build-smoke: build-ffi
601+
@echo "Building FFI smoke binary..."
602+
@mkdir -p bin
603+
toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && \
604+
ABSPATH=$$(cd {{zig_ffi}}/zig-out/lib && pwd) && \
605+
chpl {{chapel_src}}/smoke.chpl \
606+
{{chapel_src}}/FFIBridge.chpl \
607+
-o bin/docudactyl-smoke \
608+
-L{{zig_ffi}}/zig-out/lib -ldocudactyl_ffi \
609+
--ldflags="-Wl,-rpath,$$ABSPATH"'
610+
611+
# Run the FFI smoke (init/version/free without manifest or full metalayer).
612+
run-smoke: build-smoke
613+
@echo "Running FFI smoke..."
614+
bin/docudactyl-smoke
589615

590616
# Check all HPC C library dependencies and versions
591617
deps-check:
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# ADR — Docudactyl Chapel Rehabilitation (Wave 3 of the echidna#146 arc)
2+
3+
- Status: Accepted
4+
- Date: 2026-05-30
5+
- Closes: [docudactyl#29](https://github.com/hyperpolymath/docudactyl/issues/29)
6+
- Wave 1 reference: [echidna#146](https://github.com/hyperpolymath/echidna/pull/146)
7+
- Wave 2 tracker: [proven#126](https://github.com/hyperpolymath/proven/issues/126)
8+
- SPDX-License-Identifier: MPL-2.0
9+
10+
## Context
11+
12+
`docudactyl` is a distributed document-processing HPC engine for British-Library-scale corpora — 170M+ items dispatched across Chapel locales, parsing via a Zig FFI bridge to C parser libraries (poppler / tesseract / ffmpeg / libxml2 / gdal / vips). The Chapel framework lives at `src/chapel/` (11 modules, ~2,500 lines).
13+
14+
`docudactyl#29` was filed alongside `echidna#146` (the Wave-1 Chapel metalayer rehabilitation) and `proven#126` (the Wave-2 follow-on). Each Wave-N issue applies the same shape: rewrite for chpl 2.8.0 compatibility, decouple the FFI smoke from the metalayer, flip CI to strict, and decide whether the Wave-1 `ParallelSoundness.agda` invariants apply to this repo's parallel-dispatch surface.
15+
16+
## Decision
17+
18+
Port the echidna#146 pattern with **two scope adjustments**:
19+
20+
1. **No `ParallelSoundness.agda` port.** Docudactyl's parallel-dispatch surface is `forall idx in dynamic(docEntries.domain, chunkSize)` in `DocudactylHPC.chpl` — embarrassingly-parallel work distribution with deterministic per-locale-slot aggregation, not speculative search. Echidna's three theorems (first-success-wins soundness, completeness under retry, cancellation safety) are inapplicable. Docudactyl's relevant invariants — at-most-once under checkpoint resume, aggregation commutativity, L1/L2 cache coherence — are documented below and deferred to a follow-up Agda module if formal proofs become load-bearing.
21+
22+
2. **The CI `check-abi` job retains `continue-on-error: true`.** This is documented as environmental fragility (Idris2 source bootstrap on ubuntu-24.04, not docudactyl correctness). The new `build-smoke` job is the strict gate the echidna pattern actually wants.
23+
24+
## What landed
25+
26+
### Chapel 2.8.0 compatibility fixes
27+
28+
`chpl --no-codegen src/chapel/DocudactylHPC.chpl src/chapel/*.chpl` was failing with three errors before this PR; all three are now resolved. The metalayer compiles clean on chpl 2.8.0 (LLVM-built local toolchain) with only known unstable-API warnings remaining (`_pad` symbol prefixes; `dmapped`; `DynamicIters`; `string.c_str()`; cross-locale `c_addrOf`).
29+
30+
| Site | Pre-state | Fix |
31+
|---|---|---|
32+
| `DocudactylHPC.chpl:303, 311, 373` | `FileSystem.stat(string)` removed in chpl 2.8.0 | `use OS.POSIX;` + `struct_stat` + `stat(path.c_str(), c_ptrTo(sb))`; field reads via `sb.st_mtim.tv_sec` / `sb.st_size` |
33+
| `DocudactylHPC.chpl:211` | `forall` task-private intent made outer `var ndjsonWriter` const-shadow; `proc ref writeResult` rejected the const actual | `forall … with (ref ndjsonWriter) { … }` (matches the `begin with (ref timer)` pattern already at line 201) |
34+
| `NdjsonManifest.chpl:91` | `string[range]` slicing now `throws` (UTF-8 boundary check) | `try { return line[range]; } catch { return ""; }` — preserves the existing "return empty on error" contract |
35+
| `NdjsonManifest.chpl:177, 182` | `string.createCopyingBuffer(c_ptrConst(c_char))` now `throws` | `try { … } catch { }` around each optional field — the SHA-256 / title fields are skipped in the rare malformed-UTF-8 case so the `forall` task is not torn down |
36+
37+
### Decoupled FFI smoke
38+
39+
`src/chapel/smoke.chpl` (46 lines) exercises `ddac_version` / `ddac_crypto_sha256_name` / `ddac_init` / `ddac_free` against the FFI bridge **without** pulling in the full `DocudactylHPC` metalayer. This is the echidna#146 invariant: a green smoke proves the C ABI compiles + links + returns sane data even when the metalayer is broken, so regression localisation is fast.
40+
41+
The smoke is wired three places:
42+
43+
- `Justfile` recipes: `check-smoke` (parse-only), `build-smoke` (binary), `run-smoke` (executes the binary)
44+
- `.github/workflows/hpc-ci.yml` job `build-smoke`: depends on `build-ffi`, strict (no `continue-on-error`), greps for `^\[smoke\] PASS$` in the binary output
45+
- `Justfile :: check-chapel` now uses `--main-module DocudactylHPC` to disambiguate the two `proc main()` files (DocudactylHPC + smoke) when globbing `src/chapel/*.chpl`
46+
47+
### CI floor bumped
48+
49+
`CHAPEL_VERSION: '2.3.0'``'2.8.0'` in `hpc-ci.yml`. The OS.POSIX migration is the proximate cause; the wider justification is that 2.8.0 is the version the rehabilitation pattern was validated against in Wave 1.
50+
51+
## Parallel-dispatch invariants (informal — deferred to follow-up if formalised)
52+
53+
These document the invariants the docudactyl `forall idx in dynamic(...)` loop relies on. They are NOT proved here; they are recorded so a future Agda module has a target.
54+
55+
1. **At-most-once** — every document index in `docEntries.domain` is processed at most once. Currently enforced per-locale by `isAlreadyProcessed(idx)` (Checkpoint.chpl) gating on `recordCheckpoint`. Caveat: cross-locale resume after a topology change (e.g. 64 → 128 locales) is NOT global-atomic.
56+
2. **Abort-bounded** — beyond a failure-rate threshold (FaultHandler.chpl), the loop short-circuits; unprocessed indices are NOT silently dropped, they remain unprocessed and observable via the report.
57+
3. **Aggregation commutativity**`accumulate(result)` writes only to `perLocaleStats[here.id]` (per-locale slot, no cross-task race); `computeGlobal` reduces by integer addition, which is commutative. SATISFIED by construction.
58+
4. **Cache coherence (L1)**`(path, mtime, size)`-keyed LMDB lookups in the per-locale L1 are read-then-write within a single task; no concurrent writers to the same key.
59+
5. **Cache coherence (L2)** — Dragonfly L2 is shared across locales and keyed by SHA-256; concurrent `store` calls for the same key are tolerated because the value is deterministic (same SHA-256 implies same parse result). Not formally proven.
60+
6. **Checkpoint resumability** — resuming from a `recordCheckpoint(idx)` snapshot reproduces the same `succeededDocs / failedDocs` global stats modulo per-locale ordering. Stats are aggregated by addition, so order does not matter.
61+
7. **Content-type determinism** — magic-byte detection (`conduit.content_kind`) is deterministic per input file.
62+
63+
A follow-up issue should be filed if these need formal mechanisation (Agda module path `proofs/agda/DistributedAggregationInvariants.agda`); echidna's `ParallelSoundness.agda` cannot be imported unchanged.
64+
65+
## Wave-2 gate
66+
67+
Per `docudactyl#29`, this PR was deferred until `proven#126` (Wave 2) "lands". At time of writing `proven#126` is still OPEN. The owner-authorised exception: the echidna#146 + proven#135 (binding-tier-1 detachable harness) pair has stabilised the rehabilitation shape; a docudactyl rehab now does not risk pattern-divergence. Should Wave 2 settle on a different shape, this ADR is the supersedable surface — the smoke target and CI gate are the load-bearing pieces that would change.
68+
69+
## Consequences
70+
71+
- `chpl 2.8.0` is now the floor in CI; running on `2.3.0` will fail the OS.POSIX import.
72+
- The metalayer build remains the same shape; only the three stat sites + `with (ref ndjsonWriter)` clause + the two throws-wrappers in NdjsonManifest differ.
73+
- The new `build-smoke` job runs in parallel with `build-chapel` (both `needs: build-ffi`), so wallclock CI time does not regress.
74+
- `check-abi` still flagged fragile; not changed in this PR.
75+
76+
## References
77+
78+
- [echidna#146](https://github.com/hyperpolymath/echidna/pull/146) — Wave-1 metalayer rehabilitation; canonical pattern source
79+
- [proven#126](https://github.com/hyperpolymath/proven/issues/126) — Wave-2 tracker
80+
- [docudactyl#29](https://github.com/hyperpolymath/docudactyl/issues/29) — this issue
81+
- chpl 2.8.0 `OS.POSIX.stat``/usr/share/chapel/2.8/modules/standard/OS.chpl:929-965`

ffi/zig/src/financial_extract.zig

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -249,7 +249,7 @@ fn extractCurrencyAmounts(text: []const u8, result: *FinancialResult) void {
249249
}
250250
}
251251
if (is_currency) {
252-
var j = i + 4;
252+
const j = i + 4;
253253
if (j < text.len and isDigit(text[j])) {
254254
const parsed = parseDecimal(text, j);
255255
if (parsed.end > j and result.amount_count < MAX_ITEMS) {

ffi/zig/src/gpu_ocr.zig

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,8 @@ fn initTesseract() ?*anyopaque {
218218
return null; // Placeholder — actual Tesseract handle comes from ddac_init()
219219
}
220220

221-
fn freeTesseract(_handle: ?*anyopaque) void {
221+
fn freeTesseract(handle: ?*anyopaque) void {
222+
_ = handle;
222223
// Cleanup handled by ddac_free() in the main FFI
223224
}
224225

ffi/zig/src/hw_crypto.zig

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -114,10 +114,10 @@ fn cpuid(leaf: u32, subleaf: u32) CpuidResult {
114114
var edx: u32 = undefined;
115115

116116
asm volatile ("cpuid"
117-
: ({eax}) = "={eax}" -> u32,
118-
({ebx}) = "={ebx}" -> u32,
119-
({ecx}) = "={ecx}" -> u32,
120-
({edx}) = "={edx}" -> u32,
117+
: [_] "={eax}" (eax),
118+
[_] "={ebx}" (ebx),
119+
[_] "={ecx}" (ecx),
120+
[_] "={edx}" (edx),
121121
: [_] "{eax}" (leaf),
122122
[_] "{ecx}" (subleaf),
123123
);

src/chapel/DocudactylHPC.chpl

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ use Version;
3333
use FileSystem;
3434
use IO;
3535
use Path;
36+
use OS.POSIX;
3637

3738
// Minimum Chapel version required (2.7.0 for --parse-only, begin ref intent)
3839
param DOCUDACTYL_MIN_CHAPEL_MAJOR = 2;
@@ -207,7 +208,7 @@ proc main() throws {
207208
const fmtCode = outputFormatCode();
208209
const resultSize: c_size_t = 952; // sizeof(ddac_parse_result_t)
209210

210-
forall idx in dynamic(docEntries.domain, chunkSize) {
211+
forall idx in dynamic(docEntries.domain, chunkSize) with (ref ndjsonWriter) {
211212
// Each task gets its own FFI handle (owns Tesseract/GDAL contexts)
212213
var handle = ddac_init();
213214
defer ddac_free(handle);
@@ -300,17 +301,21 @@ proc main() throws {
300301
// Use conduit file_size, but still need mtime from stat()
301302
fsize = conduitResult.file_size;
302303
try {
303-
var finfo = stat(inputPath);
304-
mtime = finfo.mtime: int(64);
304+
var sb: struct_stat;
305+
if stat(inputPath.c_str(), c_ptrTo(sb)) == 0 {
306+
mtime = sb.st_mtim.tv_sec: int(64);
307+
}
305308
} catch {
306309
// stat failed — proceed with normal parse
307310
}
308311
} else {
309312
// Fall back to stat() for plain manifests without conduit
310313
try {
311-
var finfo = stat(inputPath);
312-
mtime = finfo.mtime: int(64);
313-
fsize = finfo.size: int(64);
314+
var sb: struct_stat;
315+
if stat(inputPath.c_str(), c_ptrTo(sb)) == 0 {
316+
mtime = sb.st_mtim.tv_sec: int(64);
317+
fsize = sb.st_size: int(64);
318+
}
314319
} catch {
315320
// stat failed — proceed with normal parse
316321
}
@@ -370,9 +375,11 @@ proc main() throws {
370375
fsize = entry.size;
371376
} else {
372377
try {
373-
var finfo = stat(inputPath);
374-
mtime = finfo.mtime: int(64);
375-
fsize = finfo.size: int(64);
378+
var sb: struct_stat;
379+
if stat(inputPath.c_str(), c_ptrTo(sb)) == 0 {
380+
mtime = sb.st_mtim.tv_sec: int(64);
381+
fsize = sb.st_size: int(64);
382+
}
376383
} catch {
377384
// stat failed — skip caching
378385
}

src/chapel/NdjsonManifest.chpl

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,15 @@ module NdjsonManifest {
8787
valEnd += 1;
8888
}
8989

90-
if valEnd > valStart then
91-
return line[valStart..#(valEnd - valStart)];
92-
else
90+
if valEnd > valStart {
91+
try {
92+
return line[valStart..#(valEnd - valStart)];
93+
} catch {
94+
return "";
95+
}
96+
} else {
9397
return "";
98+
}
9499
}
95100

96101
/** Extract an integer value for a given key from a JSON line.
@@ -173,15 +178,20 @@ module NdjsonManifest {
173178

174179
line += ',"parse_time_ms":' + parseTimeMs:string;
175180

176-
// Include SHA-256 if present
177-
const sha = string.createCopyingBuffer(result.sha256: c_ptrConst(c_char));
178-
if sha.size > 0 then
179-
line += ',"sha256":' + jsonEscapeString(sha);
181+
// Include SHA-256 if present (FFI strings are well-formed in practice;
182+
// throws on UTF-8 boundary errors are skipped to preserve forall task)
183+
try {
184+
const sha = string.createCopyingBuffer(result.sha256: c_ptrConst(c_char));
185+
if sha.size > 0 then
186+
line += ',"sha256":' + jsonEscapeString(sha);
187+
} catch { }
180188

181189
// Include title if present
182-
const title = string.createCopyingBuffer(result.title: c_ptrConst(c_char));
183-
if title.size > 0 then
184-
line += ',"title":' + jsonEscapeString(title);
190+
try {
191+
const title = string.createCopyingBuffer(result.title: c_ptrConst(c_char));
192+
if title.size > 0 then
193+
line += ',"title":' + jsonEscapeString(title);
194+
} catch { }
185195

186196
line += "}";
187197

0 commit comments

Comments
 (0)