Skip to content

Commit 8041b27

Browse files
hyperpolymathclaude
andcommitted
ci: add Idris2 proof gate; make Idris test suites fail-capable
Until now ZERO Idris code ran in CI — the entire proof ledger (docs/PROOFS.adoc, 19 %default-total modules) was enforced only on developer machines, and three of the four Idris test suites could not fail (printed FAIL but always exited 0). - .github/workflows/idris2.yml (new): digest-pinned ghcr.io/stefan-hoeck/idris2-pack container (ships exactly Idris 2 v0.8.0) builds ochrance.ipkg under --total, runs the three Idris suites, and runs tests/ffi/run_ffi_test.sh (Idris->Zig runtime BLAKE3 proof) with Zig 0.15.2. Dry-run verified locally in the same container image via podman before committing. - ParserTests: failure paths now route through failWith (print + exit 1) - PropertyTests: exit 1 when totalFailed > 0 (keeps full report) - IntegrationTests: testCase fail-fast (exit 1 on first Fail) Canary-verified: injected failure exits 1, clean run exits 0; all three suites currently pass (47/47 properties, 0 FAILs). - e2e.yml: drop the structural no-op 'Integration tests' job (ran cargo test with no Cargo.toml and looped over nonexistent .sh files — always green, gated nothing; not a required check, verified via API) - e2e_test.sh: fix vacuous shared-library grep (addSharedLibrary only matched a comment after the 0.15 migration; now greps 'linkage = .dynamic') Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 1d857db commit 8041b27

6 files changed

Lines changed: 107 additions & 42 deletions

File tree

.github/workflows/e2e.yml

Lines changed: 5 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# SPDX-License-Identifier: MPL-2.0
2+
# The former "Integration tests" job here was a structural no-op: it ran
3+
# `cargo test` in a repo with no Cargo.toml and looped over
4+
# tests/integration/*.sh which never existed — always green, gating nothing.
5+
# Real integration coverage now lives in idris2.yml (Idris suites + the
6+
# Idris->Zig FFI runtime test) and zig-ffi.yml (C-ABI link + Zig tests).
27
name: E2E and Integration Tests
38
on:
49
push: {branches: [main]}
@@ -14,19 +19,3 @@ jobs:
1419
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
1520
- name: Run E2E tests
1621
run: bash tests/e2e_test.sh
17-
integration:
18-
name: Integration tests
19-
runs-on: ubuntu-latest
20-
timeout-minutes: 15
21-
steps:
22-
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
23-
- name: Install Rust
24-
uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
25-
- name: Run integration tests
26-
run: |
27-
if [ -f "Cargo.toml" ]; then
28-
cargo test --test '*' 2>/dev/null || echo "No integration tests"
29-
fi
30-
if ls tests/integration/*.sh 2>/dev/null | head -1 | grep -q .; then
31-
for f in tests/integration/*.sh; do bash "$f"; done
32-
fi

.github/workflows/idris2.yml

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Idris2 proof + test gate.
3+
#
4+
# This is the job that makes the proof ledger (docs/PROOFS.adoc) CI-enforced:
5+
# `idris2 --build ochrance.ipkg` type-checks all core modules under
6+
# `%default total` (opts = "--total" in the ipkg), so every theorem in
7+
# ochrance-core either machine-checks here or the build goes red. It then
8+
# runs the three Idris test suites and the Idris->Zig runtime FFI test
9+
# (tests/ffi/run_ffi_test.sh), which proves the production Merkle path
10+
# computes real BLAKE3 across the %foreign boundary.
11+
#
12+
# Toolchain: ghcr.io/stefan-hoeck/idris2-pack ships Idris 2 v0.8.0 (the
13+
# exact version this repo requires) + chezscheme/gcc/make. Digest-pinned;
14+
# bump deliberately, not via :latest drift.
15+
name: Idris2
16+
17+
on:
18+
push:
19+
branches: [main, master]
20+
pull_request:
21+
22+
concurrency:
23+
group: ${{ github.workflow }}-${{ github.ref }}
24+
cancel-in-progress: true
25+
26+
permissions:
27+
contents: read
28+
29+
jobs:
30+
idris2:
31+
name: Idris2 proofs + tests
32+
runs-on: ubuntu-latest
33+
timeout-minutes: 30
34+
container:
35+
image: ghcr.io/stefan-hoeck/idris2-pack@sha256:7116e4fc2c68ff6345139a55d908e9dd1c7acabaf197c32e709deae3448dd7d7 # :latest @ 2026-07-01, Idris 2 v0.8.0
36+
steps:
37+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
38+
- name: Set up Zig 0.15.2 (for the FFI runtime test)
39+
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2.2.1
40+
with:
41+
version: 0.15.2
42+
- name: Build core (type-checks all modules under --total)
43+
run: idris2 --build ochrance.ipkg
44+
- name: Install package (needed by the test ipkgs)
45+
run: idris2 --install ochrance.ipkg
46+
- name: A2ML parser tests
47+
run: |
48+
idris2 --build tests/A2ML/tests.ipkg
49+
./tests/A2ML/build/exec/a2ml-tests
50+
- name: Property tests
51+
run: |
52+
idris2 --build tests/property/tests.ipkg
53+
./tests/property/build/exec/property-tests
54+
- name: Integration tests
55+
run: |
56+
idris2 --build tests/integration/tests.ipkg
57+
./tests/integration/build/exec/integration-tests
58+
- name: Idris -> Zig FFI runtime test (real BLAKE3 across %foreign)
59+
run: bash tests/ffi/run_ffi_test.sh

tests/A2ML/ParserTests.idr

Lines changed: 32 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -12,24 +12,32 @@ import Ochrance.A2ML.Lexer
1212
import Ochrance.A2ML.Parser
1313
import Ochrance.A2ML.Serializer
1414
import Ochrance.A2ML.Types
15+
import System
1516

1617
%default total
1718

1819
--------------------------------------------------------------------------------
1920
-- Test Helpers
2021
--------------------------------------------------------------------------------
2122

23+
||| Report a failure and exit nonzero, so CI actually gates on this suite
24+
||| (fail-fast: the first failure ends the run with exit code 1).
25+
failWith : String -> IO ()
26+
failWith msg = do
27+
putStrLn msg
28+
exitFailure
29+
2230
||| Assert that a value is Right
2331
assertRight : Show e => Either e a -> IO ()
24-
assertRight (Left err) = putStrLn ("FAIL: " ++ show err)
32+
assertRight (Left err) = failWith ("FAIL: " ++ show err)
2533
assertRight (Right _) = pure ()
2634

2735
||| Assert two values are equal
2836
assertEqual : (Eq a, Show a) => a -> a -> IO ()
2937
assertEqual expected actual =
3038
if expected == actual
3139
then pure ()
32-
else putStrLn ("FAIL: Expected " ++ show expected ++ ", got " ++ show actual)
40+
else failWith ("FAIL: Expected " ++ show expected ++ ", got " ++ show actual)
3341

3442
||| Test case wrapper
3543
testCase : String -> IO () -> IO ()
@@ -107,27 +115,27 @@ lexerTests = do
107115
testCase "Lex empty string" $ do
108116
case lex "" of
109117
Right tokens => assertEqual [EOF] tokens
110-
Left err => putStrLn ("FAIL: " ++ show err)
118+
Left err => failWith ("FAIL: " ++ show err)
111119

112120
testCase "Lex minimal manifest" $ do
113121
case lex minimalManifest of
114122
Right tokens => pure () -- Success if no error
115-
Left err => putStrLn ("FAIL: " ++ show err)
123+
Left err => failWith ("FAIL: " ++ show err)
116124

117125
testCase "Lex with attestation" $ do
118126
case lex attestedManifest of
119127
Right tokens => pure ()
120-
Left err => putStrLn ("FAIL: " ++ show err)
128+
Left err => failWith ("FAIL: " ++ show err)
121129

122130
testCase "Lex with policy" $ do
123131
case lex policyManifest of
124132
Right tokens => pure ()
125-
Left err => putStrLn ("FAIL: " ++ show err)
133+
Left err => failWith ("FAIL: " ++ show err)
126134

127135
testCase "Lex rejects invalid characters" $ do
128136
case lex "@manifest { version = \x00 }" of
129137
Left _ => pure () -- Should fail
130-
Right _ => putStrLn "FAIL: Should reject null bytes"
138+
Right _ => failWith "FAIL: Should reject null bytes"
131139

132140
--------------------------------------------------------------------------------
133141
-- Parser Tests
@@ -145,44 +153,44 @@ parserTests = do
145153
Right manifest => do
146154
assertEqual "0.1.0" manifest.manifestData.version
147155
assertEqual "test" manifest.manifestData.subsystem
148-
Left err => putStrLn ("FAIL: " ++ show err)
149-
Left err => putStrLn ("FAIL lex: " ++ show err)
156+
Left err => failWith ("FAIL: " ++ show err)
157+
Left err => failWith ("FAIL lex: " ++ show err)
150158

151159
testCase "Parse attested manifest" $ do
152160
case lex attestedManifest of
153161
Right tokens =>
154162
case parse tokens of
155163
Right manifest => do
156164
case manifest.attestation of
157-
Nothing => putStrLn "FAIL: Expected attestation"
165+
Nothing => failWith "FAIL: Expected attestation"
158166
Just att => do
159167
assertEqual "test-witness" att.witness
160168
assertEqual "sig123" att.signature
161-
Left err => putStrLn ("FAIL: " ++ show err)
162-
Left err => putStrLn ("FAIL lex: " ++ show err)
169+
Left err => failWith ("FAIL: " ++ show err)
170+
Left err => failWith ("FAIL lex: " ++ show err)
163171

164172
testCase "Parse policy manifest" $ do
165173
case lex policyManifest of
166174
Right tokens =>
167175
case parse tokens of
168176
Right manifest => do
169177
case manifest.policy of
170-
Nothing => putStrLn "FAIL: Expected policy"
178+
Nothing => failWith "FAIL: Expected policy"
171179
Just pol => do
172180
assertEqual Checked pol.mode
173181
case pol.maxAge of
174-
Nothing => putStrLn "FAIL: Expected max_age"
182+
Nothing => failWith "FAIL: Expected max_age"
175183
Just age => assertEqual 3600 age
176-
Left err => putStrLn ("FAIL: " ++ show err)
177-
Left err => putStrLn ("FAIL lex: " ++ show err)
184+
Left err => failWith ("FAIL: " ++ show err)
185+
Left err => failWith ("FAIL lex: " ++ show err)
178186

179187
testCase "Parse rejects duplicate sections" $ do
180188
let duplicateManifest = minimalManifest ++ "\n@manifest { version = \"0.2.0\" }\n"
181189
case lex duplicateManifest of
182190
Right tokens =>
183191
case parse tokens of
184192
Left _ => pure () -- Should fail
185-
Right _ => putStrLn "FAIL: Should reject duplicate sections"
193+
Right _ => failWith "FAIL: Should reject duplicate sections"
186194
Left _ => pure () -- Lex error also acceptable
187195

188196
--------------------------------------------------------------------------------
@@ -206,10 +214,10 @@ roundtripTests = do
206214
Right manifest2 => do
207215
assertEqual manifest1.manifestData.version manifest2.manifestData.version
208216
assertEqual manifest1.manifestData.subsystem manifest2.manifestData.subsystem
209-
Left err => putStrLn ("FAIL parse2: " ++ show err)
210-
Left err => putStrLn ("FAIL lex2: " ++ show err)
211-
Left err => putStrLn ("FAIL parse1: " ++ show err)
212-
Left err => putStrLn ("FAIL lex1: " ++ show err)
217+
Left err => failWith ("FAIL parse2: " ++ show err)
218+
Left err => failWith ("FAIL lex2: " ++ show err)
219+
Left err => failWith ("FAIL parse1: " ++ show err)
220+
Left err => failWith ("FAIL lex1: " ++ show err)
213221

214222
--------------------------------------------------------------------------------
215223
-- Error Handling Tests
@@ -226,7 +234,7 @@ errorHandlingTests = do
226234
Right tokens =>
227235
case parse tokens of
228236
Left _ => pure () -- Should fail
229-
Right _ => putStrLn "FAIL: Should require version field"
237+
Right _ => failWith "FAIL: Should require version field"
230238
Left _ => pure () -- Lex error also acceptable
231239

232240
testCase "Invalid hash format" $ do
@@ -235,14 +243,14 @@ errorHandlingTests = do
235243
Right tokens =>
236244
case parse tokens of
237245
Left _ => pure () -- Should fail
238-
Right _ => putStrLn "FAIL: Should reject invalid hash format"
246+
Right _ => failWith "FAIL: Should reject invalid hash format"
239247
Left _ => pure () -- Lex error also acceptable
240248

241249
testCase "Unterminated string" $ do
242250
let invalid = "@manifest { version = \"unterminated }"
243251
case lex invalid of
244252
Left _ => pure () -- Should fail at lex stage
245-
Right _ => putStrLn "FAIL: Should reject unterminated string"
253+
Right _ => failWith "FAIL: Should reject unterminated string"
246254

247255
--------------------------------------------------------------------------------
248256
-- Main Test Runner

tests/e2e_test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ assert "build.zig configures library" \
9090
assert "build.zig emits libochrance.so (no stale 'ochrance-shared' name)" \
9191
"$(! grep -q "ochrance-shared" "$BASE/ffi/zig/build.zig" && echo 0 || echo 1)"
9292
assert "build.zig builds a shared library" \
93-
"$(grep -q "addSharedLibrary" "$BASE/ffi/zig/build.zig" && echo 0 || echo 1)"
93+
"$(grep -q 'linkage = .dynamic' "$BASE/ffi/zig/build.zig" && echo 0 || echo 1)"
9494

9595
# ================================================================
9696
# E2E: Test coverage completeness

tests/integration/IntegrationTests.idr

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99

1010
module IntegrationTests
1111

12+
import System
1213
import Data.Vect
1314
import Ochrance.A2ML.Types
1415
import Ochrance.A2ML.Lexer
@@ -35,12 +36,16 @@ Show TestResult where
3536
show Pass = "PASS"
3637
show (Fail msg) = "FAIL: " ++ msg
3738

38-
||| Test case wrapper
39+
||| Test case wrapper. Fail-fast: a failing case ends the run with exit
40+
||| code 1, so CI actually gates on this suite instead of always going green.
3941
testCase : String -> IO TestResult -> IO ()
4042
testCase name test = do
4143
putStr (name ++ " ... ")
4244
result <- test
4345
putStrLn (show result)
46+
case result of
47+
Fail _ => exitFailure
48+
Pass => pure ()
4449

4550
||| Create a test filesystem state
4651
createTestFS : Nat -> FSState

tests/property/PropertyTests.idr

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414

1515
module PropertyTests
1616

17+
import System
1718
import Data.Vect
1819
import Data.List
1920
import Data.Bits
@@ -520,3 +521,6 @@ main = do
520521
++ show totalCount ++ " passed, "
521522
++ show totalFailed ++ " failed")
522523
putStrLn "==========================================="
524+
525+
-- Gate: nonzero exit on any failure, so CI actually enforces this suite
526+
when (totalFailed > 0) exitFailure

0 commit comments

Comments
 (0)