Skip to content

Commit 826d541

Browse files
test(aggregate): ideal Idris2 proof fixture + end-to-end fold test (#138)
`aggregate` folds external prover output (Agda/Idris2/Coq·Rocq/Lean/Isabelle/SMT) into a report, but had **no proof fixture** and no end-to-end fold test — only annotation/spec unit tests. `proven` has Idris2 files, but they are library property-tests, not detector-soundness proofs in `aggregate`'s shape. ## Changes - `tests/fixtures/proofs/UnsafeCodeSoundness.idr` — a genuine `idris2 --check`-clean, `%default total` proof (no escape hatches: no `believe_me`/`assert_total`/`idris_crash`/`postulate`, no metavariable holes) of the **UnsafeCode (PA001)** detector's **soundness and completeness**, carrying the `@name` and `@covers sound:category:UnsafeCode` annotations the folder reads. - `tests/aggregate_proof_fixture.rs` — folds the fixture via `aggregate::run` and asserts it is recognised as `Idris2`, classified `Closed` (no holes), hashed (non-repudiation), named from `@name`, and that the coverage parses to `Sound`/`Category`/`UnsafeCode`. Verified: `idris2 --check` exits 0; `cargo test --test aggregate_proof_fixture` passes (1 passed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 0d89586 commit 826d541

2 files changed

Lines changed: 126 additions & 0 deletions

File tree

tests/aggregate_proof_fixture.rs

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
// SPDX-License-Identifier: MPL-2.0
2+
// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
3+
//! End-to-end fixture test for `aggregate`'s prover-output folding.
4+
//!
5+
//! `aggregate` has unit tests for annotation/spec parsing, but no test that
6+
//! folds a *real* prover artifact end to end. This exercises that path against
7+
//! `tests/fixtures/proofs/UnsafeCodeSoundness.idr` — a genuine, `idris2
8+
//! --check`-clean, `%default total` Idris2 proof (no escape hatches) of the
9+
//! UnsafeCode (PA001) detector's soundness, carrying the `@name` / `@covers`
10+
//! annotations the folder reads. It asserts the artifact is recognised as
11+
//! Idris2, classified `Closed` (no holes), hashed, named, and that the
12+
//! `sound:category:UnsafeCode` coverage is parsed.
13+
14+
use panic_attack::aggregate::{
15+
run, AggregateConfig, CoverageKind, ProofClaim, ProofInput, ProofVerdict, Prover,
16+
};
17+
use std::path::PathBuf;
18+
19+
fn fixture() -> PathBuf {
20+
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/proofs/UnsafeCodeSoundness.idr")
21+
}
22+
23+
#[test]
24+
fn aggregate_folds_idris2_unsafecode_soundness_proof() {
25+
let path = fixture();
26+
assert!(path.exists(), "fixture missing: {}", path.display());
27+
28+
let report = run(AggregateConfig {
29+
proofs: vec![ProofInput {
30+
path,
31+
label: None,
32+
covers: Vec::new(),
33+
}],
34+
base_report: None,
35+
})
36+
.expect("aggregate should fold the Idris2 fixture");
37+
38+
assert_eq!(report.aggregated_proofs.len(), 1);
39+
let p = &report.aggregated_proofs[0];
40+
41+
// Recognised as an Idris2 artifact (by `.idr` extension).
42+
assert!(matches!(p.prover, Prover::Idris2), "prover: {:?}", p.prover);
43+
44+
// No escape hatches + a closure signal => Closed (not Holes / Indeterminate).
45+
assert!(
46+
matches!(p.verdict, ProofVerdict::Closed),
47+
"verdict: {:?}, holes: {:?}",
48+
p.verdict,
49+
p.holes
50+
);
51+
assert!(p.holes.is_empty(), "unexpected holes: {:?}", p.holes);
52+
53+
// Non-repudiation: the whole file was hashed (byte length recorded).
54+
assert!(p.bytes > 0, "no bytes hashed");
55+
56+
// Friendly name lifted from the in-file `@name "..."` annotation.
57+
assert!(
58+
p.friendly_name.contains("UnsafeCode"),
59+
"friendly name: {}",
60+
p.friendly_name
61+
);
62+
63+
// Coverage parsed from `@covers sound:category:UnsafeCode`.
64+
assert!(
65+
p.covers.iter().any(|c| matches!(c.claim, ProofClaim::Sound)
66+
&& matches!(c.kind, CoverageKind::Category)
67+
&& c.value == "UnsafeCode"),
68+
"covers: {:?}",
69+
p.covers
70+
);
71+
}
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
-- SPDX-License-Identifier: MPL-2.0
2+
-- SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
3+
-- @name "UnsafeCode detector soundness (Idris2)"
4+
-- @covers sound:category:UnsafeCode
5+
|||
6+
||| A closed Idris2 soundness witness for panic-attack's UnsafeCode (PA001)
7+
||| detector, in the exact shape `panic-attack aggregate` folds: no escape
8+
||| hatches, a total proof that a positive detection is backed by a genuine
9+
||| occurrence -- the detector raises no UnsafeCode finding on a token stream
10+
||| that contains no unsafe construct (no false positives).
11+
|||
12+
||| `aggregate` records this artifact by BLAKE3 hash and does NOT re-check it;
13+
||| the recorded verdict is conditioned on the Idris2 checker, whose input this
14+
||| file is. The `@covers sound:category:UnsafeCode` annotation marks any
15+
||| overlapping UnsafeCode finding in a base report as proof-Backed.
16+
module UnsafeCodeSoundness
17+
18+
%default total
19+
20+
||| A scanned token: ordinary code, or an unsafe construct (e.g. a Rust
21+
||| `unsafe` block) -- the minimal model the UnsafeCode detector reasons over.
22+
public export
23+
data Tok = Ordinary | Unsafe
24+
25+
||| The detector under study: positive on the first unsafe construct seen.
26+
||| Structurally the per-file first-hit scan the real analyzer performs.
27+
public export
28+
detect : List Tok -> Bool
29+
detect [] = False
30+
detect (Unsafe :: _) = True
31+
detect (Ordinary :: ts) = detect ts
32+
33+
||| A structural witness that an `Unsafe` token genuinely occurs in the stream.
34+
public export
35+
data Occurs : List Tok -> Type where
36+
Here : Occurs (Unsafe :: ts)
37+
There : Occurs ts -> Occurs (x :: ts)
38+
39+
||| SOUNDNESS (no false positives): every positive detection is backed by a
40+
||| real `Unsafe` occurrence -- exactly the `sound` claim that `aggregate`
41+
||| reconciles as `Backed` against an overlapping UnsafeCode finding.
42+
export
43+
detectSound : (ts : List Tok) -> detect ts = True -> Occurs ts
44+
detectSound (Unsafe :: _) _ = Here
45+
detectSound (Ordinary :: ts) prf = There (detectSound ts prf)
46+
detectSound [] prf = absurd prf
47+
48+
||| COMPLETENESS (no false negatives): a real occurrence forces a positive
49+
||| detection. Together with `detectSound` this pins detection as exactly
50+
||| "an unsafe construct occurs", with no hidden gap in either direction.
51+
export
52+
detectComplete : (ts : List Tok) -> Occurs ts -> detect ts = True
53+
detectComplete (Unsafe :: _) Here = Refl
54+
detectComplete (Unsafe :: _) (There _) = Refl
55+
detectComplete (Ordinary :: ts) (There o) = detectComplete ts o

0 commit comments

Comments
 (0)