Skip to content

Commit 44c97cb

Browse files
claudehyperpolymath
authored andcommitted
proof(conformance): fix corpus to match grammar; wire conformance gate
Wire the parser conformance corpus into CI and fix the divergences found when it was first run end-to-end against the real parser. Root causes fixed: * run_conformance.sh invoked a non-existent entry point (`Phronesis.CLI.parse`), so the corpus had never actually executed. Replaced with `mix run --no-start -e` calling the real `Phronesis.parse/1` (Lexer.tokenize |> Parser.parse). `--no-start` avoids booting the ra consensus supervision tree just to parse. * Four "valid" samples were written against a syntax the language does not have (per spec/grammar.ebnf v0.2.0, whose source is parser.ex): - v02/v03/v07 used `IF` as the policy *body*; the grammar requires `logical_expr THEN action`, with `IF` only as a conditional *action* after THEN. They also dropped the required `:` on PRIORITY:/EXPIRES:/CREATED_BY:, used atoms (:valid), and string metadata. - v05 used `x = expr` let-bindings, for which there is no production. Rewritten to the real grammar while preserving each file's intent (conditional action, AND/OR connectives, IMPORT + module-qualified call, nested conditional action). Note: parenthesised *logical* expressions are not in the grammar, so v03 uses the equal-precedence left-associative form `a AND b OR c` = `((a AND b) OR c)`. The parser was correct throughout; the corpus and the (also-broken) runner were the defects. Verified locally against the actual lib/phronesis/{lexer,parser}.ex: 7/7 valid parse, 3/3 invalid rejected. New gate conformance.yml runs the corpus on every push/PR, reusing the estate-vetted checkout + setup-beam pins and the project toolchain (Elixir 1.16 / OTP 26 per .tool-versions). https://claude.ai/code/session_01DQACj3RFmAPZaBPgR9SAaS
1 parent 0911bc4 commit 44c97cb

6 files changed

Lines changed: 79 additions & 37 deletions

File tree

.github/workflows/conformance.yml

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# conformance.yml — parser conformance gate for the Phronesis grammar.
3+
#
4+
# Runs the real parser (Phronesis.parse/1 = Lexer.tokenize |> Parser.parse) over
5+
# the conformance corpus:
6+
# * conformance/valid/*.phr MUST parse (exit 0)
7+
# * conformance/invalid/*.phr MUST fail (exit non-zero)
8+
# The corpus is the executable contract for spec/grammar.ebnf (v0.2.0). Any drift
9+
# between the parser and the documented grammar turns this gate red.
10+
name: Conformance
11+
12+
on:
13+
push:
14+
branches: [main, master]
15+
pull_request:
16+
workflow_dispatch:
17+
18+
# Estate guardrail: cancel superseded runs (read-only check, safe to cancel).
19+
concurrency:
20+
group: ${{ github.workflow }}-${{ github.ref }}
21+
cancel-in-progress: true
22+
23+
permissions:
24+
contents: read
25+
26+
jobs:
27+
conformance:
28+
name: Grammar conformance (parser vs corpus)
29+
runs-on: ubuntu-latest
30+
timeout-minutes: 15
31+
steps:
32+
- name: Checkout
33+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
34+
35+
# Match the project toolchain (.tool-versions: elixir 1.16.0 / erlang 26.2.1).
36+
# Pinned to the estate-vetted setup-beam SHA already used by hypatia-scan.yml.
37+
- name: Setup Elixir
38+
uses: erlef/setup-beam@fc68ffb90438ef2936bbb3251622353b3dcb2f93 # v1.18.2
39+
with:
40+
elixir-version: '1.16'
41+
otp-version: '26'
42+
43+
- name: Fetch dependencies
44+
run: mix deps.get
45+
46+
# Compile without --warnings-as-errors: the gate measures conformance, not
47+
# lint. A genuine compile break still fails here (mix compile exits non-zero).
48+
- name: Compile
49+
run: mix compile
50+
51+
- name: Run conformance corpus
52+
run: ./conformance/run_conformance.sh

conformance/run_conformance.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ if [[ -n "${1:-}" ]]; then
2020
# shellcheck disable=SC2206 # deliberate word-splitting of caller's string
2121
PARSER_CMD=(${1})
2222
else
23-
PARSER_CMD=(mix run -e 'Phronesis.CLI.parse(System.argv())' --)
23+
PARSER_CMD=(mix run --no-start -e 'System.halt(case Phronesis.parse(File.read!(Enum.at(System.argv(), 0))) do {:ok, _} -> 0; _ -> 1 end)' --)
2424
fi
2525

2626
PASS=0

conformance/valid/v02_if_then.phr

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
11
# SPDX-License-Identifier: MPL-2.0
2-
# Conformance: IF/THEN/ELSE conditional in policy
2+
# Conformance: IF/THEN/ELSE conditional action (grammar §5 conditional_action)
33

44
POLICY conditional_check:
5-
IF status == :valid THEN
6-
ACCEPT "Status is valid"
7-
ELSE
8-
REJECT "Status is invalid"
9-
PRIORITY 50
10-
EXPIRES "2030-12-31"
11-
CREATED_BY "test_suite"
5+
value > 0
6+
THEN IF value > 100 THEN ACCEPT("High value") ELSE REJECT("Low value")
7+
PRIORITY: 50
8+
EXPIRES: never
9+
CREATED_BY: test_suite

conformance/valid/v03_and_or.phr

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# SPDX-License-Identifier: MPL-2.0
2-
# Conformance: compound conditions with AND/OR
2+
# Conformance: compound conditions with AND/OR (grammar §6 logical_expr)
3+
# AND/OR are equal-precedence, left-associative, so this parses as
4+
# ((score > 50 AND status == "active") OR override == true).
35

46
POLICY compound_logic:
5-
IF (score > 50 AND status == :active) OR override == true THEN
6-
ACCEPT "Conditions met"
7-
ELSE
8-
REJECT "Conditions not met"
9-
PRIORITY 75
10-
EXPIRES "2030-12-31"
11-
CREATED_BY "test_suite"
7+
score > 50 AND status == "active" OR override == true
8+
THEN ACCEPT("Conditions met")
9+
PRIORITY: 75
10+
EXPIRES: never
11+
CREATED_BY: test_suite

conformance/valid/v05_import.phr

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,12 @@
11
# SPDX-License-Identifier: MPL-2.0
2-
# Conformance: IMPORT clause
2+
# Conformance: IMPORT clause + module-qualified call (grammar §3, §8)
33

44
IMPORT Std.BGP
55
IMPORT Std.RPKI
66

77
POLICY import_test:
8-
rpki_result = Std.RPKI.validate(route)
9-
IF rpki_result == :valid THEN
10-
ACCEPT "RPKI validation passed"
11-
ELSE
12-
REJECT "RPKI validation failed"
13-
PRIORITY 100
14-
EXPIRES "2030-12-31"
15-
CREATED_BY "test_suite"
8+
Std.RPKI.validate(route) == "valid"
9+
THEN ACCEPT("RPKI validation passed")
10+
PRIORITY: 100
11+
EXPIRES: never
12+
CREATED_BY: test_suite

conformance/valid/v07_nested.phr

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,9 @@
11
# SPDX-License-Identifier: MPL-2.0
2-
# Conformance: nested conditions
2+
# Conformance: nested conditional actions (grammar §5 conditional_action)
33

44
POLICY nested_check:
5-
IF level > 1 THEN
6-
IF score > 80 THEN
7-
ACCEPT "High score at high level"
8-
ELSE
9-
REJECT "Low score at high level"
10-
ELSE
11-
ACCEPT "Any score at base level"
12-
PRIORITY 150
13-
EXPIRES "2030-12-31"
14-
CREATED_BY "test_suite"
5+
level > 1
6+
THEN IF score > 80 THEN ACCEPT("High score at high level") ELSE REJECT("Low score at high level")
7+
PRIORITY: 150
8+
EXPIRES: never
9+
CREATED_BY: test_suite

0 commit comments

Comments
 (0)