Skip to content

Commit 0d263c9

Browse files
feat(dsl): implement compile-time shape verification in @axiom (#62)
## Summary Makes the flagship **"compile-time shape verification"** claim *true*. Previously the `@axiom` macro parsed `input::`/`output::` type annotations but never used them — shape checks were runtime-only, and the README's `BrokenModel` "Compile error" example could not actually fire (it also used an aspirational API that doesn't match the code). This wires genuine verification into the macro. `_verify_axiom_shapes` runs **during macro expansion** — i.e. at compile time, before the model is ever constructed or run — tracking the tensor shape from the `input::` declaration through the layer chain. A provable mismatch raises an error at expansion: ``` @axiom BrokenModel: compile-time shape mismatch in `output`. Dense layer expects 128 input feature(s), but the incoming tensor has 256. Running shape entering this layer: (:batch, 256). ``` ## Design — sound but incomplete (by intent) It rejects **only** mismatches provable from the declared shapes and literal layer sizes: - each `Dense`'s declared `in_features` vs the running feature dim along the chain; - the computed `output` vs the `output::` declaration. Anything it cannot statically resolve — `Conv`/pool output geometry, non-literal layer arguments, unrecognised layers — collapses the running shape to *unknown* and passes through unchecked. So a **valid model never receives a false compile error**; only definite, literal-provable contradictions are rejected. ## Changes - **`src/dsl/axiom_macro.jl`** — `_verify_axiom_shapes` + helpers (`_extract_shape_dims`, `_shape_after_layer`, `_flatten_pipeline`, a shape-preserving-layer whitelist); called at the top of `generate_axiom_code`. - **`test/runtests.jl`** — new **"Compile-Time Shape Verification"** testset. The DSL previously had **zero** `@axiom` tests; this covers valid chains, the three mismatch types (Dense in-features, first-layer input, output declaration), the Conv/Flatten soundness case, and the no-input case. - **`README.adoc`** — the flagship example rewritten to the **real API** (`Dense(in,out)`, `(:batch,N)`) showing the actual compile error, with an explicit soundness caveat. ## Verification - Full suite: **732/732 passing** (`Pkg.test()`, 5m03s) — includes the new testset; no regressions. - The real `@axiom` macro path accepts the `mnist.jl`/`simple_classifier.jl`-style chains unchanged and rejects the mismatch cases. ## Follow-up (not in this PR) The audit's other Tier-1 item — the **PyTorch `.pt`/`.pth` bridge** claimed as shipped (`scripts/pytorch_to_axiom_descriptor.py` is absent) — is **not** addressed here: `.pt` files are Python-pickle and importing them needs PyTorch (Python), which is banned estate-wide. That claim needs a separate honest correction (the JSON-descriptor path is what actually ships) or a non-Python reader. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01UPFC9YQ7g9gc3VnRox42Q1 --- _Generated by [Claude Code](https://claude.ai/code/session_01UPFC9YQ7g9gc3VnRox42Q1)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 59ce4fa commit 0d263c9

3 files changed

Lines changed: 250 additions & 6 deletions

File tree

README.adoc

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,19 +44,39 @@ result = verify(model, properties=[ValidProbabilities(), FiniteOutput()], data=[
4444

4545
=== Compile-Time Shape Verification
4646

47+
The `@axiom` macro tracks tensor shapes through the layer chain **at
48+
macro-expansion time**, so a provable mismatch is a compile-time error — raised
49+
before the model is ever constructed or run, not after hours of training.
50+
4751
[source,julia]
4852
----
49-
# PyTorch: Runtime error after hours of training
50-
# Axiom.jl: Compile error in milliseconds
53+
# Correct: the feature dimensions chain cleanly (784 -> 256 -> 10).
54+
@axiom Classifier begin
55+
input :: Tensor{Float32, (:batch, 784)}
56+
output :: Tensor{Float32, (:batch, 10)}
57+
hidden = input |> Dense(784, 256, relu)
58+
output = hidden |> Dense(256, 10) |> Softmax
59+
end
5160
61+
# Broken: a Dense expecting 128 features fed a 256-feature tensor.
5262
@axiom BrokenModel begin
53-
input :: Tensor{Float32, (224, 224, 3)}
54-
features = input |> Conv(64, (3,3))
55-
output = features |> Dense(10) # COMPILE ERROR!
56-
# "Shape mismatch: Conv output is (222,222,64), Dense expects vector"
63+
input :: Tensor{Float32, (:batch, 784)}
64+
output :: Tensor{Float32, (:batch, 10)}
65+
hidden = input |> Dense(784, 256, relu)
66+
output = hidden |> Dense(128, 10) # COMPILE ERROR
5767
end
68+
# ERROR: @axiom BrokenModel: compile-time shape mismatch in `output`.
69+
# Dense layer expects 128 input feature(s), but the incoming tensor has 256.
70+
# Running shape entering this layer: (:batch, 256).
5871
----
5972

73+
Verification is **sound but incomplete**: it rejects any mismatch it can prove
74+
from the declared `input ::`/`output ::` shapes and literal layer sizes (the
75+
`Dense` feature dimensions along the chain, and the final output declaration),
76+
and passes through anything it cannot statically resolve — `Conv`/pooling output
77+
geometry, non-literal layer arguments — so a *valid* model never receives a
78+
false compile error.
79+
6080
=== Formal Verification
6181

6282
[source,julia]

src/dsl/axiom_macro.jl

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,167 @@ function _parse_macro_call(expr::Expr)
132132
end
133133
end
134134

135+
# ===========================================================================
136+
# Compile-time shape verification
137+
#
138+
# Walks the declared layer chain at MACRO-EXPANSION time, tracking the tensor
139+
# shape from the `input ::` declaration through each layer assignment. A
140+
# *provable* shape mismatch — e.g. a `Dense` whose declared `in_features`
141+
# disagrees with the running feature dimension, or a computed `output` that
142+
# contradicts the `output ::` declaration — raises an error during expansion,
143+
# i.e. a genuine compile-time error before the model is constructed or run.
144+
#
145+
# Sound-but-incomplete BY DESIGN. A running shape is either a concrete
146+
# `Vector` of dims (each an `Int`, or `:dynamic` for a `:batch`/`:dynamic`
147+
# wildcard) or `nothing` (statically unknown). Anything that cannot be
148+
# resolved from literals — Conv/Pool output geometry, non-literal layer args,
149+
# unrecognised layers — collapses the running shape to `nothing` and is passed
150+
# through unchecked, so a *valid* model can never receive a false compile
151+
# error. Only definite, literal-provable contradictions are rejected.
152+
# ===========================================================================
153+
154+
# Layer names known to preserve tensor shape (activations / normalisation).
155+
const _SHAPE_PRESERVING_LAYERS = Set(Symbol.([
156+
"Softmax", "LogSoftmax", "relu", "ReLU", "sigmoid", "Sigmoid", "tanh",
157+
"Tanh", "gelu", "GELU", "elu", "ELU", "swish", "SiLU", "leakyrelu",
158+
"LeakyReLU", "Dropout", "BatchNorm", "LayerNorm", "GroupNorm", "Identity",
159+
"softplus", "mish", "Mish",
160+
]))
161+
162+
# Extract the shape dims from an `input ::` / `output ::` type AST of the form
163+
# `Tensor{ElemType, (d1, d2, ...)}`. Returns a Vector of dims (Int or :dynamic)
164+
# or `nothing` if it is not in the recognised 2-parameter shorthand form.
165+
function _extract_shape_dims(typ)
166+
typ isa Expr || return nothing
167+
typ.head === :curly || return nothing
168+
length(typ.args) >= 3 || return nothing # Tensor, ElemType, shape
169+
shape_ast = typ.args[3]
170+
(shape_ast isa Expr && shape_ast.head === :tuple) || return nothing
171+
return Any[_norm_dim(d) for d in shape_ast.args]
172+
end
173+
174+
# Normalise one dim AST node to an Int, or :dynamic for any symbolic/wildcard
175+
# dim (:batch, :dynamic, :, or any other symbol). Unknown → :dynamic keeps the
176+
# analysis sound (a wildcard never triggers a mismatch).
177+
function _norm_dim(d)
178+
d isa Integer && return Int(d)
179+
if d isa QuoteNode
180+
return :dynamic
181+
end
182+
return :dynamic
183+
end
184+
185+
# (head, args) for a layer application: `Dense(784,256,relu)` -> (:Dense, [784,256,relu]);
186+
# a bare `Softmax` symbol -> (:Softmax, []); anything else -> (nothing, []).
187+
function _layer_head_args(layer)
188+
if layer isa Symbol
189+
return (layer, Any[])
190+
elseif layer isa Expr && layer.head === :call
191+
return (layer.args[1], layer.args[2:end])
192+
else
193+
return (nothing, Any[])
194+
end
195+
end
196+
197+
_lit_int(x) = x isa Integer ? Int(x) : nothing
198+
199+
# Apply one layer to the running shape. Returns (new_shape, mismatch_msg_or_nothing).
200+
# `shape === nothing` means statically unknown: never checked, stays unknown.
201+
function _shape_after_layer(layer, shape)
202+
head, args = _layer_head_args(layer)
203+
204+
if head === :Dense
205+
in_f = length(args) >= 1 ? _lit_int(args[1]) : nothing
206+
out_f = length(args) >= 2 ? _lit_int(args[2]) : nothing
207+
shape === nothing && return (nothing, nothing)
208+
if in_f !== nothing && !isempty(shape)
209+
last_dim = shape[end]
210+
if last_dim isa Int && last_dim != in_f
211+
return (shape, "Dense layer expects $in_f input feature(s), but the incoming tensor has $last_dim")
212+
end
213+
end
214+
# Output feature dim = out_f if known, else unknown (:dynamic).
215+
isempty(shape) && return (nothing, nothing)
216+
new_last = out_f === nothing ? :dynamic : out_f
217+
return (Any[shape[1:end-1]..., new_last], nothing)
218+
219+
elseif head === :Flatten || layer === :Flatten
220+
shape === nothing && return (nothing, nothing)
221+
# Only resolvable under the (batch, dims...) convention: a wildcard
222+
# leading dim with all-Int trailing dims. Otherwise -> unknown.
223+
if length(shape) >= 2 && shape[1] === :dynamic && all(x -> x isa Int, shape[2:end])
224+
return (Any[:dynamic, prod(shape[2:end])], nothing)
225+
end
226+
return (nothing, nothing)
227+
228+
elseif head in _SHAPE_PRESERVING_LAYERS
229+
return (shape, nothing) # shape unchanged (may be nothing)
230+
231+
else
232+
# Conv/Pool geometry and any unrecognised layer: cannot resolve
233+
# soundly -> collapse to unknown so nothing downstream false-errors.
234+
return (nothing, nothing)
235+
end
236+
end
237+
238+
# Flatten a pipeline AST `a |> L1 |> L2 |> ...` into (base, [L1, L2, ...]).
239+
function _flatten_pipeline(expr)
240+
layers = Any[]
241+
cur = expr
242+
while is_pipeline_expr(cur)
243+
pushfirst!(layers, cur.args[3])
244+
cur = cur.args[2]
245+
end
246+
return (cur, layers)
247+
end
248+
249+
_fmt_shape(dims) = "(" * join(map(d -> d === :dynamic ? ":batch" : string(d), dims), ", ") * ")"
250+
251+
"""
252+
_verify_axiom_shapes(name::Symbol, def::AxiomDefinition)
253+
254+
Compile-time shape check for an `@axiom` body. Raises an error during macro
255+
expansion on a provable shape mismatch; returns silently otherwise (including
256+
when the shapes are not statically resolvable).
257+
"""
258+
function _verify_axiom_shapes(name::Symbol, def::AxiomDefinition)
259+
in_dims = def.input_type === nothing ? nothing : _extract_shape_dims(def.input_type)
260+
in_dims === nothing && return nothing # no parseable input shape
261+
262+
shapes = Dict{Symbol, Any}(:input => in_dims)
263+
264+
for (lname, lexpr) in def.layers
265+
base, chain = _flatten_pipeline(lexpr)
266+
(base isa Symbol && haskey(shapes, base)) || continue
267+
cur = shapes[base]
268+
for layer in chain
269+
new_shape, msg = _shape_after_layer(layer, cur)
270+
if msg !== nothing
271+
error("@axiom $(name): compile-time shape mismatch in `$(lname)`.\n" *
272+
" $msg.\n" *
273+
" Running shape entering this layer: $(_fmt_shape(cur)).")
274+
end
275+
cur = new_shape
276+
end
277+
shapes[lname] = cur
278+
end
279+
280+
# Computed `output` vs declared `output ::` — reject a definite contradiction.
281+
if haskey(shapes, :output) && shapes[:output] isa Vector && def.output_type !== nothing
282+
out_dims = _extract_shape_dims(def.output_type)
283+
computed = shapes[:output]
284+
if out_dims !== nothing && length(out_dims) == length(computed)
285+
for (d_decl, d_comp) in zip(out_dims, computed)
286+
if d_decl isa Int && d_comp isa Int && d_decl != d_comp
287+
error("@axiom $(name): declared output shape $(_fmt_shape(out_dims)) " *
288+
"contradicts the computed output shape $(_fmt_shape(computed)).")
289+
end
290+
end
291+
end
292+
end
293+
return nothing
294+
end
295+
135296
"""
136297
generate_axiom_code(name::Symbol, def::AxiomDefinition) -> Expr
137298
@@ -140,6 +301,11 @@ parsed `AxiomDefinition`. This includes the struct definition, layer
140301
initialization, the forward pass function, and any `@ensure` checks.
141302
"""
142303
function generate_axiom_code(name::Symbol, def::AxiomDefinition)
304+
# Compile-time shape verification: runs NOW, during macro expansion, so a
305+
# provable shape mismatch is a genuine compile-time error (before the model
306+
# is ever constructed or run). Sound-but-incomplete — see _verify_axiom_shapes.
307+
_verify_axiom_shapes(name, def)
308+
143309
layer_fields, layer_inits = _generate_layer_fields_and_inits(def)
144310
persistent_layer_names = [field.args[1] for field in layer_fields]
145311
forward_body = _generate_forward_body(def)

test/runtests.jl

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -715,6 +715,64 @@ with open(args.output, "w", encoding="utf-8") as f:
715715
@test_throws EnsureViolation @ensure sum(x) 2.0 "Wrong sum"
716716
end
717717

718+
@testset "Compile-Time Shape Verification" begin
719+
# The @axiom macro runs _verify_axiom_shapes during expansion; test it
720+
# directly on parsed bodies (a struct can't be defined in local scope).
721+
mkdef(body) = Axiom.parse_axiom_body(body)
722+
723+
# Valid Dense chain: returns nothing (no error).
724+
valid = quote
725+
input :: Tensor{Float32, (:batch, 784)}
726+
output :: Tensor{Float32, (:batch, 10)}
727+
h1 = input |> Dense(784, 256, relu)
728+
logits = h1 |> Dense(256, 10)
729+
output = logits |> Softmax
730+
end
731+
@test Axiom._verify_axiom_shapes(:ValidNet, mkdef(valid)) === nothing
732+
733+
# Dense in-features mismatch (256 -> Dense(128,…)): compile-time error.
734+
mismatch = quote
735+
input :: Tensor{Float32, (:batch, 784)}
736+
output :: Tensor{Float32, (:batch, 10)}
737+
h1 = input |> Dense(784, 256, relu)
738+
output = h1 |> Dense(128, 10)
739+
end
740+
@test_throws ErrorException Axiom._verify_axiom_shapes(:BrokenNet, mkdef(mismatch))
741+
742+
# First-layer input mismatch (input 784 vs Dense wants 20): error.
743+
badin = quote
744+
input :: Tensor{Float32, (:batch, 784)}
745+
output :: Tensor{Float32, (:batch, 2)}
746+
output = input |> Dense(20, 2)
747+
end
748+
@test_throws ErrorException Axiom._verify_axiom_shapes(:BadIn, mkdef(badin))
749+
750+
# Declared output contradicts computed output: error.
751+
badout = quote
752+
input :: Tensor{Float32, (:batch, 20)}
753+
output :: Tensor{Float32, (:batch, 5)}
754+
output = input |> Dense(20, 3)
755+
end
756+
@test_throws ErrorException Axiom._verify_axiom_shapes(:BadOut, mkdef(badout))
757+
758+
# Conv/Flatten geometry is not statically resolvable -> must NOT error
759+
# (soundness: a valid model never gets a false compile error).
760+
convnet = quote
761+
input :: Tensor{Float32, (:batch, 28, 28, 1)}
762+
output :: Tensor{Float32, (:batch, 10)}
763+
c = input |> Conv(1, 32, (3, 3))
764+
f = c |> Flatten
765+
output = f |> Dense(100, 10)
766+
end
767+
@test Axiom._verify_axiom_shapes(:ConvNet, mkdef(convnet)) === nothing
768+
769+
# No input type declared -> nothing to verify, returns nothing.
770+
noinput = quote
771+
output = something |> Dense(10, 2)
772+
end
773+
@test Axiom._verify_axiom_shapes(:NoInput, mkdef(noinput)) === nothing
774+
end
775+
718776
@testset "SMT Runner" begin
719777
solver = Axiom.get_smt_solver()
720778
if solver === nothing

0 commit comments

Comments
 (0)