Skip to content

Commit 0e9c60e

Browse files
vezenovmAztecBot
andauthored
fix(ci): run contract-snapshots tests in CI (#23895)
## The gap This draft exists just so display the first half of a red-green flow. We should merge the child PR #23896 into this one before merging. The contract-snapshots suite (added in #23061) has never run in CI. It was wired only into the `noir-projects/bootstrap.sh test_cmds` aggregate, but the Makefile-driven CI never invokes parent aggregates: it calls each subproject's `test_cmds` directly (`noir-protocol-circuits-tests` in `fast`, `noir-projects-txe-tests` post-build). The aggregate is a local-dev entry point only. Consequences so far: - the noir 1.0.0-beta.22 bump (#23870) landed green with stale snapshots (see #23878), - aztec-nr changes since the last regen in #23464 have drifted the snapshots unnoticed. 14 of 61 tests fail on this branch with its own pinned nargo. ## The fix - New `contract-snapshots-tests` Makefile target (mirrors `noir-protocol-circuits-tests`, depends only on `noir` for the nargo binary), included in `fast`. - Comment on the bootstrap aggregate marking it local-dev only, so the next suite added there also gets a Makefile target. ## Expected CI result This PR is intentionally red: the newly wired suite fails with the 14 drifted snapshots, demonstrating the gap. A follow-up PR targeting this branch regenerates the snapshots and turns it green. Merge order: regen PR into this branch, then this branch into the train. Draft with `ci-draft` label so CI runs while merge automation stays off until the regen lands. --------- Co-authored-by: AztecBot <tech@aztec-labs.com>
1 parent f943cad commit 0e9c60e

55 files changed

Lines changed: 829 additions & 1841 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

Makefile

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ endef
5555

5656
# Fast bootstrap.
5757
fast: release-image barretenberg boxes playground docs aztec-up \
58-
bb-tests l1-contracts-tests yarn-project-tests boxes-tests playground-tests aztec-up-tests docs-tests noir-protocol-circuits-tests release-image-tests spartan claude-tests
58+
bb-tests l1-contracts-tests yarn-project-tests boxes-tests playground-tests aztec-up-tests docs-tests noir-protocol-circuits-tests contract-snapshots-tests release-image-tests spartan claude-tests
5959

6060
# Full bootstrap.
6161
full: fast bb-full-tests bb-cpp-full yarn-project-benches
@@ -308,6 +308,9 @@ noir-projects-txe-tests:
308308
$(call test,$@,noir-projects/aztec-nr)
309309
$(call test,$@,noir-projects/noir-contracts)
310310

311+
contract-snapshots-tests: noir
312+
$(call test,$@,noir-projects/contract-snapshots)
313+
311314
# Noir Projects - Aggregate target (builds all sub-projects)
312315
noir-projects: noir-protocol-circuits mock-protocol-circuits noir-contracts aztec-nr
313316

noir-projects/bootstrap.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ function build {
3030
aztec-nr
3131
}
3232

33+
# Local-dev entry only (via ./bootstrap.sh test). CI does not run this aggregate:
34+
# the root Makefile wires each subproject's test_cmds individually, so a new
35+
# suite must also get a Makefile target to run in CI.
3336
function test_cmds {
3437
parallel -k ./{}/bootstrap.sh test_cmds ::: noir-protocol-circuits noir-contracts aztec-nr contract-snapshots
3538
}

noir-projects/contract-snapshots/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
`cargo insta` snapshot tests for noir contracts. Two test categories live here:
44

55
- **`expand/`** — runs `nargo expand` on a curated set of aztec-nr contracts (mirrors what CI benchmarks already exercise) and snapshots the expanded source. Surface for catching macro regressions that pass typechecking but silently change generated code.
6-
- **`compile_failure/`** — drives `nargo compile` on intentionally invalid aztec-nr contracts and snapshots the full stderr.
6+
- **`compile_failure/`** — drives `nargo compile` on intentionally invalid aztec-nr contracts and snapshots the full stderr. Locations inside the aztec-nr macro sources are scrubbed to `<line>`/`<col>` markers (like the `<repo>` path marker) so macro edits don't churn every snapshot; locations in the test program itself and in the stdlib are kept.
77

88
### Layout
99

noir-projects/contract-snapshots/tests/snapshots.rs

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,15 +38,71 @@ fn nargo(dir: &Path) -> Command {
3838
/// emits when concurrent test invocations contend on its git-deps cache.
3939
/// 2. Replaces the absolute repo prefix with `<repo>` so call-stack lines
4040
/// pointing into `aztec-nr/aztec/src/macros/...` are stable across machines.
41+
/// 3. Replaces `:line:col` suffixes on `<repo>` lines with `:<line>:<col>`, and
42+
/// blanks code-frame gutter numbers in frames headed by a `<repo>` path.
43+
/// Any macro edit shifts these positions, which would churn every
44+
/// snapshot with diffs that carry no signal: the tests verify error text,
45+
/// user-code locations and call-stack shape, not exact positions inside the
46+
/// macro sources. Locations in the test program itself (`src/main.nr`) and
47+
/// in the stdlib are kept. Alignment padding derived from the gutter digit
48+
/// width is left alone, so a macro line number crossing a digit-count
49+
/// boundary can still produce a whitespace-only diff.
4150
fn scrub_stderr(s: String) -> String {
4251
let prefix = format!("{}/", repo_root().display());
52+
let mut in_repo_frame = false;
4353
s.lines()
4454
.filter(|l| !l.contains("Waiting for lock"))
45-
.map(|l| l.replace(&prefix, "<repo>/"))
55+
.map(|l| {
56+
let l = l.replace(&prefix, "<repo>/");
57+
let trimmed = l.trim_start();
58+
if trimmed.is_empty() {
59+
in_repo_frame = false;
60+
} else if trimmed.starts_with("┌─") {
61+
in_repo_frame = trimmed.contains("<repo>/");
62+
}
63+
if l.contains("<repo>/") {
64+
scrub_location_suffix(&l)
65+
} else if in_repo_frame {
66+
scrub_gutter_number(&l).unwrap_or(l)
67+
} else {
68+
l
69+
}
70+
})
4671
.collect::<Vec<_>>()
4772
.join("\n")
4873
}
4974

75+
/// `...aztec.nr:111:21` -> `...aztec.nr:<line>:<col>`; lines without a
76+
/// trailing `:line:col` are returned unchanged.
77+
fn scrub_location_suffix(l: &str) -> String {
78+
let is_num = |s: &str| !s.is_empty() && s.bytes().all(|b| b.is_ascii_digit());
79+
if let Some((rest, col)) = l.rsplit_once(':') {
80+
if is_num(col) {
81+
if let Some((rest, line)) = rest.rsplit_once(':') {
82+
if is_num(line) {
83+
return format!("{rest}:<line>:<col>");
84+
}
85+
}
86+
}
87+
}
88+
l.to_string()
89+
}
90+
91+
/// ` 164 │ some code` -> ` │ some code` for code-frame gutter lines.
92+
fn scrub_gutter_number(l: &str) -> Option<String> {
93+
let trimmed = l.trim_start();
94+
let indent = l.len() - trimmed.len();
95+
let digits = trimmed.find(|c: char| !c.is_ascii_digit())?;
96+
if digits == 0 || !trimmed[digits..].starts_with(" │") {
97+
return None;
98+
}
99+
Some(format!(
100+
"{}{}",
101+
" ".repeat(indent + digits),
102+
&trimmed[digits..]
103+
))
104+
}
105+
50106
/// Asserts `nargo compile` fails for `dir` and snapshots scrubbed stderr
51107
fn run_compile_failure(name: &str, dir: PathBuf) {
52108
let out = nargo(&dir)

noir-projects/contract-snapshots/tests/snapshots/compile_failure/allow_phase_change_on_non_external_fn/snapshots__stderr.snap

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,6 @@ error: The #[allow_phase_change] attribute can only be applied to #[external("pr
1212
1: ?
1313
at src/main.nr:11:5
1414
2: allow_phase_change
15-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/functions/mod.nr:166:9
15+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/functions/mod.nr:<line>:<col>
1616

1717
Aborting due to 1 previous error

noir-projects/contract-snapshots/tests/snapshots/compile_failure/allow_phase_change_on_utility_fn/snapshots__stderr.snap

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ error: The #[allow_phase_change] attribute cannot be applied to #[external("util
1212
1: ?
1313
at src/main.nr:9:5
1414
2: external
15-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/functions/mod.nr:355:9
15+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/functions/mod.nr:<line>:<col>
1616
3: assert_valid_utility
17-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/functions/mod.nr:505:9
17+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/functions/mod.nr:<line>:<col>
1818

1919
Aborting due to 1 previous error

noir-projects/contract-snapshots/tests/snapshots/compile_failure/authorization_selector_collision/snapshots__stderr.snap

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@ source: tests/snapshots.rs
33
expression: stderr
44
---
55
error: Trait crate::authwit::authorization_interface::AuthorizationInterface not found
6-
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:36:43
6+
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:<line>:<col>
77
8-
36 │ let authorization_interface = quote { crate::authwit::authorization_interface::AuthorizationInterface };
8+
│ let authorization_interface = quote { crate::authwit::authorization_interface::AuthorizationInterface };
99
│ ---------------------------------------------------------------
1010
1111
┌─ src/main.nr:24:5
@@ -15,9 +15,9 @@ error: Trait crate::authwit::authorization_interface::AuthorizationInterface not
1515
1616

1717
error: Could not resolve 'authwit' in path
18-
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:37:54
18+
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:<line>:<col>
1919
20-
37 │ let authorization_selector_type = quote { crate::authwit::AuthorizationSelector };
20+
│ let authorization_selector_type = quote { crate::authwit::AuthorizationSelector };
2121
│ -------
2222
2323
┌─ src/main.nr:24:5
@@ -27,9 +27,9 @@ error: Could not resolve 'authwit' in path
2727
2828

2929
error: check_trait_impl_where_clause_matches_trait_where_clause: missing trait ID
30-
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:23:16
30+
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:<line>:<col>
3131
32-
23 │ let name = s.name();
32+
│ let name = s.name();
3333
│ --------
3434
3535
┌─ src/main.nr:24:5
@@ -39,9 +39,9 @@ error: check_trait_impl_where_clause_matches_trait_where_clause: missing trait I
3939
4040

4141
error: check_trait_impl_method_matches_declaration: missing trait impl
42-
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:42:16
42+
┌─ <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:<line>:<col>
4343
44-
42 │ fn get_authorization_selector(self) -> $authorization_selector_type {
44+
│ fn get_authorization_selector(self) -> $authorization_selector_type {
4545
--------------------------
4646
4747
┌─ src/main.nr:24:5
@@ -60,8 +60,8 @@ error: Selector collision detected between authorizations 'EventCollision8370082
6060
1: AuthorizationSelectorCollision::#[authorization]
6161
at src/main.nr:27:5
6262
2: authorization
63-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:65:5
63+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:<line>:<col>
6464
3: register_authorization
65-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:15:9
65+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/authorization.nr:<line>:<col>
6666

6767
Aborting due to 5 previous errors

noir-projects/contract-snapshots/tests/snapshots/compile_failure/authorize_once_from_wrong_type/snapshots__stderr.snap

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,16 +12,16 @@ error: Argument from in function foo must be of type AztecAddress, but is of typ
1212
1: #[aztec]
1313
at src/main.nr:4:1
1414
2: aztec
15-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/aztec.nr:111:21
15+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/aztec.nr:<line>:<col>
1616
3: process_functions
17-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:48:9
17+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:<line>:<col>
1818
4: [T]::map
1919
at std/vector.nr:67:33
2020
5: process_functions
21-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:48:41
21+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:<line>:<col>
2222
6: generate_public_external
23-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/public.nr:131:9
23+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/public.nr:<line>:<col>
2424
7: create_authorize_once_check
25-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/helpers.nr:72:9
25+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/helpers.nr:<line>:<col>
2626

2727
Aborting due to 1 previous error

noir-projects/contract-snapshots/tests/snapshots/compile_failure/authorize_once_missing_from_param/snapshots__stderr.snap

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,16 +12,16 @@ error: Function foo does not have a from parameter. Please specify which one to
1212
1: #[aztec]
1313
at src/main.nr:4:1
1414
2: aztec
15-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/aztec.nr:111:21
15+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/aztec.nr:<line>:<col>
1616
3: process_functions
17-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:48:9
17+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:<line>:<col>
1818
4: [T]::map
1919
at std/vector.nr:67:33
2020
5: process_functions
21-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:48:41
21+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:<line>:<col>
2222
6: generate_public_external
23-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/public.nr:131:9
23+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/public.nr:<line>:<col>
2424
7: create_authorize_once_check
25-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/helpers.nr:67:9
25+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/helpers.nr:<line>:<col>
2626

2727
Aborting due to 1 previous error

noir-projects/contract-snapshots/tests/snapshots/compile_failure/authorize_once_missing_nonce_param/snapshots__stderr.snap

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,16 +12,16 @@ error: Function foo does not have a authwit_nonce. Please specify which one to u
1212
1: #[aztec]
1313
at src/main.nr:4:1
1414
2: aztec
15-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/aztec.nr:111:21
15+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/aztec.nr:<line>:<col>
1616
3: process_functions
17-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:48:9
17+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:<line>:<col>
1818
4: [T]::map
1919
at std/vector.nr:67:33
2020
5: process_functions
21-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:48:41
21+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/mod.nr:<line>:<col>
2222
6: generate_public_external
23-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/public.nr:131:9
23+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/public.nr:<line>:<col>
2424
7: create_authorize_once_check
25-
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/helpers.nr:81:9
25+
at <repo>/noir-projects/aztec-nr/aztec/src/macros/internals_functions_generation/external/helpers.nr:<line>:<col>
2626

2727
Aborting due to 1 previous error

0 commit comments

Comments
 (0)