perf(test): disable Lambda bundling in CDK unit-test synths (~15x per synth) (#363)

[scottschreckengaust]

Summary

Reduce //cdk:test wall time by eliminating repeated full-stack synthesis in the handful of mega-suites that dominate the CDK Jest suite. Profiling (see #365) shows the cost is concentrated: the top 5 suites are 362s of 437s suite-time (83%), and the single heaviest file is 135.6s. Each new App() + Template.fromStack() is ~27s; several suites synthesize the same stack multiple times where one shared synth would do.

This is the predecessor to sharding (#365) — lowering the per-file floor is both a bigger, simpler win on its own AND what makes sharding worthwhile later. Parent: #363.

Evidence (8-core local, coverage on)

Suite	Time	Full synths in file	Notes
test/stacks/github-tags.test.ts	135.6s	4 (synthWithTags)	2 hoisted to beforeAll, 2 still inline (empty string, compute_type ecs tests)
test/constructs/task-api.test.ts	83.5s	12 (4 beforeAll + inline)	audit which inline synths can share a template
test/stacks/agent.test.ts	50.2s	2	already mostly beforeAll; verify
test/constructs/task-orchestrator.test.ts	49.9s	2	already mostly beforeAll; verify
test/bootstrap/synth-coverage.test.ts	44.4s	2	by-design coverage synth; check if reducible

Approach

For each heavy suite: synthesize each distinct stack configuration exactly once (in beforeAll), and have individual tests assert against the cached Template. Only synth again when a test genuinely needs a different context/config. Do not weaken assertions or merge tests — only remove redundant synthesis.

Concrete first target — github-tags.test.ts: hoist the two remaining inline synthWithTags(...) calls (empty string values and compute_type: 'ecs') into shared beforeAll templates keyed by their context, reused across tests. Est. ~−54s on this file alone.

Acceptance criteria

• Each targeted suite synthesizes each distinct stack config at most once; redundant inline synths removed.
• All tests still pass (2177 CDK tests) with identical assertions; coverage thresholds unchanged and met.
• //cdk:test wall-time improvement reported on real 4-core CI (before/after); target: meaningfully below the current ~298s, driven by the top-5 suites.
• No reduction in what is asserted; no test merged away to fake a speedup.
• mise run build clean; no drift; no new flakiness.

Notes for implementer

• new App() + Template.fromStack() is the expensive operation (~27s here) — cdk-nag aspects run during synth. Counting these per file is the quickest way to find waste: grep -cE "new App\(|Template.fromStack" <file>.
• Some suites already use beforeAll correctly — verify before changing; the win is only in the files that synth the same config repeatedly.
• After this lands, re-profile and decide whether perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365 (sharding) is still worth it — the floor may be low enough that the unsharded suite is already fast.

References

• Parent umbrella: perf(ci): remaining build-performance levers (shard, coverage-gate, runner, path-filter) — follow-ups to #357 #363
• Sharding (deferred predecessor-of): perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365
• Design + profiling: docs/design/CI_BUILD_PERFORMANCE.md (PR docs(design): CI build performance roadmap + implementer notes (#363) #364)

---

🤖 Generated with Claude Code

[scottschreckengaust]

Starting implementation

[scottschreckengaust]

Starting implementation. Branch feat/366-synth-reuse off latest main. Scope: hoist redundant inline full-stack synths to shared beforeAll (starting with github-tags.test.ts), plus durable regression guards in AGENTS.md and .abca/commands/review_pr.md. Will open as draft PR until verified green. Per ADR-003.

[scottschreckengaust]

Major finding — the real lever is disabling Lambda bundling at synth, not synth-dedup

I started implementation and profiled with instrumentation before committing. Two results overturn the original plan:

1. Synth-dedup (the issue's premise) does NOT help here

github-tags.test.ts does 4 synthWithTags() calls — but they use 4 distinct contexts (defaults, full-values, empty-strings, ecs), so none is redundant. Hoisting them into beforeAll is cleaner but does not reduce synth count → measured ~135s → ~119s (noise). The synths aren't the waste; each individual synth being ~28s is.

2. The 28s/synth is ~93% Lambda esbuild bundling — disable it and synth is 15× faster

Measured a single AgentStack synth (413 resources, no cdk-nag in these tests):

Synth config	Time
Default (bundles NodejsFunction lambdas via esbuild)	28,700ms
new App({ context: { 'aws:cdk:bundling-stacks': [] } })	1,869ms

That CDK context key skips asset bundling at synth time. The github-tags assertions inspect resource tags, which don't depend on bundled Lambda code, so they're unaffected.

Applied to the whole file:

github-tags.test.ts	wall
today	~135s
bundling disabled	9s (all 5 tests pass)

15× on one file, and it generalizes: 31 suites synth a stack and would each shed most of their time.

Why this changes the plan

• This beats both sharding (perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365, ~14%) and synth-dedup (~0%) by a wide margin, with no CI complexity.
• It likely obsoletes perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365 (if the unsharded suite drops from ~298s to well under a minute, sharding is moot).
• Blast radius is broader though: best applied centrally (a jest setupFilesAfterEnv setting the context for all stack tests). Risk: any test asserting on asset hashes / S3 keys / bundled output would change. Audit found one candidate — test/bootstrap/version.test.ts — to verify/exclude. Each of the 31 suites must stay green.

Proposed re-scope of #366

Retitle/refocus #366 to "speed up CDK stack-synth tests by disabling Lambda bundling at synth":

1. Add a shared test helper / jest setup that injects 'aws:cdk:bundling-stacks': [] for unit-test synths (opt-out for the few asset-hash tests).
2. Verify all 31 stack-synth suites stay green; exclude any that genuinely need bundled assets.
3. Report 4-core CI before/after; re-evaluate whether perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365 (sharding) is still worth doing.

No code committed — worktree clean. Holding for direction on the re-scope before proceeding, since this is broader than the issue's original premise and touches global test config.

cc #363 (umbrella), #365 (likely superseded).

[scottschreckengaust]

Re-scoped (see profiling comment above). Original premise (synth-dedup / beforeAll) does not yield the win — the cost is per-synth Lambda esbuild bundling (~28s/synth), not redundant synths.

New scope: disable Lambda bundling for CDK unit-test synths

CDK Template.fromStack() bundles every NodejsFunction via esbuild at synth time (~28s for the 413-resource AgentStack). Unit tests assert on CloudFormation structure/properties, not bundled Lambda code, so bundling is pure waste here. Setting the CDK context aws:cdk:bundling-stacks: [] skips it: single synth 28.7s → 1.9s (15×); github-tags.test.ts ~135s → 9s, all tests green.

Approach

1. Central opt-in helper / jest setupFilesAfterEnv that injects aws:cdk:bundling-stacks: [] into test App contexts (or a shared newTestApp() helper used by stack-synth suites).
2. Audit all 31 stack-synth suites; verify green. Exclude/opt-out any that legitimately assert on bundled assets (candidate: test/bootstrap/version.test.ts — verify).
3. Durable guard: AGENTS.md "Common mistakes" + .abca/commands/review_pr.md checklist — CDK unit tests should disable Lambda bundling at synth unless asserting on bundled assets.
4. Report real 4-core CI before/after. Re-evaluate whether perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365 (sharding) is still worthwhile (likely superseded).

Acceptance criteria

• Stack-synth unit tests skip Lambda bundling; single-synth cost drops ~15×.
• All CDK tests pass (2177); coverage thresholds unchanged and met.
• Any asset-hash-asserting test identified and explicitly opted out (kept correct).
• AGENTS.md + review_pr.md guidance added.
• 4-core CI before/after posted; perf(ci): shard the CDK jest suite across a job matrix (#363 item #2) #365 re-evaluated.
• mise run build clean; no drift.

Original synth-dedup scope withdrawn; beforeAll hygiene folded into the guidance, not a standalone change.

[scottschreckengaust]

Draft PR #371 opened with the bundling-disable implementation. Local: full //cdk:test 155s → 25s (6×), single synth 28.7s → 1.9s (15×), all 2177 tests green, coverage unchanged. Includes AGENTS.md + review_pr.md guards. Kept in draft pending 4-core CI + #364 ordering (the doc both files link).