perf: experiment with e2e speedups#712
Draft
tobybessant wants to merge 10 commits into
Draft
Conversation
Temporary — drop lint/unit/Ubuntu/macOS/pnpm cells so the iteration loop reports only the bottleneck cell. Revert before merging. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tier 1A — defender scans every file open on windows-latest. Spawn-heavy workloads (we do ~100 cold node starts per cell) pay this tax per spawn. Verification step reads the value back since Tamper Protection has been known to silently reject the setting. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-step timing on the Tier 1A run showed defender disable cost 24s and didn't move the E2E job (242s -> 255s, within noise). Reverting. Real targets: bun install (82s) and the test run (122s). This commit goes after the install: cache the bun download cache plus all workspace node_modules dirs keyed on bun.lock. First run still pays full price; subsequent runs with unchanged lockfile should drop install to a few seconds. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cache restore mangled bun's node_modules symlinks on Windows (rolldown hoisting broken, build failed). Tier 1 isn't delivering. Moving on to Tier 3 — the 122s test run is the real target. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces execAsync('node ../../packages/.../bin/generate-license-file ...')
with runCli([...args]) across all CLI e2e specs. runCli spawns a Node
worker_thread, dynamic-imports the bundled CLI entry, calls main(argv),
captures stdout/stderr/exit, then terminates.
Why: on Windows CI, each exec pays ~3s for Node cold-start + V8 init +
bundle parse. Workers skip OS process creation; per-invocation cost
drops to the bundle import plus main(). Expected biggest lift on
Windows where cold Node dominated the 122s test run.
Integrity is preserved: real entry (dist/cli.mjs), real argv parsing,
real main(), real fs writes. Only the process boundary changes.
Implementation notes:
- packages/e2e-helpers/src/lib/runCli.ts — client, resolves cliEntry
via require.resolve('generate-license-file') then dist/cli.mjs
- packages/e2e-helpers/src/lib/cliWorker.ts — worker entry, patches
stdout/stderr capture and process.exit -> throw, awaits cli.main
- Node 24's default .ts type-stripping loads the worker TS directly,
no precompile step needed
- e2e packages gained an @generate-license-file/e2e-helpers devDep
where missing (config-file, multiple-inputs, optional-dependencies)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Worker-per-call (prior commit) regressed on Windows: each Worker still paid the bundled-CLI ESM import on startup. This switches to a single worker per vitest fork that imports the CLI once and reuses main() across every runCli call. mainCommand was a module-scope Command singleton, which would bleed state between main() invocations. Changed to a createMainCommand() factory — returns a fresh Command each call. End-user CLI behaviour (node bin/generate-license-file) is unchanged; latent bug if anyone embedded main() in library code is also fixed. Locally (macOS) cold call ~950ms, warm ~750ms. Expected win on Windows is larger since cold node + bundle import there is 3-5s. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Iteration branch for measuring E2E speedup techniques. CI is temporarily slimmed to the Windows E2E cell only so each push reports the bottleneck directly.
Do not merge. Workflow changes are temporary and the matrix needs restoring before this can land in any form.
Plan
Tier 1 (zero test changes): Defender off, bun cache, inline build, runner version
Tier 2 (vitest config): pool=threads, concurrent specs
Tier 3 (in-process): CLI worker daemon, NODE_OPTIONS preload + memfs
🤖 Generated with Claude Code