feat: merge-train/fairies#22923
Merged
Merged
Conversation
`yarn-project/kv-store/yarn test:browser` runs ~11 vitest browser-mode
test files in a single vitest process, all sharing one chromium browser.
In CI3's `ISOLATE` environment (`docker run
--cpus=2 --memory=8g --tmpfs /tmp:exec,size=1g`), the run would
intermittently hang at a test-file boundary, sit silent for >80s, then
get killed by the outer CI timeout. Roughly 25% hit rate on CI; zero hit
rate on developer machines.
The bug only manifests under CPU pressure. On unconstrained hardware
(developer boxes with many cores), every chromium thread always gets
scheduled in time and the bug never surfaces.
Root caused to a CDP teardown deadlock between vitest and chromium at
test-file transitions:
1. Vitest opens 10 CDP TCP connections to chromium's `headless_shell
--type=utility --utility-sub-type=network.mojom.NetworkService `(the
chromium "network service" process) at startup.
2. Between test files, vitest tears down 6 of those 10 connections in a
batch, sending FIN on each socket.
3. The kernel transitions chromium's side of those 6 sockets to
`CLOSE_WAIT` (peer closed, application hasn't called `close()` yet).
4. Chromium's network service event loop, starved of CPU under
`--cpus=2`, never gets scheduled to drain those closed FDs and call
`close()`.
5. Vitest's teardown path is awaiting completion of the close handshake
on those 6 connections (likely a `Promise.all` over `CDPSession`
detach), and never gets the response.
6. Both processes end up with zero on-CPU threads. `vitest` parked on
`futex_wait_queue`, chromium parked on `ep_poll` -> deadlock
The hang is at upstream code (@vitest/browser-playwright ↔ playwright ↔
chromium CDP). The 6:4 close/open ratio reproduces exactly across runs.
The fix is to run each test file in a separate vitest invocation,
instead of one vitest process iterating over all files.
- `yarn-project/kv-store/scripts/run-browser-tests.sh`: new shell loop
that finds each *.test.ts under src/indexeddb/ and src/sqlite-opfs/ and
runs yarn vitest run "$file" per file, sequentially.
- `yarn-project/kv-store/package.local.json`: test:browser overridden to
bash scripts/run-browser-tests.sh
This avoids the bug entirely because the cross-file teardown path is
never exercised: each vitest process only has to tear down at its own
end-of-process, where chromium gets killed outright by the OS rather
than asked to close gracefully via CDP.
Cost: ~5–10s of vite/yarn startup per file. With 11 files, that's
~60–100s of extra wall time per yarn test:browser run. Not great, but if
we can stabilize the suite and keep it running, we can iteratively look
for better ways (eg: reducing the amount of files to reduce overhead
where sensible).
## Verification
A reliable repro for future regressions is saved at
yarn-project/kv-store/scripts/repro-browser-hang.sh. It runs the
previous (single-process) test:browser shape under docker_isolate
constraints; reproduces consistently (without this fix) in ~3 minutes.
Closes F-589
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the sqlite3 wasm version we were using with a sqlite3mc wasm (mc -> multiple ciphers) that supports page level encryption via pragmas. Some potential discussion points: - Uses ChaCha20 instead of AES due to performance issues of software implementations of the latter - Vendors the binary and glue code for sqlite3mc from upstream repo https://github.com/utelle/SQLite3MultipleCiphers. We include a vendor.sh script to verify integrity of the vendored files against SHA hashes. This is a quick way of providing an initial version. If we decide to stick to this solution, we might want to instead setup our own build pipeline, building upstream from source and publishing the results on npm. Please don't be too scared by the line-diff ~46,000 out of the ~46,900 lines in this changeset are a consequence of vendoring. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
## Summary Extends the `aztec_txe_getLastTxEffects` oracle to also surface the last tx's private logs, so contract tests can fetch them and assert on tags/payloads without a dedicated `get_logs_by_tag` oracle. As discussed in #22854 (comment). Added type aliases in `txe_oracles.nr` for readability. Once this lands, #22854 will replace its `private_logs_by_tag` queries with `get_last_tx_effects`.
Collaborator
Author
|
🤖 Auto-merge enabled after 4 hours of inactivity. This PR will be merged automatically once all checks pass. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BEGIN_COMMIT_OVERRIDE
chore: fix kv-store browser tests hangs (#22721)
feat: kv-store sqlite backend with page level encryption (#22759)
fix: install node 22 for aztec-cli acceptance test (#22917)
fix(kv-store): exclude @aztec/sqlite3mc-wasm from vitest optimizeDeps (#22929)
feat(txe): add tx private logs to tx side effects oracle (#22889)
feat(aztec-nr): add call_self stubs for utility functions (#22885)
END_COMMIT_OVERRIDE