test(js-sdk): run the full unit test suite in a browser - #1609
test(js-sdk): run the full unit test suite in a browser#1609mishushakov wants to merge 5 commits into
Conversation
Promotes the browser leg from a single React smoke test to the full unit +
connectionConfig suite running inside a real headless Chromium via
@vitest/browser + Playwright (`pnpm test:browser`) — the same coverage
test:bun / test:deno / test:cf get. 69 files / 343 tests green.
The React smoke test and its devDependencies are gone (react, react-dom,
@types/react{,-dom}, @testing-library/react, @vitejs/plugin-react,
vitest-browser-react). The one thing it uniquely covered — the SDK driven with
no `process` global at all, as in a real browser bundle — is kept as
tests/runtimes/browser/noProcessGlobal.test.ts. Chromium installation moves
from the default `pretest` hook to `pretest:browser`, so plain `pnpm test` no
longer downloads a browser.
Test bugs the leg surfaced:
- commands/kill and pty/kill asserted `rejects.toThrowError(ProcessExitError)`,
but `ProcessExitError` is not exported from src. Vite's SSR transform
resolved the named import to `undefined`, so both assertions were vacuous;
real ESM in the browser makes it a hard SyntaxError. Now CommandExitError.
- files/read.test.ts had three byte-identical duplicated tests, each
provisioning a second sandbox on every runtime.
- api/http2 and envd/http2 carried dead `vi.doUnmock('undici')` /
`vi.doUnmock('../../src/utils')` calls from an old refactor. The bare
'undici' one made Vite pre-bundle a Node HTTP client mid-run and reload the
page under the running suite — a cold-cache-only flake that broke whichever
file was being collected at the time.
Node-only test APIs are replaced with cross-runtime equivalents: node:crypto
randomUUID/createHash to WebCrypto, Buffer to TextEncoder/TextDecoder,
path.basename inlined.
Limitations a browser physically can't work around are gated by documented
capability flags in tests/setup.ts, so they report as skipped instead of
disappearing into the config's exclude list. Two are real user-facing bugs,
filed as SDK-293 (x-next-token is not in Access-Control-Expose-Headers, so
Sandbox.list pagination silently truncates) and SDK-294 (the stopped-sandbox
502 has no CORS headers, so isRunning() throws instead of returning false).
Closes SDK-292
Co-Authored-By: Claude <noreply@anthropic.com>
PR SummaryLow Risk Overview Adds a dedicated CI gains a Removes the old vitest browser React project, Shared test fixes for cross-runtime: Reviewed by Cursor Bugbot for commit bf5d7cf. Bugbot is set up for automated code reviews on this repo. Configure here. |
Package ArtifactsBuilt from 915c003. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-2.36.1-browser-test-suite.0.tgzCLI ( npm install ./e2b-cli-2.15.2-browser-test-suite.0.tgzPython SDK ( pip install ./e2b-2.35.0+browser.test.suite-py3-none-any.whl |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5626cfff60
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -0,0 +1,45 @@ | |||
| import { assert, test } from 'vitest' | |||
|
|
|||
| import { Sandbox } from '../../../src' | |||
There was a problem hiding this comment.
Import the SDK after removing the process shim
Because this static import is evaluated during test-file collection, after processEnv.setup.ts has already installed globalThis.process, this browser-only test no longer exercises SDK module evaluation in a real no-process environment. A regression that adds a bare top-level process read anywhere under ../../../src would pass here but still crash for browser users before Sandbox.create() runs; dynamically import the SDK inside the test after Reflect.deleteProperty(globalThis, 'process') (or run this file without the setup shim) to preserve the smoke test's original coverage.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Correct, and fixed in 4c94fca — the SDK is now imported inside the test body, after Reflect.deleteProperty(globalThis, "process"). As you say, the static import was evaluated at collection time with the shim still installed, so module-eval-time coverage was vacuous; only the call path was covered. Verified: the SDK module graph now evaluates with no process global and the test gets as far as Sandbox.create.
There was a problem hiding this comment.
LGTM — test-only PR that expands the browser CI leg to the full unit suite, with no production code changes.
Extended reasoning...
Overview
This PR touches only test infrastructure: the browser vitest config, test files (replacing Node-only APIs like Buffer/node:crypto with WebCrypto/TextEncoder), the CI workflow matrix, package.json devDependencies (dropping React test tooling), and pnpm-lock.yaml. No files under packages/js-sdk/src/ are modified.
Security risks
None. No auth, crypto business logic, or user-facing behavior changes — only test harness and CI configuration.
Level of scrutiny
Low-to-moderate scrutiny is appropriate: this is a test-only change with no production code path affected. I spot-checked the PR's own claims against src/: CommandExitError is indeed exported (ProcessExitError never existed, confirming the described vacuous-assertion bug fix), and the browser-specific skip flags (canFetchSandboxServers, canReadPaginationToken, canObserveStoppedSandbox) correctly mirror existing runtime !== 'browser' CORS-avoidance logic already present in src/connectionConfig.ts. The new capability-gated skips are a reasonable way to surface browser limitations without silently dropping coverage.
Other factors
The bug hunting system found no issues. The PR description is thorough and self-documents the rationale for each test change (duplicate test removal, flaky mock cleanup, Node-API replacements). No changeset is needed since this is test-only, consistent with prior similar PRs (#1600) per CLAUDE.md guidance.
These tests drive the SDK and never render anything, so vitest's browser.screenshotFailures (on by default when headless) wrote a PNG of a blank page per failed test into .vitest-attachments/. Turned off, which also lets the .gitignore entry it needed go away. Removes canReadPaginationToken: the API not exposing X-Next-Token via Access-Control-Expose-Headers is fixed upstream in infra#3388, so the six limit-based pagination tests are un-gated and run in the browser like everywhere else. Note infra#3388 is not on production yet, and the browser leg only ever runs against production (staging callers pass node-only, which drops it), so those six tests fail on `paginator.hasNext` until the deploy lands. Co-Authored-By: Claude <noreply@anthropic.com>
|
Removes canObserveStoppedSandbox. The stopped-sandbox 502 arriving from the edge without CORS headers is fixed in e2b-dev/infra#3389, so isRunning() can report false in a browser again and checkSandboxHealth can return false, which restores the actionable TimeoutError on the 24 call sites that use it. Also un-gates host.test.ts 'ping server in non-running sandbox', which was under the wrong flag: it asserts the edge's own 502 JSON envelope rather than a response from a server inside the sandbox, so infra#3389 covers it too (the same PR's content-negotiation fix is what keeps the body JSON rather than the HTML error page). Its sibling 'ping server in running sandbox' does read the user's python server, so that one keeps the gate. canFetchSandboxServers stays for the four tests that read a response from a server the test started in the sandbox — no CORS headers there, and one of them sends a custom header that would preflight into python's 501. Its doc comment no longer cites the proxy 502 as an example, since that now carries the header. Until infra#3389 reaches production these 14 tests fail on the browser leg — 13 with 'TypeError: Failed to fetch' and one with the degraded SandboxError in place of TimeoutError, matching the two documented consequences exactly. The Node/Bun/Deno/workerd legs are unaffected (7 files / 26 tests green on Node). Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit afe3fb8. Configure here.
- gate the http-server pause/resume test on canFetchSandboxServers: it fetches a `python -m http.server` the test starts itself, which sends no CORS headers. Masked today by the isRunning failure, it would have started failing the moment infra#3389 rolls out. - import the SDK dynamically in noProcessGlobal, after the process shim is deleted. A static import is evaluated at collection time while the shim is still installed, so a top-level `process` read anywhere in the module graph would have passed here and still crashed a browser app on import. Co-Authored-By: Claude <noreply@anthropic.com>
…browser gate `python -m http.server` sends no CORS headers, so a browser handed the test an opaque `TypeError: Failed to fetch` instead of the response. That was gated behind `canFetchSandboxServers` as inherent, but it isn't: the server is ours, and a real browser app's own server opts in the same way. `corsHttpServerCmd` starts one that does, so the five tests run in Chromium too and the last capability flag is gone. Verified in Chromium against a local instance of the exact generated command: cross-origin GET readable, and a custom-header request survives the preflight via do_OPTIONS. Negative control against a plain `python -m http.server` reproduces the `Failed to fetch` the browser leg was showing. Co-Authored-By: Claude <noreply@anthropic.com>

Important
Blocked on one production deploy. The SDK side is already un-gated, so no further code change is needed here — the browser leg just stays red until it rolls out (the leg only ever runs against production; staging callers pass
node-only, which drops it).infra#33886Sandbox.list({ limit })pagination testsTypeError: Failed to fetch, 1 ×SandboxErrorinstead ofTimeoutErrorEverything else is green: 8 failed files, 65 passed, 2 skipped — and every one of the 15 failing tests is in the table above.
What
Promotes the browser leg from a single React smoke test to the full unit + connectionConfig suite running inside a real headless Chromium (
@vitest/browser+ Playwright) — the same coveragetest:bun/test:deno/test:cfget. 69 files / 343 tests green locally against production.pnpm test:browser # installs Chromium via pretest:browser, then runs the suiteReact is gone as requested — the smoke test (
tests/runtimes/browser/run.test.tsx) and its seven devDependencies (react,react-dom,@types/react,@types/react-dom,@testing-library/react,@vitejs/plugin-react,vitest-browser-react). Chromium installation moves off the defaultpretesthook ontopretest:browser, so plainpnpm testno longer downloads a browser.The one thing the smoke test uniquely covered — the SDK driven with no
processglobal at all, the way a real browser bundle runs — is kept as a dedicated test, since the rest of the suite runs against aprocess.envshim that would mask a regression there:Failure screenshots are off (
browser.screenshotFailures: false): these tests never render anything, so vitest's default would write a PNG of a blank page per failed test into.vitest-attachments/.Test bugs the leg surfaced
Two assertions were vacuous.
commands/killandpty/killassertedrejects.toThrowError(ProcessExitError)— butProcessExitErrorhas never been exported fromsrc. Vite's SSR transform resolves the missing named import toundefined, sotoThrowError(undefined)asserted nothing on any runtime; real ESM in the browser turns it into a hardSyntaxError. NowCommandExitError.files/read.test.tsran three tests twice — a byte-identical duplicated block, provisioning a second real sandbox for each on every runtime leg.A cold-cache-only flake, root-caused.
api/http2andenvd/http2carried deadvi.doUnmock('undici')/vi.doUnmock('../../src/utils')calls from an old refactor (neither file mocks anything any more). The bare'undici'one made Vite discover and pre-bundle a Node HTTP client mid-run, reloading the browser page under the running suite and breaking whichever file was being collected. Verified: 1-in-4 failure before, three cold-cache runs green after.Node-only test APIs are replaced with cross-runtime equivalents:
node:cryptorandomUUID/createHash→ WebCrypto,Buffer→TextEncoder/TextDecoder,path.basename→ inline.Two real browser bugs found
Both verified over the wire; neither was fixable in the SDK alone.
Pagination silently truncated — the cursor lives in the
x-next-tokenresponse header, which the API didn't name inAccess-Control-Expose-Headers, so a browser withheld it and every page looked like the last:Fixed upstream in infra#3388, so the tests are un-gated here and just need the prod rollout (see the callout above). SDK-293.
SDK-294 —
isRunning()threw instead of returningfalse. A stopped sandbox's 502 came from the edge without CORS headers, so the probe rejected opaquely (running: 204 + ACAO: */paused: 502 + no ACAO). Because a browser always preflights the SDK's envd requests (they carryE2b-Sandbox-Id/E2b-Sandbox-Port) and a preflight needs an ok status, adding the header to the 502 alone wasn't enough — theOPTIONSpath needed a 2xx too. This also silently downgraded the kill-mid-request error from the actionableTimeoutErrorto a genericSandboxErroracross 24 call sites.Fixed upstream in infra#3389, which also fixes the content negotiation the investigation turned up: the proxy chose HTML over JSON by User-Agent sniffing, and browser
fetchcan't override its UA, so an SDK call from a browser got the HTML error page where it expects JSON (Acceptwas ignored). Both are un-gated here.How limitations are handled
No capability flags — nothing is skipped for being a browser. The five tests that read a response from a server the test itself starts in the sandbox now start a CORS-enabled one, via
corsHttpServerCmdintests/setup.ts:This was originally gated behind a
canFetchSandboxServersflag as an inherent browser limitation. It isn't one: the server is the test's own, and a real browser app's server opts into CORS exactly the same way — so the gate was hiding coverage ofgetHostand the proxy path rather than documenting a wall.Access-Control-Allow-Headersplusdo_OPTIONSalso covers the traffic-access-token test, whose custom header the browser preflights.Confirmed on the browser leg (run): of the five tests this un-gates, three now pass —
ping server in running sandbox,sandbox works without token,auto-resume wakes paused sandbox on http request. That's the CORS server working end-to-end through the proxy against production. The other two are the two infra#3389 needs anyway:pause and resume a sandbox with http servercallsisRunning()on a paused sandbox, andsandbox requires traffic access tokenreads a proxy-synthesized403(which 3389 gives CORS headers) after a preflight that can't carry the token (which 3389 answers on thegetDestinationerror path).The
excludelist is only for genuinely Node-only suites (tests/bundle/**readsdistvianode:fs,tests/undici.test.tsresolves packages offprocess.versions.node) plus the two msw suites, whosemsw/nodeentry pulls innode:httpand can't be served to a browser. Porting those needssetupWorkerplus a service worker from a public dir — tracked as a follow-up on SDK-292, and worth doing since cancellation is the most runtime-divergent part of this SDK.CI
New
browsermatrix leg onubuntu-22.04, with the Playwright cache moved onto it (no other leg needs Chromium any more). The leg is dropped bynode-onlystaging callers, and it's the only leg whose result depends on the API's CORS headers — the matrix comment now says so, rather than claiming the non-Node legs add no backend signal.Notes
test(js-sdk)commits.vi.stubEnvanddelete process.env.Xkeep working in the browser because the shim aliasesprocess.envtoimport.meta.env, which is where vitest puts configenvand wherestubEnvwrites. VerifiedE2B_DEBUG=1 pnpm test:browsercorrectly skips the debug-gated tests.python -m http.serverreproduces theTypeError: Failed to fetchthe browser leg was showing, so the probe is meaningful rather than vacuous. Also checked there's exactly oneAccess-Control-Allow-Originheader — a duplicate would make browsers reject the response.pause and resume a sandbox with http servertest now carries thecanFetchSandboxServersgate too (it fetches its ownpython -m http.server, so it was a real miss that theisRunningfailure happened to mask), andnoProcessGlobal.test.tsimports the SDK dynamically inside the test body — a static import is evaluated at collection time, while theprocessshim is still installed, so it only covered the call path and not module evaluation.ProcessExitErrorones this fixes).tests/api/list.test.tsduplicates five tests byte-for-byte (only the threebetaPauseones intentionally differ, instance vs static), andfiles/write.test.ts:130repeatswrite fileas a strict subset of the test at:6— together ~7 redundant sandboxes per run per leg. Left alone rather than deleting tests outside this change's scope.Closes SDK-292
🤖 Generated with Claude Code