feat(core): stabilize programmatic JS API with createRstest + /api surface#1372
feat(core): stabilize programmatic JS API with createRstest + /api surface#1372fi3ework wants to merge 21 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4086a9295
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Rsdoctor Bundle Diff AnalysisFound 13 projects in monorepo, 2 projects with changes. 📊 Quick Summary
📋 Detailed Reports (Click to expand)📁 core/mainPath:
📦 Download Diff Report: core/main Bundle Diff 📁 vscode/workerPath:
📦 Download Diff Report: vscode/worker Bundle Diff Generated by Rsdoctor GitHub Action |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e9cff4afde
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
from
9994853 to
ea98fcd
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ea98fcd645
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
from
ea98fcd to
83916c1
Compare
There was a problem hiding this comment.
💡 Codex Review
rstest/packages/core/src/cli/commands.ts
Lines 651 to 652 in 83916c1
Preserve embedded mode while resolving related filters
For programmatic runCli({ related: true }) / changed runs, this temporary context is built without the caller's embedded: true, so constructor-time config validation still uses CLI behavior and can call process.exit(1). One concrete case is a project-level shard mismatch: Rstest calls failConfig(embedded, ...), and because this pre-resolution context defaults to embedded: false, the host process exits before executeHostSafeRun can return an unhandledError like non-related programmatic runs do.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
from
83916c1 to
8b26a38
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b26a38d05
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
from
8b26a38 to
aeccd3f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aeccd3f382
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
2 times, most recently
from
404c847 to
8949166
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8949166c93
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 255c21b89a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 855a2fe046
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e29fcdfe23
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa1e604192
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
from
aa1e604 to
79f5d20
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60d7f136c1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: abaa811c0d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Deploying rstest with
|
| Latest commit: |
27dbfee
|
| Status: | 🚫 Build failed. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e275c1ff46
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…rface
Move all programmatic entrypoints to `@rstest/core/api`; the main
`@rstest/core` entry now exposes only in-test runtime globals, config
helpers (defineConfig/loadConfig/merge*), and types.
- Replace the standalone `runRstest` with an async `createRstest()`
factory (rsbuild-`createRsbuild`-style) returning an `RstestInstance`
with `run`/`listTests`/`mergeReports`/`close`. Construction carries
static host wiring + config: `configFile` (a path to load) and `config`
(an inline override). `config` accepts either an object (deep-merged
over the disk config) or a `(loadedConfig) => RstestConfig` callback
that receives the resolved disk config and returns the final config —
letting callers transform it directly instead of merging.
Per-invocation selection/control lives in `RunOptions`
(filters/related/changed/shard/bail/...).
- Add a jest-`runCLI`-aligned public `runCli(argv, { cwd })` that takes a
parsed-object argv (positionals in `argv._`), stays host-safe (never
calls `process.exit`), and returns a structured `TestRunResult`.
- Keep the raw-argv CLI router (`startCli`) off the public surface: build
it as its own internal `dist/cli.js` chunk (not a `package.json`
export) and have the `rstest` bin import that artifact directly.
- Rename the internal sync factory `createRstest` -> `createRstestContext`
and the internal type `RstestInstance` -> `RstestRunner`; share public
result types + assembly helpers via `src/api/result.ts`, and dedupe the
filter-resolve/context-build tail into `buildResolvedRunner`.
- Migrate the vscode worker and the programmatic e2e fixtures.
BREAKING CHANGE: programmatic APIs are no longer exported from
`@rstest/core` (`createRstest`, `initCli`, `runCLI` removed from the main
entry). Import them from `@rstest/core/api`. `runRstest(opts)` is removed
-- use `const r = await createRstest(opts); await r.run()`.
Add a programmatic `instance.watch()` entry to `@rstest/core/api`, mirroring
the CLI's watch mode: `runTests()` now returns an `RstestWatchHandle` in watch
mode so a host can stop watching via `watcher.close()`. The VS Code worker uses
it for continuous runs, restoring re-run-on-change after the API migration.
Also address programmatic-API review feedback:
- restore host `process.env` on `close()` (snapshot before `initRstestEnv`);
snapshot inside `runCli`'s guarded section too
- contain `process.exitCode` around `listTests`/`mergeReports`
- enable `includeTaskLocation` when `listTests({ printLocation: true })`
- build the eager createRstest context reporter-free (no stdout intercept)
- carry `embedded`/`cwd` into related/changed pre-resolution so config errors
surface as unhandledError instead of `process.exit(1)`
- VS Code worker: forward `run()` unhandledErrors; append (not replace) the
user's configured coverage reporters via the config function form
- bump VS Code `MIN_CORE_VERSION` to the release that ships `/api`
- migrate CodSpeed benchmarks to the `runCli` entry
- toCommonOptions: normalize `changed: false` to undefined so
`run({ changed: false })` runs the full suite instead of git-changed only
(isRelatedRun treats any defined `changed` as changed-mode)
- route `includeTaskLocation` (from listTests printLocation) and the eager
build's empty reporters through the option bag, so they reach per-project
configs via resolveProjects, not just the root config
- disable interactive CLI shortcuts for embedded watch: an embedded host with
a TTY stdin must not get raw-mode/keypress handlers that `q` can exit, and
watcher.close() doesn't dispose them
The eager build can reject (missing configFile, throwing config callback, config validation error) before the instance is returned, so the caller never gets a `close()` to restore the snapshot. Wrap it so `restoreHostState()` runs on that failure path instead of leaving an embedded host permanently in test mode. Covered by a new e2e fixture.
`isRelatedRun` treats any defined `changed` as changed-mode, so an
embedder forwarding parsed argv with boolean defaults (`changed: false`)
would wrongly run only git-changed tests / reject positional filters.
Mirror `toCommonOptions` and drop the explicit `false` so
`runCli({ changed: false })` runs the full suite.
The worker's function-form config sets `coverage.reporters` explicitly to append the extension's `CoverageReporter`. Rstest only auto-applies its default `text`/`html`/`clover`/`json` reporters when `coverage.reporters` is left undefined, so a workspace that never configured reporters lost those normal reports when running coverage from the extension. Fall back to the defaults when neither the disk config nor the per-invocation override provides any reporters.
Coverage thresholds / report generation and global teardown only signal
failure via `process.exitCode = 1` after reporters complete. The host-safe
guard restores that exit code, so `assembleTestRunResult` never observed
it and `createRstest().run()` / `runCli()` resolved `{ ok: true }` even
when the CLI would exit non-zero — misleading CI wrappers that gate on
`ok`. Capture the run's final `process.exitCode` before the guard restores
it and fold it into `ok`, covering every exit-code-only failure in one
place. Adds a programmatic e2e where a passing test still fails the run via
an unmet coverage threshold.
Capturing `process.exitCode` after the run counted a non-zero code the
embedding host had set *before* calling `run()`/`runCli()`, so a fully
passing run could resolve `{ ok: false }` just because the parent process
was already marked failed. Reset the exit code to a clean slate at the
start of the guarded run (mirroring the CLI) so only failures this run
produced affect `ok`; the guard still restores the host's original value.
Adds an e2e where a passing run started with `process.exitCode = 1`.
Stabilize `@rstest/core/api` toward 1.0 by matching rsbuild's JS API: - createRstest: drop `configFile`/`embedded` options; `config` is an object or a zero-arg factory (rsbuild-exact). Load a config file yourself via `loadConfig` inside the factory. - Construction is side-effect-free; `initRstestEnv` moves into `build` and every entry snapshots/restores host `process.env`/`exitCode`. - Remove instance-level `close()`; each `run()` self-restores host state. - Rename export `runCli` -> `runCLI` (rsbuild + jest parity). - watch(): add `WatchOptions.onResult`, delivering each run's public `TestRunResult` (run()-parity) without a custom reporter. Mirrors the `listTests(ListCommandOptions & RunOptions)` options-extension shape. - Migrate the VS Code worker to load config via `loadConfig` in the factory (append coverage reporter over the disk config). - Docs: rewrite programmatic-api.mdx (EN/ZH) following rsbuild's JS API structure and heading style.
The eager creation build forced `reporters: []` through the CLI merge to
avoid the default reporter's TTY renderer + `process.on('exit')` side
effects. But an empty array is truthy, so `mergeWithCLIOptions` overwrote
`normalizedConfig.reporters` (root and per-project) to `[]`, making a
caller that inspects `context` before any run see an empty reporter list
even when config selected reporters.
Build the eager context with the `list` command instead: it resolves the
full config/projects but constructs no reporter instances, so the side
effects are still avoided while the resolved reporter config stays intact.
fi3ework
force-pushed
the
feat/stabilize-programmatic-js-api
branch
from
19a34bc to
119c5c4
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 119c5c4563
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Promote the internal `startCli` to be `@rstest/core/api`'s public
`runCLI(options?: { argv?: string[]; cwd?: string }): Promise<void>` and
delete `startCli` — there is now a single CLI entry, aligned with rsbuild's
`runCLI`, for bridging Rstest into another CLI (e.g. a unified `rs` command)
that forwards a raw `process.argv`-shaped array. It routes every command
(run/watch/list/merge-reports/init) and owns the process.
Remove the previous host-safe `runCLI(RstestArgv) => TestRunResult` and the
`RstestArgv` type; structured "run and get results" is served solely by
`createRstest().run()`. Migrate the related-tests e2e onto
`createRstest().run()` and add coverage for the new CLI passthrough. Update the
programmatic API docs (en + zh) accordingly.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bde16864ff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Thread the resolved `cwd` into `buildResolvedRunner` for the run/list/
merge-reports commands and through the watch restart path so a
`runCLI({ cwd })` bridge launched from another directory resolves config
and root against that directory instead of the parent process cwd.
Also migrate the CodSpeed benchmark harnesses off the removed `runCli`
export to `createRstest().run()`, creating one instance per fixture up
front so the eager build stays out of the measured iterations.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e6dc7e4345
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The "with a config file" example set `createRstest({ cwd })` but called
`loadConfig({ path })` without it, so the config path resolved against
`process.cwd()` instead of the shown `cwd`. Thread the same `cwd` into
`loadConfig` in both the en and zh docs.
`listTests` accepts `ListCommandOptions & RunOptions`, so a caller reusing run options could pass `shard`. It flowed through `toCommonOptions` into `normalizedConfig.shard` and the list runner sliced the collected files down to one shard — contradicting the documented contract that execution-only fields are ignored for listing. Strip `shard`/`update`/`bail` before building the list runner, and cover it with an e2e asserting the full set is returned.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aed4321d6d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The programmatic `build` hard-coded `configFilePath: undefined`, so a run created through the `config` factory (loadConfig — as the VS Code worker and docs do) dropped the config file: it never entered `performance.buildCache.buildDependencies` (stale cache after config edits) and config-relative buildDependencies resolved against the root instead of the config directory. Mirror rsbuild: `loadConfig` already stamps the resolved path onto the config as `_privateMeta.configFilePath`, and it survives the factory's merge/spread, so read it there and thread it into the runner — no new public option. Cover it with an e2e asserting a config-relative buildDependency resolves against the config file's directory.
Narrow `RstestInstance.context` from the internal `RstestContext` to a new `RstestInstanceContext` — a read-only projection of the resolved config and projects, without the run state or the reporter/snapshot managers. Export `RstestInstanceContext`, `RstestProjectSummary`, and the resolved `NormalizedConfig` from `@rstest/core/api` so the surface is nameable, and mark the internal `embedded` flag `@internal`. Type `EnvironmentWithOptions.options` as `Record<string, unknown>` instead of `any` so no `any` leaks into the now-exported `NormalizedConfig`. Docs: fold "With a config file" into the `config` option, move the experimental warning to the top, drop the "runner" jargon, describe `runCLI` in terms of the CLI it runs, and document the curated `context` shape.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27dbfee077
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A watch rerun that rebuilds but matches no test entries logs "No test files need re-run" without a failing exit code, so a programmatic `onResult` must not report `ok: false` for it. Gate the no-tests failure in `assembleTestRunResult` on non-watch commands to mirror the CLI.
Distinguish an undefined `coverage.reporters` (→ Rstest's default text/html/clover/json) from an explicit empty array (→ suppress report files) instead of keying off the merged list's length, which re-added the defaults for a deliberately empty config. Mirrors core's `??` merge.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7dfba03780
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1 participant
Summary
Ahead of
@rstest/core1.0, the programmatic Node API needed a stable, predictable shape. Previously a single standalonerunRstest, plus host/CLI orchestration symbols (createRstest,initCli,runCLI), leaked from the main@rstest/coreentry — mixing in-test runtime globals with programmatic runners. This PR stabilizes that surface (relates to #1294).Breaking — public API:
@rstest/core/api. The main@rstest/coreentry now exposes only in-test runtime globals, config helpers (defineConfig/loadConfig/merge*), and types. No deprecated re-export shims.runRstestis replaced by an asynccreateRstest()factory (modeled on rsbuild'screateRsbuild) returning anRstestInstancewithcontext/run/listTests/mergeReports/close. Construction options carry static config + host wiring (cwd/configFile/config/embedded); per-invocation selection & control (filters/related/changed/shard/bail/ …) live inRunOptions.runCli(argv?, { cwd? })replaces the internalrunCLIand returns a structuredTestRunResultfor test-running commands. It shares the runner-construction seam (buildResolvedRunner) with therstestCLI so the two stay in lockstep — the CLI itself runs through the internalstartCli(the raw-argv layer, analogous to jest-cli'srun), not throughrunCli.Migration: import programmatic APIs from
@rstest/core/api; replacerunRstest(opts)withconst r = await createRstest(opts); await r.run().Implementation notes: shared public result types + assembly helpers are extracted to
src/api/result.ts(consumed by bothinstance.run()andrunCli); the internal sync factorycreateRstestis renamedcreateRstestContextand the internal typeRstestInstance→RstestRunner; the asynccreateRstestre-resolves config + projects per build and reuses the CLI's filter-resolution helpers. The vscode worker and the programmatic e2e fixtures are migrated to the new surface.Related Links
run({ related, filters })/run({ changed }), andrunCliaccepts the jest-namedfindRelatedTestsflag)Checklist