From 524d59a0545182f8283236f807dcd76adb3f416c Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 10 Jul 2026 17:50:48 -0700 Subject: [PATCH 1/6] Add design docs for onnxruntime-web backend consolidation Add two design documents under docs/design/ covering the plan to consolidate onnxruntime-web on the native WebGPU execution provider: - onnxruntime_web_jsep_to_webgpu_ep_migration.md: migrate WebGPU/WebNN from the JSEP TypeScript compute path to the native WebGPU EP, with a transparent default swap, a temporary /jsep escape hatch, int64 analysis, and pre-flip parity investigation items. - onnxruntime_web_remove_webgl_backend.md: deprecate and remove the legacy WebGL (onnxjs) backend, with explicit (non-transparent) removal and current WebGPU browser-coverage context. Both docs use a two-phase (deprecate, then remove) approach, adopt clean removal without tombstone shims, and point consumers at a single canonical tracking issue. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 334 ++++++++++++++++++ .../onnxruntime_web_remove_webgl_backend.md | 202 +++++++++++ 2 files changed, 536 insertions(+) create mode 100644 docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md create mode 100644 docs/design/onnxruntime_web_remove_webgl_backend.md diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md new file mode 100644 index 0000000000000..f3ab677025996 --- /dev/null +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -0,0 +1,334 @@ +# Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP + +**Status:** Draft +**Last updated:** 2026-07-10 +**Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends + +**Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md). +The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; see +§5.3 and §8 for the coupling. + +--- + +## 1. Summary + +`onnxruntime-web` implements WebGPU/WebNN compute two ways: + +- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. This is + the current default (`.`) and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). This is + the current `./webgpu` and `./jspi` bundles. + +This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU execution +provider** as the single WebGPU/WebNN path. The migration is designed to be **transparent** for the common case +(the default bundle keeps the same `webgpu` backend key and swaps the implementation underneath at build time), +with a **temporary escape hatch** for one release as insurance against undiscovered parity gaps. + +The work is split into two phases: + +- **Phase 1 (this release):** ship deprecation warnings + docs, plumb the one known config gap (`enableInt64`), + and add a temporary `onnxruntime-web/jsep` escape-hatch export. No default behavior change. +- **Phase 2 (next release):** flip the default bundle to the native EP, delete JSEP, and clean up the temporary + build flags and exports. + +--- + +## 2. Motivation + +- **Duplication of maintenance.** JSEP and the native WebGPU EP implement the same operators twice — once in + TypeScript and once in C++. The native EP is the strategic direction and receives the bulk of new op work; + JSEP is effectively a second implementation that must be kept in lockstep. +- **Single, consistent runtime semantics.** Consolidating on the native EP means the browser build shares kernel + behavior with the rest of ONNX Runtime rather than maintaining a parallel TS implementation. +- **Smaller/simpler build matrix.** Removing JSEP lets us delete the temporary `USE_WEBGPU_EP` / `DISABLE_JSEP` + build plumbing. + +--- + +## 3. Goals and non-goals + +### Goals + +- Make the native WebGPU EP the default WebGPU/WebNN backend for `onnxruntime-web`. +- Do so **transparently** for the common consumer (same import, same `webgpu` backend key, same public API). +- Give existing JSEP consumers a clearly-communicated, low-effort migration path plus a one-release safety net. +- Remove the JSEP TypeScript compute path and its build variants. + +### Non-goals + +- **WebGL removal** — tracked separately in + [onnxruntime_web_remove_webgl_backend.md](onnxruntime_web_remove_webgl_backend.md). +- **No change to the Node binding** (`onnxruntime-node` / `ort.node.*`). +- **No change to the React-Native package.** +- **No change to ORT training-web.** +- Not introducing new WebGPU features — this is a consolidation, not a feature effort. + +--- + +## 4. Background: how backend selection works today + +Backend choice is a **build-time** decision, not a runtime toggle. It is gated by `BUILD_DEFS` flags baked into +each bundle at build time: + +- `BUILD_DEFS.DISABLE_JSEP` +- `BUILD_DEFS.DISABLE_WEBGPU` (native EP) +- `BUILD_DEFS.DISABLE_WASM` + +The pivot lives in `js/web/script/build.ts` via a temporary `USE_WEBGPU_EP` flag (default `false`). Its +`DEFAULT_DEFINE` sets `DISABLE_JSEP = !!USE_WEBGPU_EP` and `DISABLE_WEBGPU = !USE_WEBGPU_EP`, so a single flag +chooses JSEP vs. the native EP for a given build. Both `USE_WEBGPU_EP` and `USE_JSPI` are already annotated in +the source as temporary and slated for removal. + +### 4.1 Current bundle / export map (`js/web/package.json`) + +| Export | Artifact | WebGPU/WebNN path | +|---|---|---| +| `.` (default) | `ort.min` / `ort.bundle.min` | JSEP WebGPU + WebNN | +| `./all` | `ort.all.*` | JSEP WebGPU + WebNN (+ WebGL) | +| `./webgpu` | `ort.webgpu.*` | **Native** WebGPU EP + native WebNN | +| `./jspi` | `ort.jspi.*` | Native WebGPU EP + JSPI | +| `./wasm` | `ort.wasm.*` | WASM/CPU only | + +The `.` (default) and `./all` bundles are the **JSEP** WebGPU path today. `./webgpu` and `./jspi` are already the +**native** EP. WebNN in the default/`all` bundles is **JSEP-hosted**; in `./webgpu` it is native. + +The raw WASM artifacts are variant-specific: `ort-wasm-simd-threaded.jsep.wasm` (JSEP), +`ort-wasm-simd-threaded.asyncify.wasm` (native EP, as used by downstream consumers), `*.jspi.wasm`, and the base +`*.wasm`. + +--- + +## 5. Proposed approach + +### 5.1 Transparent default swap (JSEP → native EP) + +The default `.` bundle already registers its WebGPU backend under the `webgpu` registry key. The native EP also +registers under `webgpu`. Because selection is a build-time swap and the registry key is identical, flipping the +default bundle from JSEP to the native EP is **transparent**: existing code that requests the `webgpu` EP +continues to work with no source changes. + +### 5.2 Escape hatch: temporary `onnxruntime-web/jsep` + +Even with strong parity confidence, flipping the default removes the *only* way for a consumer to keep the JSEP +implementation, since: + +- No `/jsep` subpath exists today. +- The only other JSEP-containing bundle is `/all`, which also drags in WebGL. + +To de-risk, Phase 1 ships a **temporary** `onnxruntime-web/jsep` export (built with `USE_WEBGPU_EP=false`) that +pins the JSEP implementation for **one release**. It is marked deprecated and removal-scheduled, and it emits a +one-time warning that doubles as a **bug funnel** — surfacing any native-EP parity gaps before JSEP is deleted. + +### 5.3 `/all` bundle: keep as a converging alias + +`./all` today differs from the default only by including WebGL. After both the WebGL removal (tracked +[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, its contents converge to +**native WebGPU EP + native WebNN**, i.e. identical to `./webgpu` and the default `.`. + +**Decision:** keep `/all` to avoid breaking existing imports. Implement it as a **real alias to the same physical +artifact** as the webgpu/default bundle rather than a separately-built `ort.all.*` that coincidentally matches — +this avoids bundle drift, drops a build target, and prevents doubling the CDN/WASM payload. The backing WASM also +shifts from `.jsep.wasm` to `.asyncify.wasm` as part of the general WASM-filename migration. The name becomes a +mild misnomer ("all" no longer implies WebGL); this is documented but not worth an export break. + +**Sequencing:** the WebGL-removal effort drops WebGL from `/all` first (while it is still JSEP); this JSEP → native +swap then converges `/all` onto the native artifact. Each effort owns its half of the `/all` transition. + +--- + +## 6. The int64 gap (analysis and decision) + +### 6.1 Mechanics + +The native EP reads int64 support as a **session config entry** keyed +`ep.webgpuexecutionprovider.enableInt64` (constant `kEnableInt64` in +`onnxruntime/core/providers/webgpu/webgpu_provider_options.h`), via `config_options.TryGetConfigEntry` in +`webgpu_provider_factory.cc`. Values are the strings `"1"` / `"0"`. When absent, the C++ default is +`enable_int64 = false`. + +The web layer never sets it through the typed API: `js/web/lib/wasm/session-options.ts` (`case 'webgpu'`) has no +`enableInt64` handling, and there is no field on `WebGpuExecutionProviderOption`. It is, however, reachable today +on the `/webgpu` bundle via the untyped `extra` map, which `iterateExtraOptions` +(`js/web/lib/wasm/wasm-utils.ts`) flattens into dotted session-config keys: + +```js +extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' } +``` + +### 6.2 Emulation semantics (verified) + +Both JSEP and the native EP, **when int64 is enabled on GPU**, use identical **i32-truncated** emulation: + +- Storage type `vec2`, value/compute type `i32`. +- Reads use only the low word (`.x`) and discard the high word (`.y`). +- Writes from `i32` sign-extend into `vec2`. +- All shader arithmetic is 32-bit. + +When int64 is **disabled** (the native default), int64 ops partition to the CPU/WASM fallback using real +`int64_t` — full, correct 64-bit behavior. + +### 6.3 Fidelity vs. the two failure modes + +- **int64 ON (GPU):** maximum JSEP fidelity (byte-identical to JSEP, including its truncation quirks), but lossy + for genuine large-integer int64 *data* (`|v| ≥ 2³¹`): magnitude truncation, 32-bit overflow wrap, and a value + type that literally cannot hold an arbitrary 64-bit result. +- **int64 OFF (CPU):** maximum correctness (true 64-bit), at the cost of a CPU round-trip for those ops. + +In practice the divergence is **usually moot**: int64 in real models carries indices/shapes/axes/counts/token +IDs, all `≪ 2³¹` (a tensor with `> 2³¹` elements cannot fit in a `< 2GB` buffer). So JSEP (GPU-truncated) and +native-int64-OFF (CPU-full) produce **identical numeric results** for realistic models; the difference is +**performance only**. + +**Prior evidence:** `transformers.js` runs the native EP with int64 **off** (Asyncify build, no `extra`) at +scale on int64-heavy token-ID models, confirming the CPU-fallback cost is acceptable in production. + +### 6.4 Decision + +Keep int64 **off by default** everywhere; use a **uniform native-EP config** across `.` and `/webgpu`. int64 is +**not** a correctness blocker for the swap — the risk is performance, not correctness, and migrating JSEP users +(always-truncating) to native-int64-off is a correctness *upgrade*. + +Follow-up actions (not blockers): + +1. Optionally expose a typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` for discoverability, plus + document the `extra` opt-in. +2. Benchmark an int64-heavy graph (tokenizer / LLM decode) JSEP vs. native-int64-off to quantify CPU-fallback + cost and confirm the perf trade-off. + +### 6.5 Graph-capture interaction (resolved) + +Graph capture and int64 are **coupled inside the EP**, so there is no conflict to design around. In +`webgpu_execution_provider.cc`: + +```cpp +// enable_int64_ is always true when enable_graph_capture_ is true +enable_int64_{config.enable_graph_capture || config.enable_int64}, +``` + +and the graph-capture kernel registry is built with `RegisterKernels(true, true)` (both flags on). Consequences: + +- **Graph capture OFF (default):** int64 stays off → int64 ops on CPU → full 64-bit correctness. +- **Graph capture ON:** the EP **auto-promotes int64 to ON**, keeping int64 ops on-GPU so the captured region is + all-GPU/capturable. + +So leaving `enableInt64` at its default never breaks graph capture — enabling capture flips int64 on +automatically. Graph-capture users implicitly accept the i32-truncation emulation, but that is exactly the +LLM-decode token-ID regime (`≪ 2³¹`) and is already the status quo today. + +--- + +## 7. Open investigation items (to resolve during implementation) + +These are potential parity concerns not yet fully verified in source. The first two could be functional/ +correctness blockers and should be checked before the Phase 2 default flip. + +1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` spins a dedicated worker + (`js/web/lib/wasm/proxy-wrapper.ts`). JSEP runs WebGPU compute over Asyncify; the native EP has its own + threading assumptions. `transformers.js` forces `proxy = false`, so it is **not** prior-art for the proxied + path. Confirm the native EP behaves identically under `wasm.proxy = true`. +2. **IO-binding parity (potential blocker).** Both `'gpu-buffer'` and `'ml-tensor'` output locations are wired in + `js/web/lib/wasm/wasm-core-impl.ts`. Verify the native EP provides the same zero-copy GPU-buffer / ML-tensor + handoff semantics. This will not be caught by op-parity tests. +3. **WebNN native-vs-JSEP parity.** In the default `.` bundle, WebNN is JSEP-hosted; the flip moves those users + to native WebNN as well — a second silent backend swap under the same default bundle. Validate separately from + WebGPU. +4. **Typed-option parity.** Audit whether any typed `WebGpuExecutionProviderOption` field is honored by only one + backend (cache modes, `preferredLayout`, buffer-pool knobs, `forceCpuNodeNames`, etc.). A JSEP-only field + silently becomes a no-op under the native EP. + +--- + +## 8. Phase 1 — Deprecation (this release) + +No default behavior change. Deliverables: + +1. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` inside the `if (!BUILD_DEFS.DISABLE_JSEP)` block, for + `epName` `'webgpu'` (and `'webnn'`). Only JSEP builds warn; native-EP builds never warn. +2. **Shared deprecation-warning utility.** Once-only, respects `env.logLevel`, with an opt-out consideration. + Shared with the WebGL-removal effort. +3. **int64 plumbing (prereq).** Plumb `enableInt64` into `js/web/lib/wasm/session-options.ts` (mirroring the + `enableGraphCapture` handling) so migrating users can opt in if they need JSEP-identical GPU int64. +4. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), marked + deprecated with a scheduled removal release. +5. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; + release notes / CHANGELOG entries. + +### Warning placement rationale + +- **Default `.` bundle (still JSEP in Phase 1):** the JSEP warn-once (#1) applies. After the Phase 2 flip it + becomes native EP and emits nothing — the swap is covered in release notes, not a runtime warning, because it + is not actionable for the consumer. +- **`/jsep` escape-hatch bundle:** warns once ("deprecated JSEP build, removed in vX; file an issue if the native + WebGPU EP does not work for you"). Doubles as a parity bug funnel. + +--- + +## 9. Phase 2 — Removal (next release) + +1. **Flip default:** native WebGPU EP becomes the `webgpu` backend in `ort` / `ort.all`; remove the temporary + `USE_WEBGPU_EP` flag. +2. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports. Repoint + `/all` to the webgpu/default artifact (§5.3). +3. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. +4. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. +5. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. +6. **Remove the temporary `onnxruntime-web/jsep` export.** + +### Removal ergonomics (no tombstones) + +The `/jsep` export is removed **cleanly** — no throwing-stub "tombstone" module and no runtime shim are left +behind. A consumer still importing `onnxruntime-web/jsep` after removal gets the native bundler error ("subpath +./jsep is not defined by exports"), which is an acceptable, immediate build-time failure on this temporary, +one-release, opt-in surface. The default `.` import is unaffected (it silently swaps to the native EP), so no +consumer following the documented path hits an error. The removal is communicated via release notes and the +pinned tracking issue rather than a runtime shim. + +### Release gate + +Before flipping the default, the **web CI test matrix must run the default-bundle suite against the native EP**. +Today that suite exercises JSEP for the `.` bundle; a green `/webgpu` run is not sufficient to prove the default +flip. This is an explicit gate on Phase 2. + +--- + +## 10. Risks and mitigations + +| Risk | Likelihood | Mitigation | +|---|---|---| +| Undiscovered native-EP parity gap vs. JSEP | Medium | Temporary `/jsep` escape hatch (1 release) + warn-once bug funnel; differential tests | +| int64 perf regression (CPU fallback) | Low–Med | int64-off is correctness-preserving; benchmark int64-heavy graph; document `extra`/typed opt-in | +| Proxy-worker path behaves differently | Unknown | **Investigate before flip** (§7.1) | +| IO-binding (gpu-buffer/ml-tensor) semantics differ | Unknown | **Investigate before flip** (§7.2) | +| WebNN behavior changes under native path | Unknown | Validate native WebNN separately (§7.3) | +| WASM artifact filename change breaks `wasmPaths` | Medium | Document migration; call out `.jsep.wasm` → `.asyncify.wasm` | +| No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch as the feedback mechanism | + +--- + +## 11. Migration guide (for consumers) + +- **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the + implementation swaps to the native EP in the removal release. +- **`onnxruntime-web/all`:** continues to work and converges to the native EP. +- **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue + if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. +- **int64-heavy models needing JSEP-identical GPU behavior:** set + `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` (or the typed option once available). Note this is + lossy for genuine large-integer int64 data; the default (CPU fallback) is more correct. +- **`wasmPaths`:** the backing WASM artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update any hardcoded + paths. + +> **Canonical source:** the authoritative, continuously-updated migration guidance for both the JSEP and WebGL +> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG +> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. + +--- + +## 12. Open questions + +1. **Default-flip timing.** Phase 1 vs. Phase 2. Recommendation: Phase 2 (conservative). +2. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so + error/fatal silences it. +3. **WebNN-on-JSEP messaging in Phase 1.** Recommendation: yes — note that `ort` / `ort.all` WebNN moves to the + native path. +4. **int64 typed option in Phase 1.** Recommendation: yes (discoverability), default off. diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md new file mode 100644 index 0000000000000..c17014b422660 --- /dev/null +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -0,0 +1,202 @@ +# Design: Remove the WebGL (onnxjs) backend from onnxruntime-web + +**Status:** Draft +**Last updated:** 2026-07-10 +**Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend + +**Related work:** +[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md). +The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; +see §5 and §6 for the coupling. + +--- + +## 1. Summary + +`onnxruntime-web` ships a legacy **WebGL** backend — the `onnxjs` implementation under `js/web/lib/onnxjs`, +registered under the `'webgl'` backend key. It predates the current WASM architecture, supports a narrower +operator set, and has known fp32/behavioral drift relative to the WebGPU path. + +This document proposes **deprecating and then removing** the WebGL backend. Unlike the JSEP → native WebGPU EP +migration, this is a straightforward **deprecate-and-delete** with an explicit migration message — there is no +transparent redirect (see §5.1). + +The work is split into two phases: + +- **Phase 1 (this release):** ship a deprecation warning + docs. No behavior change. +- **Phase 2 (next release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build + variant. + +--- + +## 2. Motivation + +- **Legacy and unmaintained.** The `onnxjs` WebGL backend predates the WASM/EP architecture and does not receive + new operator or feature work. +- **Narrower op coverage and behavioral drift.** WebGL supports fewer operators than WebGPU and exhibits fp32/op + differences, making it an inconsistent fallback. +- **Build-matrix simplification.** Removing WebGL deletes the `ort.webgl` bundle variant and the + `BUILD_DEFS.DISABLE_WEBGL` plumbing, and drops WebGL from the `ort.all` bundle. + +--- + +## 3. Goals and non-goals + +### Goals + +- Remove the WebGL (`onnxjs`) backend from `onnxruntime-web`. +- Give existing WebGL consumers a clear, actionable migration path (to the WebGPU EP or the WASM/CPU backend). +- Remove the `ort.webgl` build variant and drop WebGL from `ort.all`. + +### Non-goals + +- **JSEP → native WebGPU EP migration** — tracked separately in + [onnxruntime_web_jsep_to_webgpu_ep_migration.md](onnxruntime_web_jsep_to_webgpu_ep_migration.md). +- **No change to the Node binding, React-Native package, or ORT training-web.** + +--- + +## 4. Background + +The WebGL backend is the `onnxjs` implementation in `js/web/lib/onnxjs`, exposed via `backend-onnxjs.ts` and +registered under the `'webgl'` backend key with **negative priority** — meaning it is excluded from the default +fallback ordering and only used when explicitly requested (`executionProviders: ['webgl']`). + +It appears in two bundles: + +| Export | Artifact | Contents | +|---|---|---| +| `./webgl` | `ort.webgl.*` | WebGL only (`DISABLE_WASM: true`) | +| `./all` | `ort.all.*` | JSEP WebGPU + WebNN **+ WebGL** | + +The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or +nothing. + +### 4.1 WebGPU browser coverage (why the fallback gap is small and shrinking) + +A common historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that +is **no longer true**. Per MDN's `api.GPU` browser-compat data: + +| Browser | WebGPU | +|---|---| +| Chrome / Edge (desktop) | ✔️ 113+ | +| Chrome Android | ✔️ 121+ | +| Safari (macOS) | ✔️ 26 | +| Safari (iOS) | ✔️ 26 | +| Chrome / Edge (iOS, via WebKit) | ✔️ 26 | +| Firefox (desktop) | ✔️ 141 | +| Firefox for Android | ❌ | + +WebGPU is now broadly available across current Chrome/Edge, Safari 26 (macOS **and** iOS), and Firefox 141. The +remaining gap is older browser versions, Firefox for Android, and some Linux configurations. This narrows the set +of users for whom WebGL is the *only* GPU path to a small, shrinking tail and strengthens the case for a clean +removal. + +--- + +## 5. Proposed approach + +### 5.1 Explicit removal, not a transparent redirect + +Unlike the WebGPU JSEP → native swap, WebGL cannot be transparently redirected to another backend: + +- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to silently fall back to. +- WebGPU requires `navigator.gpu`; a silent redirect would fail on devices without WebGPU support. +- WebGL exhibits fp32/op drift, so silently swapping results would be a hidden behavioral change. + +Therefore WebGL gets an **explicit deprecation warning + removal** with an actionable migration message directing +users to the WebGPU EP (`webgpu`) or the WASM/CPU backend (`wasm`). + +### 5.2 Warning placement + +The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init()` / +`createInferenceSessionHandler`. Because the backend registers with negative priority, this only fires when WebGL +is **explicitly requested**, so it does not spam consumers who never opt into WebGL. It fires for both `ort.all` +and `ort.webgl`. + +### 5.3 Removal ergonomics (no tombstones) + +When the backend is removed in Phase 2, we do **not** ship throwing-stub "tombstone" modules for the `./webgl` +export or a runtime shim for the `'webgl'` backend key. The rationale: + +- Removing the `./webgl` export means a lingering `import 'onnxruntime-web/webgl'` fails with the native bundler + error ("subpath ./webgl is not defined by exports"). That is a clear, immediate, build-time failure — no shim + needed. +- Requesting `executionProviders: ['webgl']` alone after removal already throws the generic "no available backend + found" error from `resolveBackendAndExecutionProviders`. +- WebGL was never a production-grade, first-class backend (narrow op set, fp32 drift, negative priority), so an + elaborate soft-landing is not warranted. + +The **one** silent case worth guarding is `executionProviders: ['webgl', 'wasm']`: today the unavailable +`'webgl'` entry is silently dropped and the session runs on `wasm`. This is a *correct* outcome (the consumer +gets a working fallback), but it is silent. An optional, cheap safeguard is a **single non-silent** warn in +`resolveBackendAndExecutionProviders` when a removed backend name is dropped, pointing at the migration guide. +Given the now-broad WebGPU coverage (§4.1) and the fact that `wasm` still runs the model, this is a low-priority +nicety, not a blocker — Phase 1's explicit deprecation warning already does the real communication. + +--- + +## 6. `/all` bundle coupling + +WebGL is one of the two backends in `./all` (the other being JSEP WebGPU/WebNN). Removing WebGL drops it from +`/all`, leaving JSEP (which the +[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) subsequently converges to the native +EP). This effort owns **removing WebGL from `/all`**; the JSEP effort owns the **final convergence** of `/all` +onto the native artifact. `/all` itself is **kept** as an export to avoid breaking imports (see the JSEP doc §5.3 +for the alias decision). + +--- + +## 7. Phase 1 — Deprecation (this release) + +No behavior change. Deliverables: + +1. **WebGL warn-once.** In `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), `init()` / + `createInferenceSessionHandler`. Gate on `logLevel`; dedupe via a module flag. Uses the shared + deprecation-warning utility (built by the JSEP effort, or here — whichever lands first). +2. **Docs.** Deprecation banner + migration guidance in `js/web/README` and `js/web/docs/webgl-operators.md`; + release notes / CHANGELOG entries. + +--- + +## 8. Phase 2 — Removal (next release) + +1. **Delete WebGL:** remove the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and + `backend-onnxjs.ts`. +2. **Remove the `ort.webgl` build variant;** update `build.ts` and `package.json` exports (remove the `./webgl` + export). +3. **Drop WebGL from `ort.all`** (see §6). +4. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_WEBGL` and the code it gates; simplify `index.ts`. + +--- + +## 9. Risks and mitigations + +| Risk | Likelihood | Mitigation | +|---|---|---| +| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Low–Medium | WebGPU now broadly available (§4.1), shrinking the affected tail; explicit migration message to `wasm`; deprecation window + release notes | +| Silent breakage from `./webgl` import removal | Low | Deprecate first; document removal release; no transparent redirect (avoids hidden drift) | +| No telemetry on WebGL adoption | Medium | Warn-once as the feedback mechanism during the deprecation window | + +--- + +## 10. Migration guide (for consumers) + +- **`onnxruntime-web/webgl`:** removed. Migrate to the WebGPU EP (`executionProviders: ['webgpu']`) or the + WASM/CPU backend (`executionProviders: ['wasm']`). There is no automatic redirect because WebGL builds have no + WASM fallback and WebGPU requires `navigator.gpu`. +- **`onnxruntime-web/all`:** continues to work, but no longer includes WebGL. If you relied on WebGL as a + fallback via `/all`, add `wasm` to your `executionProviders` list. + +> **Canonical source:** the authoritative, continuously-updated migration guidance for both the WebGL and JSEP +> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG +> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. + +--- + +## 11. Open questions + +1. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so + error/fatal silences it. (Consistent with the JSEP effort.) +2. **Deprecation window length.** One release (aligned with the JSEP removal) vs. longer. Recommendation: align + with the JSEP removal for a single coordinated backend-consolidation message. From fc62ff6bda717d01ae133a1b976018a9805d266a Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 10 Jul 2026 18:06:06 -0700 Subject: [PATCH 2/6] Address design-doc review feedback Revise the onnxruntime-web backend consolidation design docs per review comments: - WebGL removal doc: reword the WebGPU browser-coverage section to reflect MDN's partial support with platform caveats (Chrome Linux GPU-gen limits; Firefox 141 excluding Linux and Intel Macs) instead of implying universal availability, and revert the fallback-risk likelihood to Medium. - WebGL removal doc: drop the paragraph about the ['webgl','wasm'] case, since the existing resolveBackendAndExecutionProviders warning already covers it and no special handling is needed. - JSEP migration doc: tighten the Phase 2 release gate to explicitly require resolving the Section 7 blockers (proxy-worker, gpu-buffer/ml-tensor IO binding, native WebNN) with targeted coverage, in addition to running the default-bundle suite against the native EP. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 24 ++++++++++-- .../onnxruntime_web_remove_webgl_backend.md | 38 +++++++++---------- 2 files changed, 38 insertions(+), 24 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index f3ab677025996..5df73be08b45b 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -285,9 +285,27 @@ pinned tracking issue rather than a runtime shim. ### Release gate -Before flipping the default, the **web CI test matrix must run the default-bundle suite against the native EP**. -Today that suite exercises JSEP for the `.` bundle; a green `/webgpu` run is not sufficient to prove the default -flip. This is an explicit gate on Phase 2. +Phase 2 (the default flip) is gated on **both** of the following: + +1. **Default-bundle suite runs against the native EP.** The web CI test matrix must exercise the default `.` + bundle against the native WebGPU EP. Today that suite runs JSEP for the `.` bundle; a green `/webgpu` run is + **not** sufficient to prove the default flip, because the two bundles differ in wiring (proxy, IO-binding, + WebNN host) even when op kernels match. +2. **Section 7 blockers resolved with targeted coverage.** The potential blockers in §7 must be closed out before + the flip — not merely tracked. Op-parity tests do **not** exercise these paths, so each needs dedicated + coverage: + - **Proxy-worker path (§7.1):** run the native EP under `wasm.proxy = true` in CI (not just the default + `proxy = false`). + - **IO-binding (§7.2):** add tests for both `'gpu-buffer'` and `'ml-tensor'` output locations that assert the + native EP's zero-copy handoff semantics. + - **Native WebNN (§7.3):** validate the native WebNN path separately from WebGPU, since the default-bundle + WebNN users are silently moved from JSEP-hosted to native WebNN by the same flip. + + §7.4 (typed-option parity) should be audited in the same pass; a JSEP-only option silently becoming a no-op is + a quieter regression than the first three but is closed the same way (test or documented removal). + +Passing gate #1 while leaving gate #2 open would let the design meet its stated bar with acknowledged +blocker-class parity gaps still unverified; both are required. --- diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index c17014b422660..a769bede7b409 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -72,25 +72,28 @@ It appears in two bundles: The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or nothing. -### 4.1 WebGPU browser coverage (why the fallback gap is small and shrinking) +### 4.1 WebGPU browser coverage (expanding, but with platform caveats) A common historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that -is **no longer true**. Per MDN's `api.GPU` browser-compat data: +is **less true than it was**, though MDN's `api.GPU` browser-compat data still reports meaningful caveats — the +Chrome and Firefox entries are **partial**, not blanket support: -| Browser | WebGPU | +| Browser | WebGPU (per MDN BCD) | |---|---| -| Chrome / Edge (desktop) | ✔️ 113+ | -| Chrome Android | ✔️ 121+ | -| Safari (macOS) | ✔️ 26 | -| Safari (iOS) | ✔️ 26 | -| Chrome / Edge (iOS, via WebKit) | ✔️ 26 | -| Firefox (desktop) | ✔️ 141 | +| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); broader/full only in a later version, with Linux limited to newer Intel-gen GPUs | +| Chrome Android | 121+ | +| Safari (macOS) | Full, 26 | +| Safari (iOS) | Full, 26 | +| Chrome / Edge (iOS, via WebKit) | Full, 26 | +| Firefox (desktop) | Partial from 141; **excludes Linux and Intel-based macOS** (Apple-Silicon-first) | | Firefox for Android | ❌ | -WebGPU is now broadly available across current Chrome/Edge, Safari 26 (macOS **and** iOS), and Firefox 141. The -remaining gap is older browser versions, Firefox for Android, and some Linux configurations. This narrows the set -of users for whom WebGL is the *only* GPU path to a small, shrinking tail and strengthens the case for a clean -removal. +So Safari/iOS coverage is now solid, and Chrome/Firefox are moving in the right direction — but there remain real +gaps: older browser versions, Firefox for Android, Linux (both Chrome's GPU-generation limits and Firefox's +exclusion), and Intel-based Macs on Firefox. WebGPU coverage is **broadening but not yet universal**, so the set +of users for whom WebGL is the *only* GPU path is shrinking rather than gone. This supports removing WebGL over a +deprecation window, but the design does **not** rely on WebGPU being universally available — the actionable +fallback for uncovered users is the WASM/CPU backend, not WebGPU. --- @@ -127,13 +130,6 @@ export or a runtime shim for the `'webgl'` backend key. The rationale: - WebGL was never a production-grade, first-class backend (narrow op set, fp32 drift, negative priority), so an elaborate soft-landing is not warranted. -The **one** silent case worth guarding is `executionProviders: ['webgl', 'wasm']`: today the unavailable -`'webgl'` entry is silently dropped and the session runs on `wasm`. This is a *correct* outcome (the consumer -gets a working fallback), but it is silent. An optional, cheap safeguard is a **single non-silent** warn in -`resolveBackendAndExecutionProviders` when a removed backend name is dropped, pointing at the migration guide. -Given the now-broad WebGPU coverage (§4.1) and the fact that `wasm` still runs the model, this is a low-priority -nicety, not a blocker — Phase 1's explicit deprecation warning already does the real communication. - --- ## 6. `/all` bundle coupling @@ -174,7 +170,7 @@ No behavior change. Deliverables: | Risk | Likelihood | Mitigation | |---|---|---| -| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Low–Medium | WebGPU now broadly available (§4.1), shrinking the affected tail; explicit migration message to `wasm`; deprecation window + release notes | +| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Medium | WebGPU coverage is broadening but still has platform caveats (§4.1); the actionable fallback for uncovered users is `wasm`, not WebGPU; explicit migration message + deprecation window + release notes | | Silent breakage from `./webgl` import removal | Low | Deprecate first; document removal release; no transparent redirect (avoids hidden drift) | | No telemetry on WebGL adoption | Medium | Warn-once as the feedback mechanism during the deprecation window | From 288864a74e02206922291c0b8ba1669222d2ebdb Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Tue, 14 Jul 2026 11:55:37 -0700 Subject: [PATCH 3/6] Tighten onnxruntime-web backend deprecation design docs Condense both the JSEP -> native WebGPU EP migration and the WebGL removal design docs, trimming the derivation narrative while preserving the settled decisions and rationale. Collapse the int64 analysis subsections into a single section, reduce the open-investigation source/runtime status to concise lines, and reframe the removal-ergonomics sections around what each phase does rather than what it omits. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 337 ++++++++---------- .../onnxruntime_web_remove_webgl_backend.md | 101 +++--- 2 files changed, 188 insertions(+), 250 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 5df73be08b45b..ccbf071c3600a 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,12 +1,11 @@ # Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP **Status:** Draft -**Last updated:** 2026-07-10 +**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends -**Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md). -The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; see -§5.3 and §8 for the coupling. +**Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md) +— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility (see §5.3, §8). --- @@ -14,22 +13,21 @@ The two efforts are independent but share the `onnxruntime-web/all` bundle and t `onnxruntime-web` implements WebGPU/WebNN compute two ways: -- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. This is - the current default (`.`) and `./all` bundles. -- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). This is - the current `./webgpu` and `./jspi` bundles. +- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. Powers the + current default (`.`) and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). Powers the + current `./webgpu` and `./jspi` bundles. -This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU execution -provider** as the single WebGPU/WebNN path. The migration is designed to be **transparent** for the common case -(the default bundle keeps the same `webgpu` backend key and swaps the implementation underneath at build time), -with a **temporary escape hatch** for one release as insurance against undiscovered parity gaps. +This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU EP** as the single +WebGPU/WebNN path. The swap is **transparent** for the common case — the default bundle keeps the `webgpu` backend +key and changes implementation at build time — with a **temporary one-release escape hatch** as insurance against +undiscovered parity gaps. -The work is split into two phases: - -- **Phase 1 (this release):** ship deprecation warnings + docs, plumb the one known config gap (`enableInt64`), - and add a temporary `onnxruntime-web/jsep` escape-hatch export. No default behavior change. -- **Phase 2 (next release):** flip the default bundle to the native EP, delete JSEP, and clean up the temporary - build flags and exports. +- **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native EP, add the temporary + `onnxruntime-web/jsep` escape-hatch export, and ship deprecation warnings + docs. The flip and the hatch ship + together, so the native default and its JSEP fallback coexist for exactly one release. The pinned tracking issue + is published **ahead of** this release as the early-warning channel — no separate warning-only release (§8). +- **Phase 2 (next release):** delete JSEP and remove the temporary `/jsep` export and build flags. --- @@ -109,203 +107,153 @@ continues to work with no source changes. ### 5.2 Escape hatch: temporary `onnxruntime-web/jsep` -Even with strong parity confidence, flipping the default removes the *only* way for a consumer to keep the JSEP -implementation, since: - -- No `/jsep` subpath exists today. -- The only other JSEP-containing bundle is `/all`, which also drags in WebGL. - -To de-risk, Phase 1 ships a **temporary** `onnxruntime-web/jsep` export (built with `USE_WEBGPU_EP=false`) that -pins the JSEP implementation for **one release**. It is marked deprecated and removal-scheduled, and it emits a -one-time warning that doubles as a **bug funnel** — surfacing any native-EP parity gaps before JSEP is deleted. +After the flip, no existing bundle keeps JSEP (no `/jsep` subpath exists today, and the flip moves both `.` and +`/all` to the native EP). To de-risk, Phase 1 adds a **temporary** `onnxruntime-web/jsep` export (built +`USE_WEBGPU_EP=false`) that pins JSEP for **one release**. It is marked deprecated with a scheduled removal and +emits a one-time warning that doubles as a **parity bug funnel** — surfacing native-EP gaps before JSEP is +deleted. ### 5.3 `/all` bundle: keep as a converging alias -`./all` today differs from the default only by including WebGL. After both the WebGL removal (tracked -[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, its contents converge to -**native WebGPU EP + native WebNN**, i.e. identical to `./webgpu` and the default `.`. +`./all` today differs from the default only by including WebGL. After the WebGL removal (tracked +[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, it converges to **native +WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. -**Decision:** keep `/all` to avoid breaking existing imports. Implement it as a **real alias to the same physical -artifact** as the webgpu/default bundle rather than a separately-built `ort.all.*` that coincidentally matches — -this avoids bundle drift, drops a build target, and prevents doubling the CDN/WASM payload. The backing WASM also -shifts from `.jsep.wasm` to `.asyncify.wasm` as part of the general WASM-filename migration. The name becomes a -mild misnomer ("all" no longer implies WebGL); this is documented but not worth an export break. +**Decision:** keep `/all` to avoid breaking imports, implemented as a **real alias to the same physical artifact** +as the webgpu/default bundle rather than a separately-built `ort.all.*` — avoiding bundle drift, dropping a build +target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → `.asyncify.wasm` with the +general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — documented, not worth an +export break. -**Sequencing:** the WebGL-removal effort drops WebGL from `/all` first (while it is still JSEP); this JSEP → native -swap then converges `/all` onto the native artifact. Each effort owns its half of the `/all` transition. +**Sequencing:** the WebGL effort drops WebGL from `/all` first (still JSEP); this swap then converges `/all` onto +the native artifact. Each effort owns its half. --- ## 6. The int64 gap (analysis and decision) -### 6.1 Mechanics - -The native EP reads int64 support as a **session config entry** keyed -`ep.webgpuexecutionprovider.enableInt64` (constant `kEnableInt64` in -`onnxruntime/core/providers/webgpu/webgpu_provider_options.h`), via `config_options.TryGetConfigEntry` in -`webgpu_provider_factory.cc`. Values are the strings `"1"` / `"0"`. When absent, the C++ default is -`enable_int64 = false`. - -The web layer never sets it through the typed API: `js/web/lib/wasm/session-options.ts` (`case 'webgpu'`) has no -`enableInt64` handling, and there is no field on `WebGpuExecutionProviderOption`. It is, however, reachable today -on the `/webgpu` bundle via the untyped `extra` map, which `iterateExtraOptions` -(`js/web/lib/wasm/wasm-utils.ts`) flattens into dotted session-config keys: - -```js -extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' } -``` - -### 6.2 Emulation semantics (verified) - -Both JSEP and the native EP, **when int64 is enabled on GPU**, use identical **i32-truncated** emulation: - -- Storage type `vec2`, value/compute type `i32`. -- Reads use only the low word (`.x`) and discard the high word (`.y`). -- Writes from `i32` sign-extend into `vec2`. -- All shader arithmetic is 32-bit. - -When int64 is **disabled** (the native default), int64 ops partition to the CPU/WASM fallback using real -`int64_t` — full, correct 64-bit behavior. - -### 6.3 Fidelity vs. the two failure modes - -- **int64 ON (GPU):** maximum JSEP fidelity (byte-identical to JSEP, including its truncation quirks), but lossy - for genuine large-integer int64 *data* (`|v| ≥ 2³¹`): magnitude truncation, 32-bit overflow wrap, and a value - type that literally cannot hold an arbitrary 64-bit result. -- **int64 OFF (CPU):** maximum correctness (true 64-bit), at the cost of a CPU round-trip for those ops. - -In practice the divergence is **usually moot**: int64 in real models carries indices/shapes/axes/counts/token -IDs, all `≪ 2³¹` (a tensor with `> 2³¹` elements cannot fit in a `< 2GB` buffer). So JSEP (GPU-truncated) and -native-int64-OFF (CPU-full) produce **identical numeric results** for realistic models; the difference is -**performance only**. - -**Prior evidence:** `transformers.js` runs the native EP with int64 **off** (Asyncify build, no `extra`) at -scale on int64-heavy token-ID models, confirming the CPU-fallback cost is acceptable in production. - -### 6.4 Decision - -Keep int64 **off by default** everywhere; use a **uniform native-EP config** across `.` and `/webgpu`. int64 is -**not** a correctness blocker for the swap — the risk is performance, not correctness, and migrating JSEP users -(always-truncating) to native-int64-off is a correctness *upgrade*. - -Follow-up actions (not blockers): - -1. Optionally expose a typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` for discoverability, plus - document the `extra` opt-in. -2. Benchmark an int64-heavy graph (tokenizer / LLM decode) JSEP vs. native-int64-off to quantify CPU-fallback - cost and confirm the perf trade-off. - -### 6.5 Graph-capture interaction (resolved) - -Graph capture and int64 are **coupled inside the EP**, so there is no conflict to design around. In -`webgpu_execution_provider.cc`: - -```cpp -// enable_int64_ is always true when enable_graph_capture_ is true -enable_int64_{config.enable_graph_capture || config.enable_int64}, -``` - -and the graph-capture kernel registry is built with `RegisterKernels(true, true)` (both flags on). Consequences: - -- **Graph capture OFF (default):** int64 stays off → int64 ops on CPU → full 64-bit correctness. -- **Graph capture ON:** the EP **auto-promotes int64 to ON**, keeping int64 ops on-GPU so the captured region is - all-GPU/capturable. - -So leaving `enableInt64` at its default never breaks graph capture — enabling capture flips int64 on -automatically. Graph-capture users implicitly accept the i32-truncation emulation, but that is exactly the -LLM-decode token-ID regime (`≪ 2³¹`) and is already the status quo today. +**Mechanics.** The native EP reads int64 support from the session-config entry +`ep.webgpuexecutionprovider.enableInt64` (`webgpu_provider_factory.cc`); absent, it defaults to `false`. The web +layer has no typed field for it, but it is reachable today on `/webgpu` via the untyped `extra` map +(`iterateExtraOptions` flattens `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` into the config key). + +**Emulation (verified).** When int64 is **enabled** on GPU, JSEP and the native EP use identical **i32-truncated** +emulation (storage `vec2`, compute `i32`, high word discarded, 32-bit arithmetic). When int64 is **disabled** +(the native default), ops with int64 *inputs* partition to the CPU/WASM fallback using real `int64_t` — full +64-bit correctness. **Exception:** casting *to* int64 stays on GPU even when int64 is off (the WebGPU `Cast` +kernel only guards int64 *inputs*), so "int64 off" narrows to int64 *inputs*, not all int64 work. + +**Why it's not a blocker.** int64 in real models carries indices/shapes/axes/token IDs, all `≪ 2³¹` (a tensor +with `> 2³¹` elements can't fit in a `< 2GB` buffer), so JSEP (GPU-truncated) and native-int64-off (CPU-full) +produce identical numeric results for realistic models — the difference is **performance only**. `transformers.js` +already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the +CPU-fallback cost is acceptable. + +**Decision.** Keep int64 **off by default** everywhere, with a uniform native-EP config across `.` and `/webgpu`. +Migrating JSEP users (always-truncating) to native-int64-off is a correctness *upgrade*. Follow-ups (not +blockers): (1) an **optional** typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` as sugar over the +`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify the +CPU-fallback cost. + +**Graph-capture interaction (resolved).** Graph capture and int64 are coupled inside the EP +(`enable_int64_{config.enable_graph_capture || config.enable_int64}` in `webgpu_execution_provider.cc`): capture +OFF (default) leaves int64 off (int64 ops on CPU, full correctness); capture ON auto-promotes int64 to keep the +captured region all-GPU/capturable. So the `enableInt64` default never breaks graph capture, and capture users +implicitly accept the i32-truncation regime — exactly the LLM-decode token-ID case (`≪ 2³¹`), the status quo +today. --- ## 7. Open investigation items (to resolve during implementation) -These are potential parity concerns not yet fully verified in source. The first two could be functional/ -correctness blockers and should be checked before the Phase 2 default flip. - -1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` spins a dedicated worker - (`js/web/lib/wasm/proxy-wrapper.ts`). JSEP runs WebGPU compute over Asyncify; the native EP has its own - threading assumptions. `transformers.js` forces `proxy = false`, so it is **not** prior-art for the proxied - path. Confirm the native EP behaves identically under `wasm.proxy = true`. -2. **IO-binding parity (potential blocker).** Both `'gpu-buffer'` and `'ml-tensor'` output locations are wired in - `js/web/lib/wasm/wasm-core-impl.ts`. Verify the native EP provides the same zero-copy GPU-buffer / ML-tensor - handoff semantics. This will not be caught by op-parity tests. -3. **WebNN native-vs-JSEP parity.** In the default `.` bundle, WebNN is JSEP-hosted; the flip moves those users - to native WebNN as well — a second silent backend swap under the same default bundle. Validate separately from - WebGPU. -4. **Typed-option parity.** Audit whether any typed `WebGpuExecutionProviderOption` field is honored by only one - backend (cache modes, `preferredLayout`, buffer-pool knobs, `forceCpuNodeNames`, etc.). A JSEP-only field - silently becomes a no-op under the native EP. +Parity concerns to close before the Phase 1 flip. Each is an end-to-end validation: the TypeScript wiring is +confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. + +1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` runs compute in a dedicated worker + (`proxy-wrapper.ts`). *Source:* the wrapper is EP-agnostic and already forbids all GPU I/O over the proxy + (CPU-I/O-only for both backends). *Runtime:* verify the native EP's device init + threading work inside a + Worker under Asyncify. (`transformers.js` forces `proxy = false`, so it is not prior art here.) +2. **IO-binding parity (potential blocker).** *Source:* the `'gpu-buffer'` / `'ml-tensor'` paths in + `wasm-core-impl.ts` have symmetric native/JSEP hooks selected by `BUILD_DEFS.DISABLE_WEBGPU` + (`webgpuRegisterBuffer`/`jsepRegisterBuffer`, etc.). *Runtime:* confirm the native WASM exports deliver true + zero-copy handoff and identical dispose/lifetime semantics. Not covered by op-parity tests. +3. **WebNN native-vs-JSEP parity.** The flip also moves default-bundle WebNN from JSEP-hosted to native — a + second silent swap. *Source:* both init paths are visible in `initEp` (`initJsep('webnn', …)` vs + `webnnInit(...)`). *Runtime:* requires a `navigator.ml`-capable environment; E2E-only. + +**Closed by source audit — typed-option parity.** The native EP branch in `session-options.ts` honors a +**superset** of JSEP's per-session options, so no JSEP-only typed field silently becomes a no-op under native +(JSEP's other configuration comes from the global `env.webgpu.*` object, not per-session options). The only +residual is a source check that those global knobs (profiling in particular) reach the native `webgpuInit` path. --- -## 8. Phase 1 — Deprecation (this release) - -No default behavior change. Deliverables: - -1. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` inside the `if (!BUILD_DEFS.DISABLE_JSEP)` block, for - `epName` `'webgpu'` (and `'webnn'`). Only JSEP builds warn; native-EP builds never warn. -2. **Shared deprecation-warning utility.** Once-only, respects `env.logLevel`, with an opt-out consideration. - Shared with the WebGL-removal effort. -3. **int64 plumbing (prereq).** Plumb `enableInt64` into `js/web/lib/wasm/session-options.ts` (mirroring the - `enableGraphCapture` handling) so migrating users can opt in if they need JSEP-identical GPU int64. -4. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), marked - deprecated with a scheduled removal release. -5. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; - release notes / CHANGELOG entries. +## 8. Phase 1 — Flip + deprecate (this release) + +Flip the default to the native EP **and** ship the escape hatch, so the swap is protected by a one-release +fallback. Deliverables: + +1. **Flip the default.** Build the default `.` and `./all` bundles with the native EP (`USE_WEBGPU_EP=true`). The + `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). Gated on the release gate below. +2. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), deprecated + with a scheduled removal, shipping in this same release. +3. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` under `if (!BUILD_DEFS.DISABLE_JSEP)`, for `epName` + `'webgpu'` (and `'webnn'`). Only the `/jsep` build warns; the native default emits nothing. +4. **Shared deprecation-warning utility.** Warn once and respect `env.logLevel` (error/fatal silences it); no + separate opt-out flag. Shared with the WebGL-removal effort. +5. **Publish the tracking issue ahead of release.** The pinned migration issue (§11) ships **before** this release + as the early-warning channel, substituting for a separate warning-only release. Consumers can stay on the + current pre-flip version until ready, then adopt the flip release and pin the `/jsep` hatch it introduces if + needed. +6. **int64 `extra` opt-in (documented).** Document `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` for + users needing JSEP-identical GPU int64; works today with no new plumbing, typed field optional (§6). +7. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; a + prominent release-notes/CHANGELOG entry (WASM filename `.jsep.wasm` → `.asyncify.wasm`, `/jsep` pin + instructions). Call out that **WebNN** in `.` / `/all` also moves from JSEP-hosted to native WebNN (the same + path `/webgpu` already ships), so WebNN users know to re-test and use `/jsep` on regression — the + WebGPU-centric framing otherwise hides this second swap (§7.3). ### Warning placement rationale -- **Default `.` bundle (still JSEP in Phase 1):** the JSEP warn-once (#1) applies. After the Phase 2 flip it - becomes native EP and emits nothing — the swap is covered in release notes, not a runtime warning, because it - is not actionable for the consumer. -- **`/jsep` escape-hatch bundle:** warns once ("deprecated JSEP build, removed in vX; file an issue if the native - WebGPU EP does not work for you"). Doubles as a parity bug funnel. +The default `.` bundle (native EP) emits nothing — the implementation swap is not actionable for the consumer and +is communicated via the tracking issue and release notes. The `/jsep` escape-hatch bundle warns once ("deprecated +JSEP build, removed in vX; file an issue if the native WebGPU EP does not work for you"), doubling as a parity bug +funnel. ---- +### Release gate -## 9. Phase 2 — Removal (next release) +The default flip (#1) is gated on **both**: -1. **Flip default:** native WebGPU EP becomes the `webgpu` backend in `ort` / `ort.all`; remove the temporary - `USE_WEBGPU_EP` flag. -2. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports. Repoint - `/all` to the webgpu/default artifact (§5.3). -3. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. -4. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. -5. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. -6. **Remove the temporary `onnxruntime-web/jsep` export.** +1. **Default-bundle suite runs against the native EP.** CI must exercise the default `.` bundle against the native + EP — a green `/webgpu` run is **not** sufficient, since the bundles differ in wiring (proxy, IO-binding, WebNN + host) even when op kernels match. +2. **§7 blockers resolved with targeted coverage** (op-parity tests don't exercise these paths): the native EP + under `wasm.proxy = true` (§7.1); both `'gpu-buffer'` and `'ml-tensor'` IO-binding zero-copy handoff (§7.2); + native WebNN validated separately (§7.3). Typed-option parity is already closed by source audit; the only + residual is a source check that the global `env.webgpu.*` knobs reach the native `webgpuInit` path. -### Removal ergonomics (no tombstones) +Both gates are required — passing #1 while leaving #2 open would ship acknowledged blocker-class parity gaps +unverified. -The `/jsep` export is removed **cleanly** — no throwing-stub "tombstone" module and no runtime shim are left -behind. A consumer still importing `onnxruntime-web/jsep` after removal gets the native bundler error ("subpath -./jsep is not defined by exports"), which is an acceptable, immediate build-time failure on this temporary, -one-release, opt-in surface. The default `.` import is unaffected (it silently swaps to the native EP), so no -consumer following the documented path hits an error. The removal is communicated via release notes and the -pinned tracking issue rather than a runtime shim. +--- -### Release gate +## 9. Phase 2 — Removal (next release) -Phase 2 (the default flip) is gated on **both** of the following: +The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. -1. **Default-bundle suite runs against the native EP.** The web CI test matrix must exercise the default `.` - bundle against the native WebGPU EP. Today that suite runs JSEP for the `.` bundle; a green `/webgpu` run is - **not** sufficient to prove the default flip, because the two bundles differ in wiring (proxy, IO-binding, - WebNN host) even when op kernels match. -2. **Section 7 blockers resolved with targeted coverage.** The potential blockers in §7 must be closed out before - the flip — not merely tracked. Op-parity tests do **not** exercise these paths, so each needs dedicated - coverage: - - **Proxy-worker path (§7.1):** run the native EP under `wasm.proxy = true` in CI (not just the default - `proxy = false`). - - **IO-binding (§7.2):** add tests for both `'gpu-buffer'` and `'ml-tensor'` output locations that assert the - native EP's zero-copy handoff semantics. - - **Native WebNN (§7.3):** validate the native WebNN path separately from WebGPU, since the default-bundle - WebNN users are silently moved from JSEP-hosted to native WebNN by the same flip. +1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove + the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3). +2. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. +3. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. +4. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. +5. **Remove the temporary `onnxruntime-web/jsep` export.** - §7.4 (typed-option parity) should be audited in the same pass; a JSEP-only option silently becoming a no-op is - a quieter regression than the first three but is closed the same way (test or documented removal). +### Removal ergonomics (no tombstones) -Passing gate #1 while leaving gate #2 open would let the design meet its stated bar with acknowledged -blocker-class parity gaps still unverified; both are required. +The `/jsep` export is removed cleanly, relying on the native bundler error: a lingering +`import 'onnxruntime-web/jsep'` fails with "subpath ./jsep is not defined by exports" — an acceptable build-time +failure on this temporary, opt-in surface. The default `.` import is unaffected (it already swaps to the native +EP), so no consumer on the documented path hits an error. Communicated via release notes and the tracking issue. --- @@ -326,7 +274,7 @@ blocker-class parity gaps still unverified; both are required. ## 11. Migration guide (for consumers) - **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the - implementation swaps to the native EP in the removal release. + implementation swaps to the native EP in Phase 1 (this release). - **`onnxruntime-web/all`:** continues to work and converges to the native EP. - **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. @@ -336,17 +284,16 @@ blocker-class parity gaps still unverified; both are required. - **`wasmPaths`:** the backing WASM artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update any hardcoded paths. -> **Canonical source:** the authoritative, continuously-updated migration guidance for both the JSEP and WebGL -> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG -> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. +> **Canonical source:** this effort has its **own** pinned tracking issue (link TBD) as the authoritative, +> continuously-updated migration guidance, independent of the +> [WebGL-removal](onnxruntime_web_remove_webgl_backend.md) issue. Console warnings, README/docs, and CHANGELOG +> entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) links to +> whichever issue is relevant. --- -## 12. Open questions +## 12. Resolved decisions -1. **Default-flip timing.** Phase 1 vs. Phase 2. Recommendation: Phase 2 (conservative). -2. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so - error/fatal silences it. -3. **WebNN-on-JSEP messaging in Phase 1.** Recommendation: yes — note that `ort` / `ort.all` WebNN moves to the - native path. -4. **int64 typed option in Phase 1.** Recommendation: yes (discoverability), default off. +- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal + silences the deprecation warning; no separate opt-out flag is added. (The WebGL-removal effort adopts the same + policy.) diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index a769bede7b409..f79f95368fd86 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -1,13 +1,12 @@ # Design: Remove the WebGL (onnxjs) backend from onnxruntime-web **Status:** Draft -**Last updated:** 2026-07-10 +**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend **Related work:** -[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md). -The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; -see §5 and §6 for the coupling. +[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md) +— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility (see §5–§6). --- @@ -15,15 +14,12 @@ see §5 and §6 for the coupling. `onnxruntime-web` ships a legacy **WebGL** backend — the `onnxjs` implementation under `js/web/lib/onnxjs`, registered under the `'webgl'` backend key. It predates the current WASM architecture, supports a narrower -operator set, and has known fp32/behavioral drift relative to the WebGPU path. +operator set, and has fp32/behavioral drift relative to the WebGPU path. -This document proposes **deprecating and then removing** the WebGL backend. Unlike the JSEP → native WebGPU EP -migration, this is a straightforward **deprecate-and-delete** with an explicit migration message — there is no -transparent redirect (see §5.1). +This document proposes **deprecating and then removing** it as a straightforward **deprecate-and-delete** with an +explicit migration message — no transparent redirect (see §5.1), unlike the JSEP → native WebGPU EP migration: -The work is split into two phases: - -- **Phase 1 (this release):** ship a deprecation warning + docs. No behavior change. +- **Phase 1 (this release):** deprecation warning + docs. No behavior change. - **Phase 2 (next release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build variant. @@ -74,26 +70,22 @@ nothing. ### 4.1 WebGPU browser coverage (expanding, but with platform caveats) -A common historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that -is **less true than it was**, though MDN's `api.GPU` browser-compat data still reports meaningful caveats — the -Chrome and Firefox entries are **partial**, not blanket support: +The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is +**less true**, though MDN's `api.GPU` browser-compat data still shows Chrome and Firefox as **partial**: | Browser | WebGPU (per MDN BCD) | |---|---| -| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); broader/full only in a later version, with Linux limited to newer Intel-gen GPUs | +| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); Linux limited to newer Intel-gen GPUs | | Chrome Android | 121+ | -| Safari (macOS) | Full, 26 | -| Safari (iOS) | Full, 26 | +| Safari (macOS / iOS) | Full, 26 | | Chrome / Edge (iOS, via WebKit) | Full, 26 | | Firefox (desktop) | Partial from 141; **excludes Linux and Intel-based macOS** (Apple-Silicon-first) | | Firefox for Android | ❌ | -So Safari/iOS coverage is now solid, and Chrome/Firefox are moving in the right direction — but there remain real -gaps: older browser versions, Firefox for Android, Linux (both Chrome's GPU-generation limits and Firefox's -exclusion), and Intel-based Macs on Firefox. WebGPU coverage is **broadening but not yet universal**, so the set -of users for whom WebGL is the *only* GPU path is shrinking rather than gone. This supports removing WebGL over a -deprecation window, but the design does **not** rely on WebGPU being universally available — the actionable -fallback for uncovered users is the WASM/CPU backend, not WebGPU. +Safari/iOS is now solid and Chrome/Firefox are improving, but real gaps remain (older versions, Firefox for +Android, Linux, Intel Macs on Firefox). Coverage is **broadening but not universal**, so the pool of users for +whom WebGL is the *only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the +actionable fallback for uncovered users is the WASM/CPU backend. --- @@ -101,45 +93,40 @@ fallback for uncovered users is the WASM/CPU backend, not WebGPU. ### 5.1 Explicit removal, not a transparent redirect -Unlike the WebGPU JSEP → native swap, WebGL cannot be transparently redirected to another backend: +WebGL cannot be transparently redirected (unlike the WebGPU JSEP → native swap): -- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to silently fall back to. -- WebGPU requires `navigator.gpu`; a silent redirect would fail on devices without WebGPU support. -- WebGL exhibits fp32/op drift, so silently swapping results would be a hidden behavioral change. +- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to redirect to. +- WebGPU requires `navigator.gpu`; a silent redirect would fail where WebGPU is unavailable. +- WebGL's fp32/op drift means silently swapping results would be a hidden behavioral change. -Therefore WebGL gets an **explicit deprecation warning + removal** with an actionable migration message directing -users to the WebGPU EP (`webgpu`) or the WASM/CPU backend (`wasm`). +So WebGL gets an **explicit deprecation warning + removal** directing users to the WebGPU EP (`webgpu`) or the +WASM/CPU backend (`wasm`). ### 5.2 Warning placement The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init()` / -`createInferenceSessionHandler`. Because the backend registers with negative priority, this only fires when WebGL -is **explicitly requested**, so it does not spam consumers who never opt into WebGL. It fires for both `ort.all` -and `ort.webgl`. +`createInferenceSessionHandler`. Because the backend has negative priority, it only fires when WebGL is +**explicitly requested** — no spam for consumers who never opt in. Fires for both `ort.all` and `ort.webgl`. ### 5.3 Removal ergonomics (no tombstones) -When the backend is removed in Phase 2, we do **not** ship throwing-stub "tombstone" modules for the `./webgl` -export or a runtime shim for the `'webgl'` backend key. The rationale: +Phase 2 relies on the native failure modes rather than adding tombstone stubs or a `'webgl'`-key shim: + +- A lingering `import 'onnxruntime-web/webgl'` fails with the native bundler error ("subpath ./webgl is not + defined by exports") — a clear build-time failure. +- `executionProviders: ['webgl']` throws "no available backend found" from `resolveBackendAndExecutionProviders`. -- Removing the `./webgl` export means a lingering `import 'onnxruntime-web/webgl'` fails with the native bundler - error ("subpath ./webgl is not defined by exports"). That is a clear, immediate, build-time failure — no shim - needed. -- Requesting `executionProviders: ['webgl']` alone after removal already throws the generic "no available backend - found" error from `resolveBackendAndExecutionProviders`. -- WebGL was never a production-grade, first-class backend (narrow op set, fp32 drift, negative priority), so an - elaborate soft-landing is not warranted. +WebGL was never a first-class backend (narrow op set, fp32 drift, negative priority), so these built-in errors are +a sufficient soft-landing. --- ## 6. `/all` bundle coupling -WebGL is one of the two backends in `./all` (the other being JSEP WebGPU/WebNN). Removing WebGL drops it from -`/all`, leaving JSEP (which the -[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) subsequently converges to the native -EP). This effort owns **removing WebGL from `/all`**; the JSEP effort owns the **final convergence** of `/all` -onto the native artifact. `/all` itself is **kept** as an export to avoid breaking imports (see the JSEP doc §5.3 -for the alias decision). +`./all` holds two backends: WebGL and JSEP WebGPU/WebNN. This effort owns **removing WebGL from `/all`**; the +[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns the **final convergence** of the +remaining JSEP backend onto the native EP. `/all` is **kept** as an export to avoid breaking imports (see JSEP doc +§5.3 for the alias decision). --- @@ -184,15 +171,19 @@ No behavior change. Deliverables: - **`onnxruntime-web/all`:** continues to work, but no longer includes WebGL. If you relied on WebGL as a fallback via `/all`, add `wasm` to your `executionProviders` list. -> **Canonical source:** the authoritative, continuously-updated migration guidance for both the WebGL and JSEP -> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG -> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. +> **Canonical source:** this effort has its **own** pinned tracking issue (link TBD) as the authoritative, +> continuously-updated migration guidance, independent of the +> [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) issue. Console warnings, README/docs, +> and CHANGELOG entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) +> links to whichever issue is relevant. --- -## 11. Open questions +## 11. Resolved decisions -1. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so - error/fatal silences it. (Consistent with the JSEP effort.) -2. **Deprecation window length.** One release (aligned with the JSEP removal) vs. longer. Recommendation: align - with the JSEP removal for a single coordinated backend-consolidation message. +- **Deprecation window length.** Default to **one release**, kept **independent** of the JSEP removal's schedule + rather than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant + consumers report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any + extension. +- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal + silences the deprecation warning; no separate opt-out flag is added. (Same policy as the JSEP effort.) From 3788336138e3f6228dff7faa72f51f1d3c711db1 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Tue, 14 Jul 2026 12:39:25 -0700 Subject: [PATCH 4/6] Reconcile /all sequencing and refine onnxruntime-web backend deprecation docs Make the /all bundle convergence order-independent across both design docs: the JSEP->native flip and the WebGL removal are independent efforts that may land in either order or different releases, and /all collapses into a single physical alias only once both have landed (whichever lands second performs the repoint). Also: elevate the global env.webgpu.* settings parity gap to a prerequisite (JSEP open item, release gate, and risk row); correct the WebGPU browser-coverage table to MDN's current data (Chrome/Edge full from 144); fold the WebGL deprecation-window decision into the Phase 2 section; drop the Status: Draft lines and the redundant Resolved-decisions sections; and align Phase 2 wording (subsequent release). Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 80 +++++++++++-------- .../onnxruntime_web_remove_webgl_backend.md | 61 +++++++------- 2 files changed, 74 insertions(+), 67 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index ccbf071c3600a..0a4cd52bb2dfc 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,6 +1,5 @@ # Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP -**Status:** Draft **Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends @@ -27,7 +26,7 @@ undiscovered parity gaps. `onnxruntime-web/jsep` escape-hatch export, and ship deprecation warnings + docs. The flip and the hatch ship together, so the native default and its JSEP fallback coexist for exactly one release. The pinned tracking issue is published **ahead of** this release as the early-warning channel — no separate warning-only release (§8). -- **Phase 2 (next release):** delete JSEP and remove the temporary `/jsep` export and build flags. +- **Phase 2 (subsequent release):** delete JSEP and remove the temporary `/jsep` export and build flags. --- @@ -115,18 +114,22 @@ deleted. ### 5.3 `/all` bundle: keep as a converging alias -`./all` today differs from the default only by including WebGL. After the WebGL removal (tracked -[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, it converges to **native -WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. +`./all` today bundles two **independent** things: the WebGPU/WebNN backend (JSEP) **and** WebGL. Two independent +efforts change it — this migration flips its WebGPU/WebNN half from JSEP to the native EP, and the +[WebGL removal](onnxruntime_web_remove_webgl_backend.md) drops WebGL. Once **both** have landed, `/all` converges +to **native WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. -**Decision:** keep `/all` to avoid breaking imports, implemented as a **real alias to the same physical artifact** -as the webgpu/default bundle rather than a separately-built `ort.all.*` — avoiding bundle drift, dropping a build -target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → `.asyncify.wasm` with the -general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — documented, not worth an -export break. +**Decision:** keep `/all` to avoid breaking imports. Its **end state** is a **real alias to the same physical +artifact** as the webgpu/default bundle (rather than a separately-built `ort.all.*`) — avoiding bundle drift, +dropping a build target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → +`.asyncify.wasm` with the general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — +documented, not worth an export break. -**Sequencing:** the WebGL effort drops WebGL from `/all` first (still JSEP); this swap then converges `/all` onto -the native artifact. Each effort owns its half. +**Sequencing (order-independent).** The native flip (this effort) and the WebGL drop (the WebGL effort) are +independent and may land in either order, or in different releases. Until both land, `/all` stays a **distinct +bundle**: after this effort's Phase 1 flip but before WebGL is removed, `/all` is **native WebGPU/WebNN + WebGL** +(still not identical to default/`./webgpu`). Whichever effort lands second collapses `/all` into the physical +alias — see §9 (this effort's Phase 2) and the WebGL doc §8. --- @@ -166,8 +169,9 @@ today. ## 7. Open investigation items (to resolve during implementation) -Parity concerns to close before the Phase 1 flip. Each is an end-to-end validation: the TypeScript wiring is -confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. +Parity concerns to close before the Phase 1 flip. Most are end-to-end validations — the TypeScript wiring is +confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. Item +4 is not a runtime unknown but a known, addressable wiring gap — a prerequisite, not a blocker. 1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` runs compute in a dedicated worker (`proxy-wrapper.ts`). *Source:* the wrapper is EP-agnostic and already forbids all GPU I/O over the proxy @@ -180,11 +184,21 @@ confirmable by source audit (noted below), but the runtime behavior needs a real 3. **WebNN native-vs-JSEP parity.** The flip also moves default-bundle WebNN from JSEP-hosted to native — a second silent swap. *Source:* both init paths are visible in `initEp` (`initJsep('webnn', …)` vs `webnnInit(...)`). *Runtime:* requires a `navigator.ml`-capable environment; E2E-only. - -**Closed by source audit — typed-option parity.** The native EP branch in `session-options.ts` honors a -**superset** of JSEP's per-session options, so no JSEP-only typed field silently becomes a no-op under native -(JSEP's other configuration comes from the global `env.webgpu.*` object, not per-session options). The only -residual is a source check that those global knobs (profiling in particular) reach the native `webgpuInit` path. +4. **Global `env.webgpu.*` settings parity (prerequisite).** A known, addressable gap: the default bundle's + global WebGPU knobs are **not** honored by the native EP today. *Source:* `wasm-core-impl.ts` reads + `env.webgpu.adapter` / `powerPreference` / `forceFallbackAdapter` only to call `navigator.gpu.requestAdapter()` + on the JS side, then calls `webgpuInit()` **without passing that adapter** to the native EP — the JSEP branch + does forward it to `initJsep`, so the native path drops it. The C++ factory has a `powerPreference` config key + (`webgpu_provider_factory.cc`) that the TS layer never populates, and `startProfiling()` in + `session-handler-inference.ts` is still a `// TODO` on the native path while JSEP consumes + `env.webgpu.profiling.ondata`. So after the flip, consumers relying on these globals silently lose them. + *Action (required before the flip):* either wire the relevant settings into the native path (or typed + per-session replacements), or explicitly document the dropped/changed settings with migration guidance — a + release-gate item (§8). + +**Closed by source audit — per-session typed-option parity.** The native EP branch in `session-options.ts` honors +a **superset** of JSEP's per-session typed options, so no JSEP-only typed field silently becomes a no-op under +native. JSEP's *global* `env.webgpu.*` configuration is a separate, still-open gap — see item 4. --- @@ -194,7 +208,8 @@ Flip the default to the native EP **and** ship the escape hatch, so the swap is fallback. Deliverables: 1. **Flip the default.** Build the default `.` and `./all` bundles with the native EP (`USE_WEBGPU_EP=true`). The - `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). Gated on the release gate below. + `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). `/all` still bundles WebGL until + the WebGL effort removes it, so it stays a distinct bundle for now (§5.3). Gated on the release gate below. 2. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), deprecated with a scheduled removal, shipping in this same release. 3. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` under `if (!BUILD_DEFS.DISABLE_JSEP)`, for `epName` @@ -229,26 +244,30 @@ The default flip (#1) is gated on **both**: host) even when op kernels match. 2. **§7 blockers resolved with targeted coverage** (op-parity tests don't exercise these paths): the native EP under `wasm.proxy = true` (§7.1); both `'gpu-buffer'` and `'ml-tensor'` IO-binding zero-copy handoff (§7.2); - native WebNN validated separately (§7.3). Typed-option parity is already closed by source audit; the only - residual is a source check that the global `env.webgpu.*` knobs reach the native `webgpuInit` path. + native WebNN validated separately (§7.3); and the global `env.webgpu.*` settings gap (§7.4) closed by wiring + `adapter` / `powerPreference` / `forceFallbackAdapter` / `profiling` into the native path (or typed per-session + replacements), or by documenting the dropped/changed settings with migration guidance. Per-session + typed-option parity is already closed by source audit. Both gates are required — passing #1 while leaving #2 open would ship acknowledged blocker-class parity gaps unverified. --- -## 9. Phase 2 — Removal (next release) +## 9. Phase 2 — Removal (subsequent release) The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. 1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove - the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3). + the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3) — valid only once + WebGL has also been dropped from `/all` (WebGL doc §8); if WebGL is still present, `/all` stays a distinct + bundle and the repoint waits on that removal. 2. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. 3. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. 4. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. 5. **Remove the temporary `onnxruntime-web/jsep` export.** -### Removal ergonomics (no tombstones) +### Removal ergonomics The `/jsep` export is removed cleanly, relying on the native bundler error: a lingering `import 'onnxruntime-web/jsep'` fails with "subpath ./jsep is not defined by exports" — an acceptable build-time @@ -266,12 +285,13 @@ EP), so no consumer on the documented path hits an error. Communicated via relea | Proxy-worker path behaves differently | Unknown | **Investigate before flip** (§7.1) | | IO-binding (gpu-buffer/ml-tensor) semantics differ | Unknown | **Investigate before flip** (§7.2) | | WebNN behavior changes under native path | Unknown | Validate native WebNN separately (§7.3) | +| Global `env.webgpu.*` settings (adapter/powerPreference/forceFallbackAdapter/profiling) silently dropped on native | Medium | Wire into the native path or document dropped/changed settings with migration guidance — release gate (§7.4, §8) | | WASM artifact filename change breaks `wasmPaths` | Medium | Document migration; call out `.jsep.wasm` → `.asyncify.wasm` | | No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch as the feedback mechanism | --- -## 11. Migration guide (for consumers) +## 11. Migration guide - **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the implementation swaps to the native EP in Phase 1 (this release). @@ -289,11 +309,3 @@ EP), so no consumer on the documented path hits an error. Communicated via relea > [WebGL-removal](onnxruntime_web_remove_webgl_backend.md) issue. Console warnings, README/docs, and CHANGELOG > entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) links to > whichever issue is relevant. - ---- - -## 12. Resolved decisions - -- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal - silences the deprecation warning; no separate opt-out flag is added. (The WebGL-removal effort adopts the same - policy.) diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index f79f95368fd86..1dabd06148e80 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -1,6 +1,5 @@ # Design: Remove the WebGL (onnxjs) backend from onnxruntime-web -**Status:** Draft **Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend @@ -20,7 +19,7 @@ This document proposes **deprecating and then removing** it as a straightforward explicit migration message — no transparent redirect (see §5.1), unlike the JSEP → native WebGPU EP migration: - **Phase 1 (this release):** deprecation warning + docs. No behavior change. -- **Phase 2 (next release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build +- **Phase 2 (subsequent release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build variant. --- @@ -68,30 +67,29 @@ It appears in two bundles: The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or nothing. -### 4.1 WebGPU browser coverage (expanding, but with platform caveats) +### 4.1 WebGPU browser coverage The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is -**less true**, though MDN's `api.GPU` browser-compat data still shows Chrome and Firefox as **partial**: +**less true**, though per MDN's `api.GPU` browser-compat data gaps remain: | Browser | WebGPU (per MDN BCD) | |---|---| -| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); Linux limited to newer Intel-gen GPUs | -| Chrome Android | 121+ | -| Safari (macOS / iOS) | Full, 26 | -| Chrome / Edge (iOS, via WebKit) | Full, 26 | -| Firefox (desktop) | Partial from 141; **excludes Linux and Intel-based macOS** (Apple-Silicon-first) | +| Chrome / Edge (desktop) | Full from 144 (Linux: Intel Gen12+ only) | +| Chrome Android | Full from 121 | +| Safari (macOS / iOS) | Full from 26 | +| Firefox (desktop) | Partial from 141 | | Firefox for Android | ❌ | -Safari/iOS is now solid and Chrome/Firefox are improving, but real gaps remain (older versions, Firefox for -Android, Linux, Intel Macs on Firefox). Coverage is **broadening but not universal**, so the pool of users for -whom WebGL is the *only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the -actionable fallback for uncovered users is the WASM/CPU backend. +Safari/iOS and Chrome/Edge desktop are solid, but gaps remain (older versions, Firefox for Android, Firefox +desktop still partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the +*only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback +for uncovered users is the WASM/CPU backend. --- ## 5. Proposed approach -### 5.1 Explicit removal, not a transparent redirect +### 5.1 Explicit removal WebGL cannot be transparently redirected (unlike the WebGPU JSEP → native swap): @@ -108,7 +106,7 @@ The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init( `createInferenceSessionHandler`. Because the backend has negative priority, it only fires when WebGL is **explicitly requested** — no spam for consumers who never opt in. Fires for both `ort.all` and `ort.webgl`. -### 5.3 Removal ergonomics (no tombstones) +### 5.3 Removal ergonomics Phase 2 relies on the native failure modes rather than adding tombstone stubs or a `'webgl'`-key shim: @@ -123,10 +121,12 @@ a sufficient soft-landing. ## 6. `/all` bundle coupling -`./all` holds two backends: WebGL and JSEP WebGPU/WebNN. This effort owns **removing WebGL from `/all`**; the -[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns the **final convergence** of the -remaining JSEP backend onto the native EP. `/all` is **kept** as an export to avoid breaking imports (see JSEP doc -§5.3 for the alias decision). +`./all` bundles two **independent** things: WebGL **and** the WebGPU/WebNN backend. This effort owns **removing +WebGL from `/all`**; the [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns flipping +the WebGPU/WebNN half from JSEP to the native EP. The two changes are independent and may land in either order, or +in different releases; `/all` collapses into a single physical alias of the webgpu/default bundle only once +**both** have landed (whichever lands second performs the repoint). `/all` is **kept** as an export to avoid +breaking imports (see JSEP doc §5.3 for the alias decision). --- @@ -142,13 +142,19 @@ No behavior change. Deliverables: --- -## 8. Phase 2 — Removal (next release) +## 8. Phase 2 — Removal (subsequent release) + +Timed a **single release** after Phase 1 by default, kept **independent** of the JSEP removal's schedule rather +than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant consumers +report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any extension. 1. **Delete WebGL:** remove the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and `backend-onnxjs.ts`. 2. **Remove the `ort.webgl` build variant;** update `build.ts` and `package.json` exports (remove the `./webgl` export). -3. **Drop WebGL from `ort.all`** (see §6). +3. **Drop WebGL from `ort.all`** (see §6). If the JSEP → native flip has already landed, this is the step that + collapses `/all` into the webgpu/default alias; if not, `/all` remains a distinct JSEP WebGPU/WebNN bundle + until that flip lands. 4. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_WEBGL` and the code it gates; simplify `index.ts`. --- @@ -163,7 +169,7 @@ No behavior change. Deliverables: --- -## 10. Migration guide (for consumers) +## 10. Migration guide - **`onnxruntime-web/webgl`:** removed. Migrate to the WebGPU EP (`executionProviders: ['webgpu']`) or the WASM/CPU backend (`executionProviders: ['wasm']`). There is no automatic redirect because WebGL builds have no @@ -176,14 +182,3 @@ No behavior change. Deliverables: > [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) issue. Console warnings, README/docs, > and CHANGELOG entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) > links to whichever issue is relevant. - ---- - -## 11. Resolved decisions - -- **Deprecation window length.** Default to **one release**, kept **independent** of the JSEP removal's schedule - rather than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant - consumers report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any - extension. -- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal - silences the deprecation warning; no separate opt-out flag is added. (Same policy as the JSEP effort.) From e3c889f7ef7f4d4a8540fcd2a8419b42513e8387 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Tue, 14 Jul 2026 12:45:34 -0700 Subject: [PATCH 5/6] Document JSEP deprecation-window extension in Phase 2 Mirror the WebGL doc: note the single-release default for the /jsep escape hatch and that extension is driven by native-EP parity regressions surfacing via /jsep fallback usage, not a fixed calendar. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 0a4cd52bb2dfc..99a599e96a936 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -258,6 +258,10 @@ unverified. The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. +Timed a **single release** after Phase 1 by default, keeping the `/jsep` escape hatch (and thus the JSEP +implementation) available for exactly one release. Extend only if native-EP parity regressions surface in the wild +— real fallback usage via `/jsep`, not a fixed calendar, drives any extension. + 1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3) — valid only once WebGL has also been dropped from `/all` (WebGL doc §8); if WebGL is still present, `/all` stays a distinct From a358c0a1a2c803338cf55b0264e90ae48e7647fe Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Wed, 15 Jul 2026 10:46:50 -0700 Subject: [PATCH 6/6] Address PR 29716 review feedback on onnxruntime-web deprecation docs - WebGPU coverage table (WebGL doc 4.1): switch from pinned MDN 'full support' versions to qualitative status, note WebGPU has shipped by default in Chrome/Edge since 113, keep Firefox as plain 'partial' (no roadmap claim), and link MDN api.GPU as the live source (per Copilot reviewer). - Migration guide (JSEP doc 11): add guidance to prefer onnxruntime-web/jspi on JSPI-capable browsers for a smaller WASM and lower per-call overhead, with Asyncify as the universal fallback (per qjia7). - int64 gap (JSEP doc 6): replace the misleading 'CPU-fallback cost' phrasing with the actual cost (CPU/WASM execution of int64-input ops plus partition-boundary copies; no GPU-to-CPU download of int64 data) (per qjia7). - Remove the Last updated field from both docs. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 14 ++++++---- .../onnxruntime_web_remove_webgl_backend.md | 28 ++++++++++--------- 2 files changed, 24 insertions(+), 18 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 99a599e96a936..147f99aae7cac 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,6 +1,5 @@ # Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP -**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends **Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md) @@ -149,14 +148,15 @@ kernel only guards int64 *inputs*), so "int64 off" narrows to int64 *inputs*, no **Why it's not a blocker.** int64 in real models carries indices/shapes/axes/token IDs, all `≪ 2³¹` (a tensor with `> 2³¹` elements can't fit in a `< 2GB` buffer), so JSEP (GPU-truncated) and native-int64-off (CPU-full) produce identical numeric results for realistic models — the difference is **performance only**. `transformers.js` -already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the -CPU-fallback cost is acceptable. +already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the overhead of +running those int64-input ops on CPU/WASM (plus the partition-boundary copies — int64 inputs typically originate on +CPU, and results are uploaded back to the GPU) is acceptable. **Decision.** Keep int64 **off by default** everywhere, with a uniform native-EP config across `.` and `/webgpu`. Migrating JSEP users (always-truncating) to native-int64-off is a correctness *upgrade*. Follow-ups (not blockers): (1) an **optional** typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` as sugar over the -`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify the -CPU-fallback cost. +`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify that +cost (CPU/WASM execution of the int64-input ops plus the partition-boundary copies). **Graph-capture interaction (resolved).** Graph capture and int64 are coupled inside the EP (`enable_int64_{config.enable_graph_capture || config.enable_int64}` in `webgpu_execution_provider.cc`): capture @@ -300,6 +300,10 @@ EP), so no consumer on the documented path hits an error. Communicated via relea - **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the implementation swaps to the native EP in Phase 1 (this release). - **`onnxruntime-web/all`:** continues to work and converges to the native EP. +- **Lower-overhead build (`onnxruntime-web/jspi`):** when you can target JSPI-capable browsers, prefer the `./jspi` + bundle. It runs the **same native WebGPU EP** but bridges async via **JSPI** instead of Asyncify, giving a + smaller WASM binary and lower per-call overhead. The default (Asyncify) build remains the universal fallback for + browsers without JSPI. - **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. - **int64-heavy models needing JSEP-identical GPU behavior:** set diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index 1dabd06148e80..cb033c4be5c3f 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -1,6 +1,5 @@ # Design: Remove the WebGL (onnxjs) backend from onnxruntime-web -**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend **Related work:** @@ -70,20 +69,23 @@ nothing. ### 4.1 WebGPU browser coverage The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is -**less true**, though per MDN's `api.GPU` browser-compat data gaps remain: +**less true** — WebGPU has shipped enabled-by-default in Chrome/Edge since **113** (2023), in Safari (macOS/iOS), +and in Chrome for Android; Firefox support is partial. Per MDN's +[`api.GPU`](https://developer.mozilla.org/docs/Web/API/GPU#browser_compatibility) browser-compat data, gaps +remain: -| Browser | WebGPU (per MDN BCD) | +| Browser | WebGPU | |---|---| -| Chrome / Edge (desktop) | Full from 144 (Linux: Intel Gen12+ only) | -| Chrome Android | Full from 121 | -| Safari (macOS / iOS) | Full from 26 | -| Firefox (desktop) | Partial from 141 | -| Firefox for Android | ❌ | - -Safari/iOS and Chrome/Edge desktop are solid, but gaps remain (older versions, Firefox for Android, Firefox -desktop still partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the -*only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback -for uncovered users is the WASM/CPU backend. +| Chrome / Edge (desktop) | ✅ (default since 113; Linux support landed later) | +| Chrome (Android) | ✅ | +| Safari (macOS / iOS) | ✅ | +| Firefox (desktop) | ⚠️ partial | +| Firefox (Android) | ❌ | + +Chrome/Edge and Safari/iOS are solid, but gaps remain (older versions, Firefox for Android, Firefox desktop still +partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the *only* GPU path +is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback for uncovered +users is the WASM/CPU backend. ---