Skip to content

Commit b4203aa

Browse files
authored
Refresh backend README with current progress (pytorch#20356)
Summary: Updates `backends/webgpu/README.md` to reflect work landed since the last refresh. Moves the Dawn/Tint runtime switch and the per-pass dispatch-ordering / scratch-buffer support out of "In review" into the landed milestones table, and adds three newly landed milestones: fused scaled-dot-product attention (`sdpa_with_kv_cache`) with `update_cache`, GPU timestamp-query profiling, and 4-bit weight-only quantized linear (`linear_q4gsw`). Replaces the "In review" section with the Milestone 3 operators (`embedding_q4gsw`, `apply_rotary_emb`, `prepack`). Adds the newly landed operators to the Operator Support table with their WGSL shader files, and corrects the runtime-engine references from wgpu-native to Dawn (Tint), now the sole native backend. Docs-only; no code changes. Authored with Claude Code. Differential Revision: D108928366
1 parent 66feb0a commit b4203aa

1 file changed

Lines changed: 18 additions & 7 deletions

File tree

backends/webgpu/README.md

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# WebGPU Backend
22

3-
Run ExecuTorch models on the GPU via [WebGPU](https://www.w3.org/TR/webgpu/). The backend compiles delegated subgraphs into WGSL compute shaders executed natively through [wgpu-native](https://github.com/gfx-rs/wgpu-native) (Metal on macOS, Vulkan on Linux/Windows).
3+
Run ExecuTorch models on the GPU via [WebGPU](https://www.w3.org/TR/webgpu/). The backend compiles delegated subgraphs into WGSL compute shaders executed natively through [Dawn](https://dawn.googlesource.com/dawn), whose Tint compiler is the reference WGSL implementation (Metal on macOS, Vulkan on Linux/Windows).
44

5-
> **Status: Prototype.** The backend supports `add` and `rms_norm` today and is under active development. See [Progress](#progress) for shipped milestones.
5+
> **Status: Prototype, under active development.** The backend runs the core of transformer inference today — `add`, `rms_norm`, fused scaled-dot-product attention with KV cache, and 4-bit weight-only quantized linear — plus quantized embedding, rotary embedding, and constant prepacking. See [Progress](#progress) for shipped milestones.
66
77
## Progress
88

@@ -15,13 +15,19 @@ Milestones landed on `main`:
1515
| 2026-06 | Made sure every change is automatically tested — added WebGPU to ExecuTorch's standard backend test suite, running on Linux/x86 in CI | [#19964](https://github.com/pytorch/executorch/pull/19964) |
1616
| 2026-06 | Removed a class of bugs and manual upkeep — the WGSL shaders are now generated automatically, with a build-time check that fails the build on shader/source drift | [#19981](https://github.com/pytorch/executorch/pull/19981) |
1717
| 2026-06 | Got the test suite to actually run work on the GPU — added operator-allowlist delegation (unsupported operations fall back to the CPU) and a process-wide GPU device context, so models execute on the GPU during testing | [#20036](https://github.com/pytorch/executorch/pull/20036) |
18+
| 2026-06 | Made testing match the WebGPU standard exactly — switched the native runtime and tests to Google's Dawn shader compiler (Tint, the source-of-truth WGSL implementation) running on SwiftShader for headless GPU execution | [#20079](https://github.com/pytorch/executorch/pull/20079) |
19+
| 2026-06 | Strengthened correctness for models that run in several GPU passes — added dispatch-ordering and scratch-buffer (temporary GPU memory) support and tests | [#20080](https://github.com/pytorch/executorch/pull/20080) |
20+
| 2026-06 | Added the attention core of transformer inference — fused scaled-dot-product attention (`sdpa_with_kv_cache`) with an `update_cache` operator for autoregressive decode | [#20086](https://github.com/pytorch/executorch/pull/20086), [#20087](https://github.com/pytorch/executorch/pull/20087) |
21+
| 2026-06 | Added on-GPU kernel timing via WebGPU timestamp queries, for true GPU-side profiling | [#20201](https://github.com/pytorch/executorch/pull/20201) |
22+
| 2026-06 | Added the dominant compute in quantized LLMs — 4-bit weight-only quantized linear (`linear_q4gsw`), a dequantize-and-matmul kernel | [#20226](https://github.com/pytorch/executorch/pull/20226), [#20227](https://github.com/pytorch/executorch/pull/20227) |
1823

1924
In review:
2025

2126
| Milestone | Pull Request |
2227
|---|---|
23-
| Makes testing match the WebGPU standard exactly — switches the tests to Google's Dawn shader compiler (Tint, the source-of-truth WGSL implementation) running on SwiftShader for headless GPU execution | [#20079](https://github.com/pytorch/executorch/pull/20079) |
24-
| Strengthens correctness for models that run in several GPU passes — adds dispatch-ordering and scratch-buffer (temporary GPU memory) tests | [#20080](https://github.com/pytorch/executorch/pull/20080) |
28+
| Adds 4-bit quantized embedding (`embedding_q4gsw`) | [#20263](https://github.com/pytorch/executorch/pull/20263) |
29+
| Adds rotary position embedding / RoPE (`apply_rotary_emb`) | [#20264](https://github.com/pytorch/executorch/pull/20264) |
30+
| Adds constant prepacking (`prepack`) for end-to-end model weight handling | [#20265](https://github.com/pytorch/executorch/pull/20265) |
2531

2632
## Architecture
2733

@@ -38,7 +44,7 @@ Edge Dialect IR
3844
.pte file (with VH00/VK00 delegate blob)
3945
4046
41-
Native runtime (wgpu-native → Metal / Vulkan)
47+
Native runtime (Dawn/Tint → Metal / Vulkan)
4248
│ WebGPUGraph::build → creates GPU buffers, pipelines, bind groups
4349
│ WebGPUGraph::execute → encodes + submits compute passes
4450
@@ -56,8 +62,13 @@ Key design choices:
5662
|---|---|---|
5763
| `aten.add.Tensor` | `binary_add.wgsl` | Element-wise with alpha: `out = in1 + alpha * in2` |
5864
| `et_vk.rms_norm.default` | `rms_norm.wgsl` | Root-mean-square normalization |
65+
| `sdpa_with_kv_cache.default` | `sdpa_compute_attn_weights.wgsl`, `sdpa_softmax.wgsl`, `sdpa_compute_out.wgsl` | Fused scaled-dot-product attention (QK / softmax / AV) with KV cache |
66+
| `llama.update_cache.default` | `update_cache.wgsl` | In-place KV cache update for autoregressive decode |
67+
| `et_vk.linear_q4gsw.default` | `q4gsw_linear.wgsl` | 4-bit weight-only quantized linear (dequantize + matmul) |
5968

60-
**Planned:** scaled-dot-product attention (KV cache), quantized linear (4-bit weight-only and 8da4w post-training quantization), quantized embedding, RoPE, `mul`, `sigmoid`, and shape ops (`view`, `permute`, `slice`, `select`, `cat`, `squeeze`/`unsqueeze`).
69+
**In review:** quantized embedding (`embedding_q4gsw`), rotary embedding (`apply_rotary_emb`), and constant prepacking (`prepack`).
70+
71+
**Planned:** `mul`, `sigmoid`, shape ops (`view`, `permute`, `slice`, `select`, `cat`, `squeeze`/`unsqueeze`), and `index` — the remaining ops needed for end-to-end Llama 3.2 1B.
6172

6273
## Quick Start
6374

@@ -107,7 +118,7 @@ backends/webgpu/
107118
│ ├── WebGPUBackend.h/cpp # BackendInterface (init/execute)
108119
│ ├── WebGPUGraph.h/cpp # GPU graph: buffers, pipelines, dispatch
109120
│ ├── WebGPUDelegateHeader.h/cpp # VH00 header parser
110-
│ ├── WebGPUDevice.h/cpp # wgpu-native device abstraction
121+
│ ├── WebGPUDevice.h/cpp # Dawn device abstraction
111122
│ ├── WebGPUUtils.h # Workgroup-size helpers
112123
│ └── ops/
113124
│ ├── OperatorRegistry.h/cpp # Op dispatch table

0 commit comments

Comments
 (0)