From e6479a7c49d03617f1de45d080e0e1c119de748a Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:08:07 -0700 Subject: [PATCH 01/15] Update README with features added since the fork Add sections for fastcache, CUDA graphs, perf_dispatch, Tile16x16, subgroup primitives, autodiff/adstack, BufferView, qd.Tensor, kernel coverage, Python backend, and debug mode. Remove stale gstaichi release references. Link each feature to its user guide page. Co-authored-by: Cursor --- README.md | 53 ++++++++++++++++++++++------------------------------- 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 5f96636320..a5d68e5cf3 100644 --- a/README.md +++ b/README.md @@ -27,58 +27,49 @@ While the repository still resembles upstream in structure, major changes includ ### Modernized infrastructure -* Revamped CI +* Revamped CI with kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) (device-side branch coverage in standard `coverage.py` format) * Support for Python 3.10–3.13 * Support for macOS up to 15 * Significantly improved reliability (≥90% CI success on correct code) ### Structural improvements -* Added `dataclasses.dataclass` structs: - - * Work with both ndarrays and fields - * Can be passed into child `ti.func` functions - * Can be nested - * No kernel runtime overhead (kernels see only underlying arrays) +* `dataclasses.dataclass` structs — work with ndarrays and fields, nestable, passable to `qd.func`, zero kernel-runtime overhead +* [`qd.Tensor`](https://genesis-embodied-ai.github.io/quadrants/user_guide/tensor.html) — unified API over fields and ndarrays with per-tensor layout control, pickle support, and a `backend=` switch +* [`BufferView`](https://genesis-embodied-ai.github.io/quadrants/user_guide/buffer_view.html) — safe sub-range ndarray access with bounds checking in debug mode ### Removed components -To focus the compiler and reduce maintenance burden, we removed: - -* GUI / GGUI -* C-API -* AOT -* DX11 / DX12 -* iOS / Android -* OpenGL / GLES -* argpack -* CLI - -### Performance improvements +To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API, AOT, DX11/DX12, iOS/Android, OpenGL/GLES, argpack, CLI. -#### Reduced launch latency +### Performance -* Release 4.0.0 improved non-batched ndarray CPU performance by **4.5×** in Genesis benchmarks. -* Release 3.2.0 improved ndarray performance from **11× slower than fields** to **1.8× slower** (on a 5090 GPU, Genesis benchmark). +* **Reduced launch latency** — ndarray CPU performance improved **4.5×**; ndarray GPU performance went from 11× slower than fields to 1.8× slower (5090 GPU, Genesis benchmark) +* **[Fastcache](https://genesis-embodied-ai.github.io/quadrants/user_guide/fastcache.html)** — opt-in source-level cache (`@qd.kernel(fastcache=True)`) that bypasses front-end AST parsing; reduces warm-cache kernel load from **7.2 s → 0.3 s** on Genesis benchmarks +* **[CUDA Graphs](https://genesis-embodied-ai.github.io/quadrants/user_guide/graph.html)** — `@qd.kernel(graph=True)` captures kernel sequences into a graph; `qd.graph_do_while` runs GPU-side iteration loops (hardware conditional nodes on SM 9.0+) +* **[perf_dispatch](https://genesis-embodied-ai.github.io/quadrants/user_guide/perf_dispatch.html)** — auto-benchmarks multiple kernel implementations and selects the fastest at runtime +* **[Zero-copy interop](https://genesis-embodied-ai.github.io/quadrants/user_guide/interop.html)** — `to_torch(copy=False)` / `to_numpy(copy=False)` via DLPack on CUDA, CPU, AMDGPU, and Metal; direct torch tensor pass-through into kernels -#### Reduced warm-cache latency +### SIMT primitives -On Genesis simulator (Linux + NVIDIA 5090): +* **[Tile16x16](https://genesis-embodied-ai.github.io/quadrants/user_guide/tile16.html)** — register-resident 16×16 matrix tiles with Cholesky, triangular solve, and rank-1 updates; 5× faster than shared-memory baselines on blocked linear algebra +* **[Subgroup ops](https://genesis-embodied-ai.github.io/quadrants/user_guide/subgroup.html)** — cross-platform `shuffle`, `shuffle_down`, `reduce_add`, `reduce_all_add` across CUDA, AMDGPU, and Vulkan -* `single_franka_envs.py` cache load time reduced from **7.2s → 0.3s** +### Autodiff -#### Zero-copy Torch interop - -* Added `to_dlpack` -* Enables zero-copy memory sharing between PyTorch and Quadrants -* Avoids kernel-based accessors -* Significantly improves performance +* **[Reverse-mode AD with dynamic loops](https://genesis-embodied-ai.github.io/quadrants/user_guide/autodiff.html)** — heap-backed adstack on all backends (LLVM + SPIR-V) with automatic launch-time sizing; no manual stack-size tuning required +* Forward-mode AD, custom gradients (`@qd.ad.grad_replaced`), `qd.ad.Tape` ### Compiler upgrades * Upgraded to LLVM 22 * Enabled ARM support +### Debugging & development + +* **[Python backend](https://genesis-embodied-ai.github.io/quadrants/user_guide/python_backend.html)** — `qd.init(qd.python)` runs kernels as plain Python for debugging without a GPU +* **[Debug mode](https://genesis-embodied-ai.github.io/quadrants/user_guide/debug.html)** — field bounds checking, adstack overflow detection, in-kernel assertions, IR dumping (`QD_DUMP_IR`) + --- # Installation From e414cae676c89d1378f95a4a47124c93a6627920 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:09:29 -0700 Subject: [PATCH 02/15] Rename CUDA Graphs to GPU Graphs in README Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a5d68e5cf3..9cfa12beb9 100644 --- a/README.md +++ b/README.md @@ -46,7 +46,7 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API * **Reduced launch latency** — ndarray CPU performance improved **4.5×**; ndarray GPU performance went from 11× slower than fields to 1.8× slower (5090 GPU, Genesis benchmark) * **[Fastcache](https://genesis-embodied-ai.github.io/quadrants/user_guide/fastcache.html)** — opt-in source-level cache (`@qd.kernel(fastcache=True)`) that bypasses front-end AST parsing; reduces warm-cache kernel load from **7.2 s → 0.3 s** on Genesis benchmarks -* **[CUDA Graphs](https://genesis-embodied-ai.github.io/quadrants/user_guide/graph.html)** — `@qd.kernel(graph=True)` captures kernel sequences into a graph; `qd.graph_do_while` runs GPU-side iteration loops (hardware conditional nodes on SM 9.0+) +* **[GPU Graphs](https://genesis-embodied-ai.github.io/quadrants/user_guide/graph.html)** — `@qd.kernel(graph=True)` captures kernel sequences into a graph; `qd.graph_do_while` runs GPU-side iteration loops (hardware conditional nodes on SM 9.0+) * **[perf_dispatch](https://genesis-embodied-ai.github.io/quadrants/user_guide/perf_dispatch.html)** — auto-benchmarks multiple kernel implementations and selects the fastest at runtime * **[Zero-copy interop](https://genesis-embodied-ai.github.io/quadrants/user_guide/interop.html)** — `to_torch(copy=False)` / `to_numpy(copy=False)` via DLPack on CUDA, CPU, AMDGPU, and Metal; direct torch tensor pass-through into kernels From 2cd2317a26aa15ffcf64989f6bced2409736dc5d Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:17:55 -0700 Subject: [PATCH 03/15] Remove debug mode from README (not new to Quadrants) Co-authored-by: Cursor --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 9cfa12beb9..1c96e4c0ef 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,6 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API ### Debugging & development * **[Python backend](https://genesis-embodied-ai.github.io/quadrants/user_guide/python_backend.html)** — `qd.init(qd.python)` runs kernels as plain Python for debugging without a GPU -* **[Debug mode](https://genesis-embodied-ai.github.io/quadrants/user_guide/debug.html)** — field bounds checking, adstack overflow detection, in-kernel assertions, IR dumping (`QD_DUMP_IR`) --- From 7974baece2f9f7f5bd8f774c9846e25e6122082a Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:19:12 -0700 Subject: [PATCH 04/15] =?UTF-8?q?Fix=20ndarray=20GPU=20perf=20number:=20~3?= =?UTF-8?q?0%=20slower=20than=20fields,=20not=201.8=C3=97?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1c96e4c0ef..2be680ad1d 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API ### Performance -* **Reduced launch latency** — ndarray CPU performance improved **4.5×**; ndarray GPU performance went from 11× slower than fields to 1.8× slower (5090 GPU, Genesis benchmark) +* **Reduced launch latency** — ndarray CPU performance improved **4.5×**; ndarray GPU performance went from 11× slower than fields to ~30% slower (5090 GPU, Genesis benchmark) * **[Fastcache](https://genesis-embodied-ai.github.io/quadrants/user_guide/fastcache.html)** — opt-in source-level cache (`@qd.kernel(fastcache=True)`) that bypasses front-end AST parsing; reduces warm-cache kernel load from **7.2 s → 0.3 s** on Genesis benchmarks * **[GPU Graphs](https://genesis-embodied-ai.github.io/quadrants/user_guide/graph.html)** — `@qd.kernel(graph=True)` captures kernel sequences into a graph; `qd.graph_do_while` runs GPU-side iteration loops (hardware conditional nodes on SM 9.0+) * **[perf_dispatch](https://genesis-embodied-ai.github.io/quadrants/user_guide/perf_dispatch.html)** — auto-benchmarks multiple kernel implementations and selects the fastest at runtime From affbd1fbf9e44fc2e8ceff67c20d867d506f32f0 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:19:31 -0700 Subject: [PATCH 05/15] Add Metal to subgroup ops backend list Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2be680ad1d..128a9393cb 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API ### SIMT primitives * **[Tile16x16](https://genesis-embodied-ai.github.io/quadrants/user_guide/tile16.html)** — register-resident 16×16 matrix tiles with Cholesky, triangular solve, and rank-1 updates; 5× faster than shared-memory baselines on blocked linear algebra -* **[Subgroup ops](https://genesis-embodied-ai.github.io/quadrants/user_guide/subgroup.html)** — cross-platform `shuffle`, `shuffle_down`, `reduce_add`, `reduce_all_add` across CUDA, AMDGPU, and Vulkan +* **[Subgroup ops](https://genesis-embodied-ai.github.io/quadrants/user_guide/subgroup.html)** — cross-platform `shuffle`, `shuffle_down`, `reduce_add`, `reduce_all_add` across CUDA, AMDGPU, Metal and Vulkan ### Autodiff From cd0ebaaa734ad780e112abd4576fbbd04e3d995b Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:20:49 -0700 Subject: [PATCH 06/15] Break out CI into its own README section Co-authored-by: Cursor --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 128a9393cb..8ef57c9fd5 100644 --- a/README.md +++ b/README.md @@ -27,11 +27,17 @@ While the repository still resembles upstream in structure, major changes includ ### Modernized infrastructure -* Revamped CI with kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) (device-side branch coverage in standard `coverage.py` format) * Support for Python 3.10–3.13 * Support for macOS up to 15 * Significantly improved reliability (≥90% CI success on correct code) +### CI + +* Revamped CI pipeline with ≥90% success rate on correct code +* Kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) — device-side branch coverage in standard `coverage.py` format, integrated with pytest-cov +* Per-PR diff coverage reports posted as GitHub Check annotations +* AI-driven checks for line wrapping, deleted comments, test coverage, and feature factorization + ### Structural improvements * `dataclasses.dataclass` structs — work with ndarrays and fields, nestable, passable to `qd.func`, zero kernel-runtime overhead From 520aaeaad391f0b4c8c2dafbfefb43614fc175fa Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:21:59 -0700 Subject: [PATCH 07/15] Fix Python backend description: debugger support, not GPU-less Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8ef57c9fd5..3d54471d2c 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API ### Debugging & development -* **[Python backend](https://genesis-embodied-ai.github.io/quadrants/user_guide/python_backend.html)** — `qd.init(qd.python)` runs kernels as plain Python for debugging without a GPU +* **[Python backend](https://genesis-embodied-ai.github.io/quadrants/user_guide/python_backend.html)** — `qd.init(qd.python)` interprets kernels as plain Python so they can be stepped through in a standard Python debugger --- From 043c58dec176fce441555a13d9f0558fee7c3dc5 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:22:51 -0700 Subject: [PATCH 08/15] Remove per-PR diff coverage line from CI section Co-authored-by: Cursor --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 3d54471d2c..6d3d90b0d6 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,6 @@ While the repository still resembles upstream in structure, major changes includ * Revamped CI pipeline with ≥90% success rate on correct code * Kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) — device-side branch coverage in standard `coverage.py` format, integrated with pytest-cov -* Per-PR diff coverage reports posted as GitHub Check annotations * AI-driven checks for line wrapping, deleted comments, test coverage, and feature factorization ### Structural improvements From 128bb24632f70e06c63a1a00f52fba039129ec2d Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:25:40 -0700 Subject: [PATCH 09/15] Clarify SM 9.0+ as CUDA SM 9.0+ Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6d3d90b0d6..2a2997b038 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API * **Reduced launch latency** — ndarray CPU performance improved **4.5×**; ndarray GPU performance went from 11× slower than fields to ~30% slower (5090 GPU, Genesis benchmark) * **[Fastcache](https://genesis-embodied-ai.github.io/quadrants/user_guide/fastcache.html)** — opt-in source-level cache (`@qd.kernel(fastcache=True)`) that bypasses front-end AST parsing; reduces warm-cache kernel load from **7.2 s → 0.3 s** on Genesis benchmarks -* **[GPU Graphs](https://genesis-embodied-ai.github.io/quadrants/user_guide/graph.html)** — `@qd.kernel(graph=True)` captures kernel sequences into a graph; `qd.graph_do_while` runs GPU-side iteration loops (hardware conditional nodes on SM 9.0+) +* **[GPU Graphs](https://genesis-embodied-ai.github.io/quadrants/user_guide/graph.html)** — `@qd.kernel(graph=True)` captures kernel sequences into a graph; `qd.graph_do_while` runs GPU-side iteration loops (hardware conditional nodes on CUDA SM 9.0+) * **[perf_dispatch](https://genesis-embodied-ai.github.io/quadrants/user_guide/perf_dispatch.html)** — auto-benchmarks multiple kernel implementations and selects the fastest at runtime * **[Zero-copy interop](https://genesis-embodied-ai.github.io/quadrants/user_guide/interop.html)** — `to_torch(copy=False)` / `to_numpy(copy=False)` via DLPack on CUDA, CPU, AMDGPU, and Metal; direct torch tensor pass-through into kernels From 0ccb588a110a533897250a0e1a30634164ea6924 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:26:46 -0700 Subject: [PATCH 10/15] Merge 'Modernized infrastructure' and 'Compiler upgrades' into 'Platform support' Co-authored-by: Cursor --- README.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 2a2997b038..de15ff717c 100644 --- a/README.md +++ b/README.md @@ -25,15 +25,14 @@ This project is now fully independent and does not aim to maintain backward comp While the repository still resembles upstream in structure, major changes include: -### Modernized infrastructure +### Platform support -* Support for Python 3.10–3.13 -* Support for macOS up to 15 -* Significantly improved reliability (≥90% CI success on correct code) +* Python 3.10–3.13 +* macOS up to 15, Windows, Ubuntu 22.04–24.04 +* LLVM 22, ARM (aarch64) support +* Significantly improved reliability (≥90% CI success rate on correct code) ### CI - -* Revamped CI pipeline with ≥90% success rate on correct code * Kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) — device-side branch coverage in standard `coverage.py` format, integrated with pytest-cov * AI-driven checks for line wrapping, deleted comments, test coverage, and feature factorization @@ -65,10 +64,6 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API * **[Reverse-mode AD with dynamic loops](https://genesis-embodied-ai.github.io/quadrants/user_guide/autodiff.html)** — heap-backed adstack on all backends (LLVM + SPIR-V) with automatic launch-time sizing; no manual stack-size tuning required * Forward-mode AD, custom gradients (`@qd.ad.grad_replaced`), `qd.ad.Tape` -### Compiler upgrades - -* Upgraded to LLVM 22 -* Enabled ARM support ### Debugging & development From 64d14cd52399be36ebb7c4999e7038ea9dba7438 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:27:12 -0700 Subject: [PATCH 11/15] Deduplicate CI reliability line: move to CI section Co-authored-by: Cursor --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index de15ff717c..d8b7164e45 100644 --- a/README.md +++ b/README.md @@ -30,9 +30,10 @@ While the repository still resembles upstream in structure, major changes includ * Python 3.10–3.13 * macOS up to 15, Windows, Ubuntu 22.04–24.04 * LLVM 22, ARM (aarch64) support -* Significantly improved reliability (≥90% CI success rate on correct code) ### CI + +* ≥90% CI success rate on correct code * Kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) — device-side branch coverage in standard `coverage.py` format, integrated with pytest-cov * AI-driven checks for line wrapping, deleted comments, test coverage, and feature factorization From c78babf1538a7056fea716f52718ab6a0d213480 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:28:22 -0700 Subject: [PATCH 12/15] Fix macOS version wording Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d8b7164e45..5ba0aa6e9e 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ While the repository still resembles upstream in structure, major changes includ ### Platform support * Python 3.10–3.13 -* macOS up to 15, Windows, Ubuntu 22.04–24.04 +* macOS 15, Windows, Ubuntu 22.04–24.04 * LLVM 22, ARM (aarch64) support ### CI From 9d97afc7713cf9e78ff02fe2ea6a1150442e7ecc Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:28:54 -0700 Subject: [PATCH 13/15] Remove Python/OS version lines (not changes from upstream) Co-authored-by: Cursor --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 5ba0aa6e9e..90d6cf276e 100644 --- a/README.md +++ b/README.md @@ -27,8 +27,6 @@ While the repository still resembles upstream in structure, major changes includ ### Platform support -* Python 3.10–3.13 -* macOS 15, Windows, Ubuntu 22.04–24.04 * LLVM 22, ARM (aarch64) support ### CI From 61c59e0bb519ef7c991c6e18f8a529ac6b605830 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Sun, 3 May 2026 03:29:08 -0700 Subject: [PATCH 14/15] Remove CI success rate claim Co-authored-by: Cursor --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 90d6cf276e..1af085da10 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,6 @@ While the repository still resembles upstream in structure, major changes includ ### CI -* ≥90% CI success rate on correct code * Kernel-level [code coverage](https://genesis-embodied-ai.github.io/quadrants/user_guide/kernel_coverage.html) — device-side branch coverage in standard `coverage.py` format, integrated with pytest-cov * AI-driven checks for line wrapping, deleted comments, test coverage, and feature factorization From 73affbf51e259b6e8bfb6dc60ea94a034b641084 Mon Sep 17 00:00:00 2001 From: Hugh Perkins Date: Mon, 4 May 2026 07:51:59 -0700 Subject: [PATCH 15/15] Use duburcqa's proposed autodiff bullet wording Co-authored-by: Cursor --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1af085da10..76a647002b 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ To focus the compiler and reduce maintenance burden, we removed: GUI/GGUI, C-API ### Autodiff -* **[Reverse-mode AD with dynamic loops](https://genesis-embodied-ai.github.io/quadrants/user_guide/autodiff.html)** — heap-backed adstack on all backends (LLVM + SPIR-V) with automatic launch-time sizing; no manual stack-size tuning required +* [**Autodiff with dynamic loops**](https://genesis-embodied-ai.github.io/quadrants/user_guide/autodiff.html#autodiff-with-dynamic-loops) — computes the gradient of any kernel transparently using reverse-mode differentiation and runtime-based memory allocation * Forward-mode AD, custom gradients (`@qd.ad.grad_replaced`), `qd.ad.Tape`