From 1e08906384384ea77bb49ff88301a97936210cb4 Mon Sep 17 00:00:00 2001 From: enthropy7 Date: Wed, 24 Jun 2026 20:33:54 +0300 Subject: [PATCH] explain how SIMD relates to SIMT, SPMD, and scalable hardware --- beginners-guide.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/beginners-guide.md b/beginners-guide.md index c56873ea4b8..b85e0e2eefe 100644 --- a/beginners-guide.md +++ b/beginners-guide.md @@ -13,6 +13,24 @@ This might seem a tiny bit weird at first, but there's a good reason for it. Bac * One of them you're probably familiar with: Multi-core processors. By giving a processor more than one core, each core can do its own work, and because they're physically distant (at least on the CPU's scale) the heat can still be managed. Unfortunately, not all tasks can just be split up across cores in an efficient way. * The second strategy is SIMD. If you can't make the register go any faster, you can still make the register *wider*. This lets you process more data at a time, which is *almost* as good as just having a faster CPU. As with multi-core programming, SIMD doesn't fit every kind of task, so you have to know when it will improve your program. +## SIMD, SIMT, and SPMD + +SIMD is one point in a larger space of "run the same work over lots of data" designs, and two of its neighbors cause the most confusion: the GPU's *SIMT* model and the *SPMD* programming model. If you're coming from a GPU or threading background, mapping those onto SIMD up front saves a lot of grief. + +**CPU SIMD vs. GPU SIMT.** With SIMD — what this crate is about — you work with an explicit, fixed-width vector. One instruction operates on all of its lanes (the individual slots in a vector; see [Terms](#terms) below) at once, inside a single thread of execution, and you handle any per-lane conditionals yourself with masks. GPUs instead use *SIMT* (Single Instruction, Multiple Threads): you write ordinary-looking scalar code for one "thread", and the hardware runs a whole group of them in lockstep — NVIDIA calls such a group a *warp* (32 threads), AMD a *wavefront* (32 or 64). Under the hood that group really is a vector and the threads are its lanes, but the model hides the vector: each thread has its own registers, addresses its own memory, and when threads take different branches the hardware masks off the inactive ones for you (*divergence*). So SIMD puts the vector — and the bookkeeping for width, masking, and data layout — in your hands, while SIMT presents independent-looking scalar threads and manages the lanes in hardware, with divergence as its main performance cliff. `std::simd` is a CPU SIMD API; it does not target GPUs. + +**SIMD vectors vs. SPMD threads.** *SPMD* (Single Program, Multiple Data) is a programming model: run one program over many data elements, each instance proceeding on its own. SIMT is essentially SPMD on a GPU, and on CPUs compilers such as [ISPC](https://ispc.github.io/) take SPMD-style scalar source and emit SIMD instructions, mapping each program instance onto a lane. The distinction worth holding onto is that a SIMD vector's lanes are *not* independent threads: they share one program counter and advance in lockstep, so they can't individually branch, loop, or block — anything that looks like per-lane control flow is really masking. SPMD and GPU "threads" are independent flows of control. Reaching for `std::simd` means choosing the explicit-vector view rather than the many-independent-instances view. + +## Fixed-Width vs. Scalable Vectors + +SIMD hardware itself comes in two broad styles, and the difference decides whether the *width* of your code is pinned down or flexible. + +**Fixed-width ("packed") SIMD** — x86's SSE/AVX/AVX-512 and Arm's NEON — gives you registers whose size is baked into the instruction set: 128, 256, or 512 bits. You know the lane count at compile time and choose it directly (`f32x4`, `f32x8`, …). This is the model `std::simd` is built around: `Simd` takes a lane count `N` that you fix at compile time. + +**Scalable (length-agnostic) vector architectures** take a different path, going back to the Cray-1 and seen today in RISC-V's "V" extension (RVV) and Arm's SVE/SVE2. They don't bake the vector length into the program. Instead the code is written without committing to a width, and at run time it works through the data in hardware-sized chunks — RVV asks the hardware "how many elements can you do right now?" each iteration (the `vsetvl` instruction and a vector-length register), while SVE uses predicated, "vector-length-agnostic" loops. The *same binary* then uses the full width of both a narrow core and a wide server core, with no recompile. The Cray-1 already had this shape — vector registers plus a vector-length and a vector-mask register. (Arm's SVE sits between the two styles: each chip has a fixed width, chosen as a multiple of 128 bits up to 2048, but because the code is length-agnostic one binary still spans chips of different widths.) Those widths really do vary in the wild: Fujitsu's A64FX, the chip behind the Fugaku supercomputer, runs SVE at 512 bits, AWS's Graviton3 at 256, and most current Armv9 phone and laptop cores at the 128-bit minimum — and the same SVE binary uses whatever each one offers. + +**Where `std::simd` fits.** The portable API is fixed-width today — you commit to a lane count at compile time. That maps directly onto packed SIMD, and on RVV and SVE the compiler can still lower those fixed-size types to scalable instructions when that's the better choice. What it can't do yet is let a single binary *stretch its lane count* to whatever width the scalable hardware offers — that would need variable-size vector types, which portable-simd doesn't have yet. For now, treat the lane count you choose as a fixed property of your code. + ## Terms SIMD has a few special vocabulary terms you should know: