|
| 1 | ++++ |
| 2 | +title = "fearless_simd v0.6 brings AVX-512 and ways to disable it" |
| 3 | +authors = ["Shnatsel"] |
| 4 | ++++ |
| 5 | + |
| 6 | +[`fearless_simd`](https://crates.io/crates/fearless_simd) takes `unsafe` out of SIMD. |
| 7 | + |
| 8 | +No matter what level of abstraction you're after, be it autovectorization and multiversioning, portable SIMD, or safe access to raw intrinsics and nothing more, `fearless_simd` has you covered! |
| 9 | + |
| 10 | +It features zero dependencies, from-scratch build time under 1 second, safe public APIs, and [very little](https://gist.github.com/Shnatsel/61fc294987a1e051ce3835c97dc0fc19) `unsafe` under the hood - orders of magnitude less than the alternatives! |
| 11 | + |
| 12 | +The major addition in v0.6 is [AVX-512](https://en.wikipedia.org/wiki/AVX-512) support for all operations, including support for safely invoking AVX-512 intrinsics via the [kernel!](https://docs.rs/fearless_simd/latest/fearless_simd/macro.kernel.html) macro. |
| 13 | + |
| 14 | +## Why AVX-512 |
| 15 | + |
| 16 | +AVX-512 is rather poorly named. |
| 17 | +It would be better to call it AVX3, because extending the vector width to 512 bits is the least interesting addition, and many implementations process 512-bit vectors on 256-bit execution units anyway. |
| 18 | + |
| 19 | +The real meat of AVX-512 is in the new instructions it brings, as well as in doubling the number of registers for _all_ vector widths, or even more if you use masks. |
| 20 | +So you can see significant performance gains from AVX-512 without ever touching 512-bit vectors. |
| 21 | + |
| 22 | +AVX-512 has a bad rep due to the early implementations from Intel being quite poor. |
| 23 | +[Intel Skylake](https://en.wikipedia.org/wiki/Skylake_(microarchitecture)) processors would [immediately reduce the frequencies](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Downclocking) of all CPU cores upon encountering a single AVX-512 instruction anywhere. |
| 24 | +This usually hurts performance more than it helps. |
| 25 | + |
| 26 | +The downclocking issue was fixed by Intel in [Ice Lake](https://en.wikipedia.org/wiki/Ice_Lake_(microprocessor)), and was never present on AMD. |
| 27 | +Because of that `fearless_simd` **only enables AVX-512 on Ice Lake and later** on Intel. |
| 28 | +Using Ice Lake as a baseline also gives us access to more instructions, enabling more efficient implementations of some operations. |
| 29 | + |
| 30 | +We've already seen a [15% end-to-end runtime improvement](https://github.com/linebender/fearless_simd/pull/231#issuecomment-4760357890) compared to AVX2 in [Vello CPU](https://github.com/linebender/vello/tree/main/sparse_strips/vello_cpu). |
| 31 | +This measurement was performed on an (AMD Zen 4 desktop CPU), which doesn't even have native 512-bit vectors, and didn't require any code changes. |
| 32 | + |
| 33 | +## Why not AVX-512 |
| 34 | + |
| 35 | +Hardware has quirks. |
| 36 | +For example, on AMD Zen4, AVX-512 scatter/gather instructions are slower than their scalar equivalent, but LLVM will sometimes emit them when automatically vectorizing loops, resulting in a [dramatic performance drop](https://github.com/llvm/llvm-project/issues/91370) on AVX-512 compared to AVX2 in some cases. |
| 37 | +Intel has same scatter/gather performance pothole on AVX2 in Haswell. |
| 38 | + |
| 39 | +## Controlling the instruction sets |
| 40 | + |
| 41 | +Since the entire point of SIMD is performance, it is crucial for any SIMD abstraction to provide enough control to work around hardware quirks. |
| 42 | + |
| 43 | +In addition, every x86 instruction set increases the binary size, because each SIMD function needs to be separately compiled for every instruction set and the final one selected at runtime. |
| 44 | +This can be mitigated by compiling with e.g. |
| 45 | +`-C target-cpu=x86-64-v3`, which will make the compiled program assume that AVX2 is always present, and omit the SSE4.2 and fallback code. |
| 46 | +But such a program will crash on CPUs without AVX2. |
| 47 | +This also would still compile the AVX-512 implementations, but getting rid of them can be desirable if you know the hardware you target doesn't have AVX-512 or doesn't benefit from it for your workload. |
| 48 | +So the `target-cpu` flag is useful but doesn't provide enough flexibility. |
| 49 | + |
| 50 | +That's why Fearless SIMD now provides controls for disabling certain instruction sets, both [for select functions](https://github.com/linebender/fearless_simd/blob/3d10e36bae31987855e784de907de308803f90e5/fearless_simd/examples/disable_avx2_for_one_function.rs#L1) and [for an entire binary](https://github.com/linebender/fearless_simd/tree/main/fearless_simd#multiversioning-on-x86). |
| 51 | +The defaults are fine for most users, but the flexibility is there if you need it. |
| 52 | + |
| 53 | +It's also worth noting that if binary size is a concern, you should try setting [`codegen-units=1`](https://nnethercote.github.io/perf-book/build-configuration.html#codegen-units) or [`lto=true`](https://nnethercote.github.io/perf-book/build-configuration.html#link-time-optimization) in your Cargo.toml before resorting to disabling SIMD instruction sets. |
| 54 | + |
| 55 | +### Why not Cargo features? |
| 56 | + |
| 57 | +We have considered using Cargo features to control which instruction sets get compiled, but they turned out to be a poor fit. |
| 58 | +Cargo features are a library-level switch, but the decision on which instruction sets are to be included should be made by whoever compiles the final binary, not the library authors. |
| 59 | +(If you are a library author and you really want to get rid of a certain instruction set for parts of the code, you can still [opt out in code rather than config](https://github.com/linebender/fearless_simd/blob/3d10e36bae31987855e784de907de308803f90e5/fearless_simd/examples/disable_avx2_for_one_function.rs)). |
| 60 | +Also, if any dependency anywhere in your tree enables a Cargo feature, you cannot get rid of it without forking and patching the dependency. |
| 61 | + |
| 62 | +We instead use `cfg` flags, which turned out to be the perfect fit. |
| 63 | +They can be set by whoever builds the final binary and apply to the entire dependency tree. |
| 64 | +You can run `RUSTFLAGS=--cfg=disable_dispatch_avx512 cargo build --release` for a one-off build without AVX-512, or set the configuration via [`.cargo/config.toml`](https://doc.rust-lang.org/cargo/reference/config.html) if you want to use it for all builds of your project. |
| 65 | +See [the README](https://github.com/linebender/fearless_simd/tree/main/fearless_simd#multiversioning-on-x86) for details. |
| 66 | + |
| 67 | +This design is somewhat unconventional, but we believe it provides the best solution for this particular problem. |
| 68 | +You can also easily combine it with `-C target-cpu` flags, and the configuration for both is very similar! |
| 69 | + |
| 70 | +## Let us know what you think! |
| 71 | + |
| 72 | +This release closes the last major gap in Fearless SIMD in AVX-512 support and expands on the crate's flexibility. |
| 73 | +We believe `fearless_simd` can become _the_ foundational SIMD crate for the Rust ecosystem, whether you want [automatic vectorization](https://github.com/linebender/fearless_simd/tree/main/fearless_simd#automatic-vectorization), [portable SIMD](https://github.com/linebender/fearless_simd/tree/main/fearless_simd#portable-simd), [safe access to intrinsics](https://github.com/linebender/fearless_simd/tree/main/fearless_simd#explicit-intrinsics), or all of the above. |
| 74 | + |
| 75 | +Are you building something exciting with it? Is there something `fearless_simd` is missing? Please let us know [on Zulip](https://xi.zulipchat.com/#narrow/channel/514230-simd/topic/v0.2E6.2E0/with/609515457)! |
0 commit comments