Description
The current native build script (scripts/build-zstd.sh) compiles Zstandard with -O3, but it is currently missing a few advanced compiler flags. By leveraging Link-Time Optimization (LTO) and modern instruction sets, we can extract more throughput from the native hot path.
Because zstd is split across many .c files, the lack of -flto prevents the compiler from inlining functions across object boundaries. Additionally, the default x86-64 target does not utilize AVX2/BMI2 vector instructions.
Proposed Changes
- Enable LTO: Add
-flto to both the CFLAGS and the final zig cc -shared linking step.
- Target Modern x86-64: Inject
-march=x86-64-v3 for linux-x86_64, osx-x86_64, and windows-x86_64 targets to enable AVX2 instructions.
- Target Modern ARM: Inject
-march=armv8-a+crc for aarch64 targets to enable hardware CRC acceleration.
Code Snippet Reference
Update CFLAGS and linking step in build-zstd.sh:
# 1. Add -flto to CFLAGS
CFLAGS="-O3 -flto -DNDEBUG -DZSTD_DISABLE_ASM=1 -DXXH_NAMESPACE=ZSTD_ $VIS_FLAG \
-I$ZSTD_LIB -I$ZSTD_LIB/common -fPIC"
# 2. Add architecture specific targets
case "$CLASSIFIER" in
*-x86_64) CFLAGS="$CFLAGS -march=x86-64-v3" ;;
*-aarch64) CFLAGS="$CFLAGS -march=armv8-a+crc" ;;
esac
# ... existing compilation loop ...
# 3. Add -flto to the linker
zig cc -target "$ZIG_TARGET" -shared -flto $STRIP_FLAG$SONAME_FLAG $LINK_EXTRA -o "$DEST_DIR/$LIB_NAME" "$WORK"/*.o
# More
[ ] drop DZSTD_DISABLE_ASM
[ ] enable FLTO
[ ] add DZSTD_LEGACY_SUPPORT=4 like zstd-jni
[ ] security hardening -Wl,-z,relro,-z,now
Description
The current native build script (
scripts/build-zstd.sh) compiles Zstandard with-O3, but it is currently missing a few advanced compiler flags. By leveraging Link-Time Optimization (LTO) and modern instruction sets, we can extract more throughput from the native hot path.Because
zstdis split across many.cfiles, the lack of-fltoprevents the compiler from inlining functions across object boundaries. Additionally, the default x86-64 target does not utilize AVX2/BMI2 vector instructions.Proposed Changes
-fltoto both theCFLAGSand the finalzig cc -sharedlinking step.-march=x86-64-v3forlinux-x86_64,osx-x86_64, andwindows-x86_64targets to enable AVX2 instructions.-march=armv8-a+crcforaarch64targets to enable hardware CRC acceleration.Code Snippet Reference
Update
CFLAGSand linking step inbuild-zstd.sh: