Skip to content

Optimize the native build #70

Description

@dfa1

Description

The current native build script (scripts/build-zstd.sh) compiles Zstandard with -O3, but it is currently missing a few advanced compiler flags. By leveraging Link-Time Optimization (LTO) and modern instruction sets, we can extract more throughput from the native hot path.

Because zstd is split across many .c files, the lack of -flto prevents the compiler from inlining functions across object boundaries. Additionally, the default x86-64 target does not utilize AVX2/BMI2 vector instructions.

Proposed Changes

  1. Enable LTO: Add -flto to both the CFLAGS and the final zig cc -shared linking step.
  2. Target Modern x86-64: Inject -march=x86-64-v3 for linux-x86_64, osx-x86_64, and windows-x86_64 targets to enable AVX2 instructions.
  3. Target Modern ARM: Inject -march=armv8-a+crc for aarch64 targets to enable hardware CRC acceleration.

Code Snippet Reference

Update CFLAGS and linking step in build-zstd.sh:

# 1. Add -flto to CFLAGS
CFLAGS="-O3 -flto -DNDEBUG -DZSTD_DISABLE_ASM=1 -DXXH_NAMESPACE=ZSTD_ $VIS_FLAG \
        -I$ZSTD_LIB -I$ZSTD_LIB/common -fPIC"

# 2. Add architecture specific targets
case "$CLASSIFIER" in
    *-x86_64) CFLAGS="$CFLAGS -march=x86-64-v3" ;;
    *-aarch64) CFLAGS="$CFLAGS -march=armv8-a+crc" ;;
esac

# ... existing compilation loop ...

# 3. Add -flto to the linker
zig cc -target "$ZIG_TARGET" -shared -flto $STRIP_FLAG$SONAME_FLAG $LINK_EXTRA -o "$DEST_DIR/$LIB_NAME" "$WORK"/*.o

# More
[ ] drop DZSTD_DISABLE_ASM
[ ] enable FLTO
[ ] add DZSTD_LEGACY_SUPPORT=4 like zstd-jni
[ ] security hardening -Wl,-z,relro,-z,now

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions