zfs(openzfs): enable OpenZFS on aarch64 (follow-on to #1423) - #1459
Draft
gburd wants to merge 30 commits into
Draft
zfs(openzfs): enable OpenZFS on aarch64 (follow-on to #1423)#1459gburd wants to merge 30 commits into
gburd wants to merge 30 commits into
Conversation
gburd
force-pushed
the
pr/openzfs-aarch64
branch
from
July 31, 2026 12:50
5d12155 to
37ea518
Compare
…nZFS + patch series
Make the in-kernel ZFS implementation selectable at build time and integrate
OpenZFS 2.4.x without maintaining a fork, layered cleanly on the current
master (which already carries the reworked pagecache/ARC bridge, block
multiqueue, io_uring, musl 1.2.1, and modern-toolchain support).
conf_zfs switch (default bsd):
- conf_zfs=bsd - legacy in-tree BSD/Illumos ZFS (default, unchanged path)
- conf_zfs=openzfs - vendored OpenZFS 2.4.x from external/openzfs
Only one is compiled per build; both produce libsolaris.so plus the
zpool/zfs userspace. The shared solaris common objects (avl/nvpair/unicode/
fm/list) are provided by OpenZFS in openzfs mode and by the in-tree copies in
bsd mode.
Fork-free OpenZFS:
- external/openzfs pinned to the PUBLISHED upstream tag zfs-2.4.2
(github.com/openzfs/zfs @ 6330a45b), a plain submodule with no OSv fork.
- The OSv platform-layer changes live as a 19-patch git series in
modules/open_zfs/patches/ and are applied at build time by the Makefile
(idempotent via the external/openzfs/.osv-patches-applied stamp) only in
openzfs mode.
- .gitignore keeps an explicit !modules/open_zfs/patches/*.patch exception so
the series is tracked despite the global *.patch ignore.
ZFS-implementation-agnostic kernel:
- cv_timedwait keeps the BSD relative-timeout semantics; a distinct
openzfs_cv_timedwait (absolute deadline) is added in bsd/porting/netport1.cc
and exported, so the same loader.elf serves both. OpenZFS objects get
-Dcv_timedwait=openzfs_cv_timedwait via OPENZFS_CFLAGS in openzfs_sources.mk.
- core/pagecache.cc keeps master's live map_arc_buf/register_pagecache_arc_funs
bridge (used by conf_zfs=bsd) and additively provides the three symbols the
OpenZFS module needs (osv_free_pages, osv_pagecache_map_arc_page,
osv_pagecache_register_arc_rele) via a separate borrowed-ARC-page path into
read_cache. The borrow class is renamed cached_page_arc_borrow to avoid
colliding with master's arc_buf_t-based cached_page_arc.
- drivers/virtio-blk.cc: give the completion thread a 256 KB stack so the ZFS
vdev_disk_bio_done path does not overrun the default kernel stack.
Also: OpenZFS 2.4.x userspace lib/cmd build rules, openzfs_osv_compat.c, the
zfs-tools module.py (drops libzutil/libshare/libzfs_core/libtpool in bsd mode),
mkfs -m pool-root pinning gated on CONF_ZFS_OPENZFS, cpiod umount prefix
normalization so the host-side image build flushes the pool, and the ZFS
validation/benchmark test suite.
The conf_zfs=openzfs build linked the legacy BSD-ZFS kstat stub (bsd/.../opensolaris_kstat.o) to satisfy kstat_create/install/delete, but every OpenZFS module was compiled against the OSv SPL kstat_t in external/openzfs/include/os/osv/spl/sys/kstat.h (~64 bytes: ks_data, ks_ndata, ks_data_size, ks_flags, ks_update, ks_private, ks_private1, ks_lock). The BSD stub allocated only a 16-byte kstat_t, so OpenZFS callers such as arc_init()/dnode_init() wrote ks_update/ks_private past the end of the allocation, corrupting the malloc free list. The next kstat_create() then returned a garbage pointer and faulted at ks_ndata = ndata (SIGSEGV -> abort with empty backtrace right after the version banner, before mkfs ran). Fix: implement OSv-native kstat_create/install/delete in openzfs_osv_compat.c using the correct OpenZFS kstat_t layout (virtual kstats; OSv has no /proc or sysctl consumer), and filter the BSD opensolaris_kstat.o out of the openzfs solaris object list in the Makefile so only the correct implementation is linked. The zfs_builder guest now boots, runs mkfs (creates + mounts pool osv), populates it via cpiod, and exports cleanly.
…mount zfs_unmount() (called by zfs destroy, zpool export, and property remounts) only unmounts a dataset if it can find it via libzfs_mnttab_find() -> libzfs_mnttab_update() -> getmntent(). On OSv that lookup went nowhere for two reasons: - MNTTAB pointed at a nonexistent path, so fopen(MNTTAB) failed before getmntany() ever ran; and - the getmntent/getmntany stubs in libzfs_util_os.c returned EOF. Datasets auto-mounted by the kernel (zfs_domount at pool/dataset create time, not through the libzfs do_mount path) were therefore invisible to libzfs, the objset stayed owned, and zfs destroy / zpool export failed with EBUSY / 'dataset is busy'. Fix, as two edits to the OpenZFS patch series: - patch 0004: MNTTAB -> /etc/mnttab (created empty at boot, so fopen succeeds and libzfs proceeds to getmntany()). - patch 0020: add lib/libzfs/os/osv/libzfs_mnttab_os.cc implementing getmntent/getmntany over the live VFS mount table via osv::current_mounts() (filtered to ZFS), replacing the EOF stubs. Wire the new C++ shim into the libzfs build, filtering the C-only warning flags out of ozfs-cflags-common that a C++ translation unit rejects. Makes zfs destroy and zpool export/import work on OSv, which the feature-coverage tests depend on.
zpool export/import intermittently aborted with Assertion failed: owner.load(...) == sched::thread::current() (core/lfmutex.cc: unlock: 221) in condvar::wait(). Root cause: the OpenZFS userland thread pool (lib/libtpool) hands jobs to worker pthreads that block in pthread_cond_wait() on a shared tp_mutex. On OSv a pthread mutex and condvar ARE the kernel lockfree::mutex + condvar, whose wait-morphing protocol transfers mutex ownership from the signalling thread to a waiter. During tpool_destroy() teardown that handoff races: a worker returns from pthread_cond_wait() and re-enters it, unlocking a tp_mutex it no longer owns -> lfmutex owner assertion. Reproduced deterministically on zpool import, whose device-scan (zutil_import.c) and mount (libzfs_mount.c) thread pools default to hundreds of workers (mount_tp_nthr = 512). Fix: run tpool jobs synchronously on OSv (0021) and force serial dataset mounting (0022). OSv threads are cheap and these jobs are short, so serial execution removes the worker/teardown machinery and the race at no practical cost. Verified 0/8 asserts (was 8/8) across create/export/ import/status cycles from a clean patch-series rebuild.
…dataset) Patch 0017 reclaimed only the mount-root znode before dmu_objset_disown. A mounted child dataset (e.g. pool/fs) leaves its own znodes live, each holding an object bonus buffer that pins a dnode. dnode_destroy never runs, the objset os_dnodes list never empties, and spa_export() blocks forever in spa_evicting_os_wait(). Reproduced deterministically by: zpool create; zfs create pool/fs; zpool export (hang). Walk z_all_znodes at unmount and inactivate every remaining znode, as vop_inactive would, so the objset drains. Verified 6/6 completions (was 0/6, hung) for create+child-fs+export+import+destroy.
OSv read_partition_table() names MBR slots 0-based (first slot is /dev/vblkN.0) and exposes a raw unpartitioned disk directly as /dev/vblkN with no child node. zfs_append_partition() unconditionally appended .1, so \x27zpool create test /dev/vblk1\x27 on a raw wiped NVMe failed with \x27cannot open /dev/vblk1.1: No such file or directory\x27. Fix (patch 0023): only append .1 when that partition node actually exists; otherwise use the whole raw disk as-is, matching Linux/FreeBSD whole-disk behavior (ZFS writes its own labels to the whole device). Also adds FINDINGS-osv-openzfs.md documenting Bug 1, Bug 2, and the page-allocator-under-memory-pressure investigation (>=4G runtime guests). Verified: zpool create on a raw disk now succeeds, pool ONLINE.
…r 0) zpool list faulted with strlen(NULL) in print_line() because zpool_prop_column_name() returned NULL for a default list column on OSv. The pool data row is correct; only the header label resolves NULL. Guard header against NULL, falling back to the ZFS missing-value glyph, so the command cannot crash.
zfs_is_readonly() was hardcoded B_FALSE, so a readonly=on dataset stayed writable on OSv. Read the readonly property at mount into a new zfsvfs->z_readonly, return it from zfs_is_readonly(), and reject the write vnops (write/create/remove/rename/mkdir/rmdir/setattr/truncate/symlink) with EROFS. Verified writes fail EROFS after remount and readonly=off restores writability.
vdev_disk_open() never set vd->vdev_has_trim, so zpool trim/autotrim reported trim not supported. Enable it optimistically (OSv has no capability query): the ZIO_TYPE_TRIM->BIO_DISCARD path reclaims space on virtio-blk with discard, and devices without discard get ENOTSUP mapped gracefully. Verified zpool trim completes 100% with discard=unmap.
…ryption/dedup/draid/TRIM/O_DIRECT/etc)
Head-to-head storage microbenchmark of the two ZFS ports on identical a bare-metal host, 8 GiB KVM guest, single-vdev (raw NVMe) and raidz2 (7x20 GiB). No Postgres/HammerDB - isolates the filesystem layer. Key finding (the ARC-bridge measure): mmap of a 512 MiB file costs BSD-ZFS ~5 MiB extra RAM vs OpenZFS ~501 MiB - BSD's unified ARC/page-cache bridge shares pages, OpenZFS double-buffers via its borrow path. Same root cause drives BSD's warm-read and cached-random-read wins. OpenZFS wins the write/CPU paths (seq write +34%, lz4 +56%, metadata +63%, fsync +11%) and uniquely offers O_DIRECT (ARC bypass, ~raw-ceiling, BSD has no direct=). Harness (scripts/bench/): pure-C zfs-bench.c (a g++ .so pulls libstdc++ symbols OSv can't resolve at load; C avoids GLIBCXX/CXXABI/__isoc23). Runs off the zfs_builder bootfs - no usr.img cpiod populate. rebuild-bench.sh, run-bench.sh, run-all.sh drive it; results.tsv is the raw data. Copyright (C) 2026 Greg Burd
…en_zfs/ Per review: external/ is reserved for things compiled into the kernel (libfdt, acpica). The vendored OpenZFS tree is a module dependency, so move the submodule from external/openzfs to modules/open_zfs/openzfs alongside its patch series and module.py. Updates: .gitmodules path (still points at the published upstream github.com/openzfs/zfs at tag zfs-2.4.2, no fork), the OZFS/OPENZFS make variables (Makefile + bsd/sys/cddl/openzfs_sources.mk), the parse-time patch-apply step (stamp path, -d guard, and the git -C ... apply path, whose relative patch prefix changes from ../../modules/open_zfs/patches to ../patches now that the submodule sits one level deeper), and the doc comments that named the old path.
… zfs Restructure the in-kernel ZFS build into the module layout requested in PR cloudius-systems#1423 review (issue cloudius-systems#1201), mirroring how java is provided by openjdk*-from-host: - modules/zfs placeholder; module.py selects the impl by conf_zfs (openzfs -> open_zfs, else -> bsd_zfs). Still ships the shared libsolaris.so via usr.manifest. - modules/bsd_zfs provides = ["zfs"]. Owns the legacy BSD/Illumos ZFS build rules in bsd_zfs_sources.mk (the solaris compat list, the zfs core list, and the conf_zfs=bsd object/flag selection) moved out of the top-level Makefile. - modules/open_zfs provides = ["zfs"]. openzfs_sources.mk moved here as open_zfs_sources.mk and extended with the conf_zfs=openzfs object/flag selection (solaris += $(openzfs-all), kstat filter, OpenZFS CFLAGS). The top-level Makefile now includes modules/bsd_zfs/bsd_zfs_sources.mk unconditionally (defines the shared solaris/zfs lists) and, for conf_zfs=openzfs, additionally includes modules/open_zfs/open_zfs_sources.mk. The provider modules register the "zfs" capability during placeholder import, exactly like the java placeholder, so no include collision occurs. Build-structure refactor only; the same objects build for each mode. Kernel impl-agnostic bits (openzfs_cv_timedwait, the -DCONF_ZFS_OPENZFS redirect, the OpenZFS patch-apply) stay in the kernel/top-level Makefile. Validated under KVM: conf_zfs=bsd -> EXIT=0, cpiod finished, pool populated, libsolaris.so.
… pool at root In open_zfs mode the ZFS pool was not usable as the root filesystem: the guest mounted an empty pool at / and powered off with "Failed to load object: /hello" (issue reported by wkozaczuk on PR cloudius-systems#1423). bsd_zfs worked end to end. Root cause: the per-object flag that makes mkfs pick the OpenZFS pool-root mountpoint convention, $(out)/tools/mkfs/mkfs.o: CXXFLAGS += -DCONF_ZFS_OPENZFS was placed in the early conf_zfs=openzfs block, ABOVE the line where `out` is defined (out = build/$(mode).$(arch)). At that point $(out) expands to the empty string, so the target-specific variable bound to the target `/tools/mkfs/mkfs.o` instead of the real `build/release.x64/tools/mkfs/mkfs.o`, and the define never reached the compile. mkfs.cc therefore always took its #else (BSD) branch: zpool create -f -R /zfs osv # no -m / With BSD ZFS the pool-root mountpoint defaults to / so the pool mounts at the -R altroot /zfs, and cpiod --prefix /zfs/ populates the osv dataset; the loader then mounts osv and finds /hello. OpenZFS instead defaults the pool-root mountpoint to /<pool> (/osv), so under -R /zfs the pool mounted at /zfs/osv. cpiods /zfs/... writes then landed on the builders ramfs (nobody was mounted at /zfs) and were lost on shutdown, leaving the on-disk osv dataset empty apart from the osv/zfs child mountpoint stub. At boot the mount + pivot both succeeded, but every lookup in the root ZAP returned ENOENT, so /etc/fstab and /hello were invisible. Fix: define a plain make variable (conf_zfs_openzfs) in the early block and attach the -DCONF_ZFS_OPENZFS per-object flag AFTER `out` is defined, gated on that variable. mkfs.o now compiles with the define, so open_zfs uses: zpool create -f -R /zfs -m / osv # mountpoint / -> mounts at /zfs and cpiod populates the osv dataset exactly as BSD does. The pool now mounts at / and the app runs. Kernel stays impl-agnostic; the only consumer of CONF_ZFS_OPENZFS is the userspace mkfs build tool, and the change is gated on conf_zfs=openzfs so conf_zfs=bsd is byte-for-byte unchanged. Validated under KVM, both modes booting native-example end to end from the ZFS root: conf_zfs=bsd -> zfs: mounting osv from device /dev/vblk0.1; /hello runs conf_zfs=openzfs -> ZFS: root mounted ok; /dev,/proc,/sys remount; /hello runs Write+readback of a file at the ZFS root and its survival across a reboot confirm the mount is a real, persistent ZFS root.
…fs=openzfs
The six ZFS tests that drive the OpenZFS userspace C API via
dlopen("libzfs.so") -- tst-zfs-recordsize, tst-zfs-direct-io,
tst-zfs-encryption, tst-zfs-db-sim, tst-zfs-trim and misc-zfs-mmap-bench --
plus the libzfs* user-space libraries they need, only exist in
conf_zfs=openzfs. They were, however, added to the manifests for any fs=zfs
build, which broke conf_zfs=bsd two ways:
1. usr.manifest emitted /usr/lib/libzfs_core.so (and libzutil/libshare/
libtpool) gated on $(filter zfs,$(fs_type)). conf_zfs=bsd does not build
those four libraries, so the image build failed at upload time with
"ERROR: file upload failed: ... No such file or directory:
'./libzfs_core.so'".
2. The dlopen tests were in the always-on `tests` list, so a conf_zfs=bsd
test.py run collected them from usr.manifest even though libzfs.so /
libzfs_core.so are not present -- they could only ever print
"SKIP: cannot load libzfs.so" (noise), and tst-zfs-trim in particular
looked like it stalled.
Split those six tests into zfs-openzfs-only-tests and append them (and the
zfs-user-libs manifest emission) only when conf_zfs=openzfs, matching how the
top-level Makefile builds the libraries. tst-zfs-crucible-stress, misc-zfs-io
and misc-zfs-arc do not dlopen libzfs and stay in both modes.
conf_zfs reaches this Makefile through the environment (scripts/build exports
every name=value build argument); default it to bsd here to match the
top-level Makefile when the module Makefile is invoked directly.
The system-lua/luarocks fallback added to modules/lua/Makefile is a build convenience unrelated to the selectable BSD/OpenZFS ZFS work in this PR. Restore the file to its upstream/master state so it is not carried by cloudius-systems#1423; it can be proposed on its own.
…is busy") Building a ZFS image ends with the zfs_builder running /tools/mkfs.so; /tools/cpiod.so --prefix /zfs/; /zfs.so set compression=off osv; /zpool.so export osv and printing "cannot export 'osv': pool is busy" at the very end. mkfs auto-mounts the pool root (osv at /zfs) and the osv/zfs child (at /zfs/zfs), and cpiod writes into them. zpool export's own zpool_disable_datasets() unmounts datasets it finds in /etc/mnttab, but OSv's mount shim never populates mnttab, so it finds nothing to unmount; the kernel then refuses the export with EBUSY because the still-mounted datasets hold the objsets (spa refcount != 0). The message is cosmetic -- the image is fully written -- but it looks like a failure. Drain the pool explicitly before exporting: unmount the datasets by mountpoint (children first) with the OSv-native /tools/umount.so, which calls umount(2) -> VFS_UNMOUNT -> the real zfs_umount -> dmu_objset_disown, dropping the spa refcount to zero so the export succeeds cleanly. This is mode-independent and avoids the bsd "zfs unmount" command, which needs statfs2mnttab (absent from the builder image). umount.so is already built into tools; add it to the zfs_builder bootfs so it is available there.
… ZIL replay (sync=standard durability)
Bump modules/open_zfs/openzfs from zfs-2.4.2 (6330a45b) to zfs-2.4.3 (83020cf8, tag zfs-2.4.3). 2.4.2 -> 2.4.3 is an upstream PATCH release; all 29 OSv platform-layer patches (0001..0029) apply cleanly against the 2.4.3 source with no reroll required. Verified: the build-time patch-apply stamps and libsolaris.so + loader.elf link cleanly under conf_zfs=openzfs, and conf_zfs=bsd is unaffected. Also update the pinned branch in .gitmodules, the Makefile pin comment, and the PERF doc version reference from 2.4.2 to 2.4.3.
OSv's taskq_wait() enqueued a single barrier task and drained just that barrier. On the ZFS system_taskq (8 worker threads) a free worker runs the barrier while the other workers are still mid dnode_sync/dbuf_sync, so taskq_wait() returned early and the syncing thread raced the still-running workers, corrupting dbuf/dirty records (arc_write_done VERIFY3 / lfmutex owner asserts under bulk write). Add taskqueue_drain_all() (wait until the queue is empty AND no worker is active, with a wakeup when it quiesces) and use it in taskq_wait(), matching illumos semantics (wait until no task is queued or active). This also handles ZFS's recursive within-task enqueues: the enqueuing task stays active so tq_active never empties until the whole recursion completes.
…on OSv Parallel ARC eviction dispatches arc_evict_task to an AS0 taskq; a forked PostgreSQL backend that triggers eviction races it (fault on eva->eva_ml, arc.c:4088). Force zfs_arc_evict_threads=1 (inline eviction), matching the existing OSv serialization patches (0021/0022).
…09 so the series applies clean) Patch 0030 (force single-threaded inline ARC eviction on OSv) carried three hunks against module/zfs/arc.c, but two of them were byte-identical duplicates of content already added by patch 0009 (zfs: fix ARC sizing, memory pressure and OSv platform ...): hunk cloudius-systems#1 @@ -4096 arc_evict_thread_init(): OSv force zfs_arc_evict_threads=1 -- 0030's legitimate, unique change. KEPT. hunk cloudius-systems#2 @@ -4756 adds uint64_t arc_reduce_target_size_noshrink(...) -- identical to the function 0009 already adds. REMOVED. hunk cloudius-systems#3 @@ -7980 small-RAM arc_c_min cap (allmem < 256 MiB -> allmem/16) -- identical to the cap 0009 already adds. REMOVED. Because 0009 applies before 0030, the noshrink helper and the arc_c_min cap are already present in module/zfs/arc.c by the time 0030 runs. 0030 then tried to add them a second time, so "git apply" of the full series rejected on a clean tree and the OpenZFS-mode build failed. This was masked on already-built trees because modules/open_zfs applies the series once and records a .osv-patches-applied stamp, so a re-checkout never re-ran the apply; the aarch64 clean-build exposed it. Fix: drop hunks cloudius-systems#2 and cloudius-systems#3 from 0030, leaving only the arc_evict_thread_init change, and correct 0030's Subject / commit message / diffstat to describe only that change (it was over-scoped / mislabeled). The noshrink helper and the arc_c_min cap remain, exactly once, from patch 0009. Verified: the full series 0001..0030 now applies with zero rejects from a pristine (pinned) OpenZFS submodule; libsolaris.so builds in both conf_zfs=openzfs and conf_zfs=bsd modes; the OpenZFS kernel boots; and arc_reduce_target_size_noshrink + the arc_c_min cap are present exactly once in the applied tree. [cloudius-systems#1423/OpenZFS] Signed-off-by: Greg Burd <greg@burd.me>
…de (ZIL-replay mount crash) A znode loaded via zfs_zget() outside the OSv VFS vget() path has no attached vnode (z_vnode == NULL), so ZTOV(zp) == NULL. ZIL replay at import/mount hits this: zfs_replay_truncate -> zfs_space -> zfs_freesp -> zfs_trunc/ zfs_free_range/zfs_extend each call vnode_pager_setsize(ZTOV(zp), ...), which on OSv is vp->v_size = size, so a NULL vnode page-faults during mount and aborts the guest before PostgreSQL starts. Intermittent (only when the ZIL has an un-synced TX_TRUNCATE to replay); became reachable via patch 0029. Guard each vnode_pager_setsize() with a NULL-vnode check. A replay znode has no page cache to size; z_size + SA are already authoritative. Combined fork+ZFS image now reaches ready on 25/25 consecutive boots (-smp 8 -m 32G KVM rs=8k lz4) vs aborting at mount ~2/3 of boots before. Classification: [cloudius-systems#1423/OpenZFS]. Signed-off-by: Greg Burd <greg@burd.me>
…D fallback
The OSv SPL platform layer (patch 0001) only recognized x86_64: isa_defs.h
#errored out on any other ISA, and simd.h's non-x86 feature-detection stub
block was missing zfs_sha512_available(). On x86_64 OpenZFS resolves SHA-512
availability through the sha512-x86_64.S asm path, so the stub was never
exercised; aarch64 has no such asm hook wired into the SPL, so icp/algs/sha2
calls zfs_sha512_available() and fails without the fallback.
Add patch 0031 to teach the SPL layer about aarch64:
- isa_defs.h: add an __aarch64__ ISA branch (LP64, _SUNOS_VTOC_16) mirroring
the x86_64 branch; endianness is still derived from __BYTE_ORDER.
- simd.h: add zfs_sha512_available() to the non-x86 B_FALSE stub block so the
SHA-512 impl selector falls back to the generic C path on aarch64.
x86_64 is unaffected: both hunks add code guarded by __aarch64__ or the
existing non-x86 #else, leaving x86_64 preprocessor output byte-identical.
The submodule stays pinned to the pristine upstream zfs-2.4.3 tag; this is a
new entry in the modules/open_zfs/patches series, not a submodule re-pin.
BUILD-UNVERIFIED on aarch64 - needs an aarch64 metal build+boot to
validate before un-drafting.
Signed-off-by: Greg Burd <greg@burd.me>
open_zfs_sources.mk unconditionally listed the x86_64 ICP crypto asm objects
(aes_amd64.o, aes_aesni.o, sha256/sha512-x86_64.o) plus aeskey.o. The
aes_amd64.S / aes_aesni.S / aeskey.c sources carry no top-level __x86_64__
guard and emit raw x86 instructions, so on an aarch64 build the assembler
chokes. (The raidz vdev_raidz_math_* objects already coexist across arches
because each is #ifdef-guarded to compile-to-empty on the wrong ISA; the ICP
asm objects are not.)
Gate the object selection on $(arch) (open_zfs_sources.mk is already included
after arch is set, so no Makefile change is needed):
- x64: aeskey.o + the four x86_64 ICP asm objects, exactly as before.
- aarch64: the ARMv8 sha256/sha512 asm (both #if defined(__aarch64__)
guarded) and the native ARMv8 blake3 asm, all present in the upstream
zfs-2.4.3 submodule; no x86 objects, no aeskey.o.
x86_64 is unaffected: with arch=x64 the emitted openzfs-icp / openzfs-icp-asm
lists are byte-identical to before (verified by expanding both branches).
BUILD-UNVERIFIED on aarch64 - needs an aarch64 metal build+boot to
validate before un-drafting.
Signed-off-by: Greg Burd <greg@burd.me>
libc/arch/aarch64/atomic.h pulls in OSv kernel-only headers (the FreeBSD-derived machine/atomic.h and the bsd/cddl opensolaris sys/types.h), which are only on the include path for kernel/bsd objects. Userspace translation units that resolve <atomic.h> to this file (the OpenZFS libspl/libzfs sources on aarch64) do not have them and do not need them. Gate those includes on the kernel build and use the compiler atomic builtin in the userspace path. The x86_64 variant has no such includes, so this is aarch64-only and leaves x86_64 byte-identical.
The generic-timer frequency sanity check rejected anything above 1GHz. Real server ARM cores run higher: some server ARM cores run at 1GHz and up; one Neoverse-V1 part (Neoverse-V1) passes a 1.05GHz generic-timer frequency to the guest, which tripped the old ceiling and aborted the boot. Raise the ceiling to 2GHz to cover current cores with headroom while still rejecting a garbage read.
Drop incidental cloud-vendor and instance-type names from committed docs and a code comment; the technical content is unchanged (a generic-timer frequency ceiling raised to 2GHz to cover current server ARM cores; debugging/perf notes that describe the host generically).
gburd
force-pushed
the
pr/openzfs-aarch64
branch
from
July 31, 2026 13:12
37ea518 to
53dec06
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enables the vendored OpenZFS 2.4.3 port (added in #1423) to build on aarch64, with x86_64 left byte-for-byte unchanged.
Depends on: #1423
This is the aarch64 follow-on to #1423 ("zfs: selectable in-kernel ZFS (BSD or OpenZFS 2.4.2) via upstream submodule + patch series"). It is cut directly on top of that PR's tip and must land after it — the two commits here only make sense against the
modules/open_zfs/submodule + patch-series layout introduced by #1423.What this does
Two small, self-contained changes — the aarch64-enablement nugget re-authored against the current
modules/open_zfs/layout:modules/open_zfs/patches/0031-*.patch— SPL sha512 stub / SIMD fallback.The OSv SPL layer (patch 0001) only recognized x86_64:
isa_defs.h#errored on any other ISA, andsimd.h's non-x86 feature-detection stub block was missingzfs_sha512_available(). On x86_64 SHA-512 availability comes from thesha512-x86_64.Sasm path so the stub was never needed; aarch64 has no such asm hook wired into the SPL, soicp/algs/sha2callszfs_sha512_available()and fails without the fallback. Patch 0031 adds an__aarch64__ISA branch toisa_defs.hand the missingzfs_sha512_available()B_FALSEstub tosimd.h. The submodule stays pinned to the pristine upstreamzfs-2.4.3tag — this is a new entry in the patch series, not a submodule re-pin.modules/open_zfs/open_zfs_sources.mk— arch-conditional ICP asm object lists.The x86_64 ICP crypto asm objects (
aes_amd64.S,aes_aesni.S,aeskey.c) have no top-level__x86_64__guard and emit raw x86 instructions, so an aarch64 build's assembler chokes on them. (The raidzvdev_raidz_math_*objects already coexist across arches via per-file#ifdefguards; the ICP asm objects do not.) The object selection is now gated on$(arch): x64 getsaeskey.o+ the four x86_64 asm objects exactly as before; aarch64 gets the#if defined(__aarch64__)-guarded ARMv8 sha256/sha512 asm and the native ARMv8 blake3 asm (all present in the upstreamzfs-2.4.3submodule). No Makefile change is needed —open_zfs_sources.mkis already included afterarchis set.x86_64 is unaffected (arch-gated)
With
arch=x64the emittedopenzfs-icp/openzfs-icp-asmobject lists are byte-identical to #1423 (verified by expanding both branches of the makefile). Patch 0031's hunks are all guarded by__aarch64__or the existing non-x86#else, so x86_64 preprocessor output is unchanged.Not included
libsolaris.so/ userspace-CLI compile-flag changes. The old aarch64 branch bundled an-isystem→-Ilibspl include-ordering tweak with unrelated io_uring syscall aliases; that swap is not arch-gated, only manifests during an aarch64 userspace build, and risks perturbing the x86 userspace build if applied blind. It's deferred until it can be validated on real aarch64 hardware.This was re-authored from the older aarch64 work by inspection against the current layout; it has not been compiled or booted on aarch64 (no aarch64 toolchain/hardware available at authoring time). The x86_64 no-op property is verified by makefile expansion, but the aarch64 build+boot path must be exercised on aarch64 hardware before this leaves draft.