build: macOS 26 SDK arm64-macos shim for Zig 0.15.x#2499
Conversation
macOS 26's CommandLineTools SDK ships .tbd library stubs that export arm64e-macos only — arm64-macos was dropped. Zig 0.15.x bundles a libSystem.tbd that still has arm64-macos but is pinned to macOS 15.5; its auto-detection picks the higher-numbered system SDK and the arm64 link fails with ~30 undefined libSystem / CoreFoundation / SystemConfiguration symbols. scripts/darwin-sdk-shim.sh assembles a hybrid SDK at .lp-cache/darwin-sdk-shim/sdk/: usr/include is symlinked from the system SDK, libSystem.tbd is swapped for Zig's bundled copy, every other .tbd is rewritten to also export arm64-macos. The script also emits an xcrun wrapper at .lp-cache/darwin-sdk-shim/bin/xcrun that returns the shim path for --show-sdk-path queries and passes everything else through to /usr/bin/xcrun. The Makefile detects when the system libSystem.tbd lacks arm64-macos (via awk on the first targets: block) and makes test / build-dev / build-v8-snapshot depend on a marker target that runs the shim script once. PATH and LIGHTPANDA_DARWIN_SDK_SHIM are exported so nested zig invocations resolve the shim SDK. Hosts that don't need the shim — older macOS, Linux, any future Apple SDK that puts arm64-macos back — skip the entire block; the Makefile is identical to before for them. ZIGFLAGS="$V8" make darwin-sdk-shim rebuilds the shim explicitly; ZIGFLAGS="$V8" make darwin-sdk-shim-clean removes it; ZIGFLAGS="$V8" make clean also wipes it. CONTRIBUTING.md documents the cache directory and the manual env export needed for bare zig build invocations $V8.
|
My preference is to wait until we upgrade to Zig 0.16 and ask people to install xcode 26.3 in the meantime. (I realize that's a big ask, but I feel less guilty about asking that than about running a bash script that modifies their system in a way I don't really understand and that doesn't seem trivial). |
|
The main hesitation seems to be the script modifying contributors' systems — and it doesn't. Everything lands under
Given that nothing outside the repo is touched, would you reconsider or would you still rather wait for If you'd rather wait, do you have a rough ETA for the 0.16 bump? |
Address review concern about the script modifying the contributor's system. The shim only reads the system SDK (via symlinks) and writes patched .tbd copies into .lp-cache/ (gitignored); nothing global is touched and it is fully reverted by ZIGFLAGS= make clean. State this explicitly in CONTRIBUTING.md (a [!NOTE] callout) and at the top of the script header instead of leaving it implicit.
|
I also baked that reassurance into the diff rather than leaving it in a comment (4c56ece): a |
State explicitly that the shim becomes unnecessary once the project upgrades to Zig 0.16+ (whose bundled libc ships the macOS 26 SDK and links arm64-macos natively), and that it should be removed deliberately at that point — it does not self-disable, since detection keys off the system SDK, which still lacks arm64-macos on macOS 26.
|
On the 0.16 question specifically (f743ea3): I've documented the shim as an explicit, temporary stopgap. Both |
What this fixes
make test/make build-dev/zig buildfail on macOS 26+ (Apple silicon) with ~30 undefinedlibSystem/CoreFoundation/SystemConfigurationsymbols at the build-runner link stage, before any project code is touched.Root cause: macOS 26's CommandLineTools SDK dropped
arm64-macosfrom every system.tbdlibrary stub — onlyarm64e-macosremains. Zig 0.15.x bundleslib/libc/darwin/libSystem.tbdthat does exportarm64-macos, but itsSDKSettings.jsonis pinned to macOS 15.5. Zig's host detection sees the higher-numbered system SDK at/Library/Developer/CommandLineTools/SDKs/MacOSX.sdkand picks it; the arm64 link then can't resolve symbols Apple still ships exclusively underarm64e-macos.Concretely (without this PR, on a stock macOS 26.5 + Zig 0.15.2 host):
The failure is in the build runner itself (
build_zcu.o), so it precedes anythingbuild.zigcould detect and can't be fixed from insidebuild.zig.What this PR changes
scripts/darwin-sdk-shim.sh(new, +198) — assembles a hybrid macOS SDK shim under.lp-cache/darwin-sdk-shim/:sdk/usr/include,sdk/usr/share,sdk/Library,sdk/System/iOSSupportsymlinked from the system SDK (headers don't need patching, the system mirror keepsclangetc. working)sdk/usr/lib/libSystem.tbdcopied from Zig's bundledlibc/darwin/libSystem.tbd(the one witharm64-macosexports).tbdunderusr/lib/andSystem/Library/(Private)Frameworks/rewritten to also exportarm64-macos. The patch is a singlesed: append, arm64-macosto every occurrence ofarm64e-macos. The .tbd format groups symbols by target list, so the linker resolves arm64-macos lookups using the same symbol definitions Apple ships for arm64e-macos.bin/xcrun— wrapper script that returns the shim path forxcrun --show-sdk-path/--show-sdk-versionwhen--sdkismacosx(or unspecified). Everything else isexec /usr/bin/xcrun "$@"so non-Zig tooling (xcrun clang,xcrun --find swiftc, …) keeps working untouched. The shim path is read fromLIGHTPANDA_DARWIN_SDK_SHIMat runtime, not baked into the script, so moving.lp-cache/doesn't break the wrapper.Makefile— detects when the systemlibSystem.tbd's firsttargets:block lacksarm64-macos. When the gap is present,test,build-dev, andbuild-v8-snapshotdepend on a marker file that runs the shim script once;PATHandLIGHTPANDA_DARWIN_SDK_SHIMare exported so theziginvocation inside each recipe resolves the shim SDK. Newmake darwin-sdk-shimandmake darwin-sdk-shim-cleantargets expose the cache explicitly.make cleanalso wipes the shim.CONTRIBUTING.md— new "macOS 26+ (Apple silicon)" section explaining what the shim does, how to rebuild after Xcode CommandLineTools / Zig version bumps, and how to export the env vars for barezig buildinvocations outsidemake.Hosts that don't need the shim — macOS 15 and earlier, Linux, any future Apple SDK that puts
arm64-macosback — skip the entireifeq ($(NEEDS_DARWIN_SDK_SHIM), yes)block. The Makefile and contributor workflow are identical to before for them.Notable design choices
Why a Makefile-level workaround rather than something in
build.zig. The link failure is inbuild_zcu.o— the compiledbuild.zigitself, before any project logic runs. There's no Zig-level way to intercept that compilation. The same goes forzig build --sysroot …and--libc <paths-file>: both flags are parsed by the build runner after it links, so they only affect childziginvocations, not the build runner's own linking. The fix has to run beforezig build, which means eithermake, a wrapper script, or a contributor-managed environment.Why patch every
.tbdinstead of onlylibSystem.tbd. The first link failure islibSystem, but as soon as that's resolved the project pulls inCoreFoundation,SystemConfiguration,Foundation, and a fewusr/lib/system/*.tbdsiblings. Their.tbdfiles have the same arm64-macos gap. Patching them all once at shim-build time is ~5 s on an M3, vs. discovering each new framework gap one CI failure at a time.Why
[^e]arm64-macosin the detection awk.arm64-macosis a substring ofarm64e-macos, so a naivegrep "arm64-macos"always matches. The Makefile reads only the FIRSTtargets:block inlibSystem.tbd— that's the mainlibSystem.B.dylibentry the linker resolves-lSystemagainst. Sub-library entries lower in the file may still listarm64-macoseven when the main one doesn't, so an unfiltered scan reports "no shim needed" when in fact the link will fail.Why
LIGHTPANDA_DARWIN_SDK_SHIMinstead of baking the path into the wrapper. Keeps the generated wrapper portable across repo locations and developers. The wrapper is regenerated whenever the script changes (Make tracks it as a dep on the marker), so any wrapper protocol change ships atomically.Why this isn't a Zig version bump. Zig 0.16.0 ships SDK 26.4 in its bundled libc and would fix this natively. But it breaks
build.zig(std.fs.File.stderr()was renamed,process.SpawnOptions.StdIo.Ignoreremoved) plus probably the rest of the project. Bumping the toolchain is its own conversation; this PR keeps the project on Zig 0.15.2 and unblocks the macOS 26 host.Test plan
make darwin-sdk-shimbuilds the shim in ~5 s, printsdarwin-sdk-shim: 5534 .tbd files patched at …/sdk.make darwin-sdk-shim-cleanremoves it; rerun ofmake darwin-sdk-shimrebuilds from scratch.make build-devproduces a workingarm64Mach-Ozig-out/bin/lightpanda. Verifiedlightpanda versionprints the expected build hash.zig build $V8 testwithPATHandLIGHTPANDA_DARWIN_SDK_SHIMexported runs the full suite — 693/693 tests pass.make cleanwipes the shim cache.NEEDS_DARWIN_SDK_SHIMevaluates to empty whenlibSystem.tbd's firsttargets:block already listsarm64-macos), the entire shim block is skipped;test/build-devrecipes are unchanged.shellcheck scripts/darwin-sdk-shim.shclean.Caveats
.lp-cache/(already in.gitignore) rather than committed.zig buildoutsidemakedoesn't pick upPATH/ env automatically. Documented in CONTRIBUTING.md with the manual exports.script -q /dev/null …wrapper aroundzig build testinterferes with cargo's jobserver for thehtml5evercrate — independent of this PR, present before and after. Workaround:zig build $V8 testdirectly (matches AGENTS.md). I haven't touched that wrapper here.make darwin-sdk-shim-clean && make …). Xcode CommandLineTools updates and Zig version bumps both invalidate the shim contents but won't auto-detect — CONTRIBUTING.md spells this out.