Skip to content

Commit 9f9f34e

Browse files
committed
refactor(ffi): split pb_* C exports into ffi.zig so the importable module emits no C symbols
The printable_binary Zig module and the static FFI lib shared one root source file, so any Zig consumer importing the module pulled in all 12 'export fn pb_*' C symbols. Two static (musl) consumers of pb then collide on 'duplicate symbol: pb_*' under ld.lld -- which blocked difz's static build. Move the C ABI into src/zig/ffi.zig (the static lib's new root, importing the pure printable_binary.zig). The importable module now defines zero pb_* symbols; libprintable_binary.a keeps all 12. FFI unit tests move alongside the exports. Adds a falsifiable regression test (test/test_module_no_ffi_symbols, nm as an independent oracle) wired as the test-no-ffi-symbols Garnix check.
1 parent dd0f801 commit 9f9f34e

7 files changed

Lines changed: 396 additions & 302 deletions

File tree

.dirtree-state

Lines changed: 3 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -1,94 +1,7 @@
11
# Dirtree: Stateful directory trees in the CLI for humans and LLMs. https://github.com/pmarreck/dirtree
22
ver=1.2
3-
depth=3
4-
color=false
5-
63
annotate=[
7-
.envrc = direnv: auto-enter the Nix dev shell
8-
.github = GitHub Actions CI configuration
9-
.github/workflows/ci.yml = GitHub Actions: nix build/test (linux/darwin), cross-compile, windows
10-
.gitignore = Git ignore rules
11-
.nojekyll = Disable Jekyll on GitHub Pages so symlinks/underscored paths are served
12-
AGENTS.md = TDD and workflow discipline for AI agents working in this repo
13-
CHARACTER_WIDTHS.md = Notes on terminal character-width (East Asian Width) handling
14-
CLAUDE.md = Repository-specific instructions for Claude Code
15-
COMPARISON_WITH_BAT.md = Feature/behavior comparison against the bat pager
16-
C_VS_LUAJIT_PERFORMANCE.md = Performance notes comparing the C and LuaJIT implementations
17-
FUTURE_WORK.md = Roadmap and ideas not yet scheduled
18-
LICENSE = Project license
19-
Makefile = Build targets: native C, WASM (emcc), APE (cosmocc), PGO
20-
NEXT_STEPS.md = Short-term planned next steps
21-
README.md = Project overview, usage, and cross-implementation performance table
22-
README_IMPLEMENTATIONS.md = Per-implementation details and how they relate
23-
WEB_README.md = Docs for the web/WASM browser demo
24-
bin = Runnable CLI entry points (Lua/Node) + prebuilt APE binary and the runtime map
25-
bin/character_map.txt = Symlink -> ../character_map.txt (runtime map for the Lua CLI)
26-
bin/printable-binary = LuaJIT CLI (the original reference implementation)
27-
bin/printable-binary-node.js = Node.js CLI (ESM)
28-
bin/printable_binary_ape.com = Prebuilt Actually-Portable-Executable (Cosmopolitan) C binary
29-
bin/utf16to8 = Helper: transcode UTF-16 stdin -> UTF-8 (iconv wrapper; auto-detects BOM, else LE)
30-
bin/utf8to16 = Helper: transcode UTF-8 stdin -> UTF-16 (iconv wrapper; default LE + BOM)
31-
bm = Ad-hoc benchmark scripts (C vs Lua vs Zig)
32-
bm/benchmark-zig-opt = Zig optimization benchmark
33-
bm/benchmark_c_vs_lua.sh = C vs Lua benchmark script
34-
bm/benchmark_comparison.lua = Cross-implementation comparison benchmark (Lua)
35-
build = Convenience build script
36-
build.zig = Zig build: CLI, C-ABI static lib (PIC), unit tests
37-
build.zig.zon = Zig package manifest
38-
character_map.txt = SOURCE OF TRUTH byte->glyph map (256 lines, ## comments) all builds derive from
39-
character_map_embedded.h = Generated C embedding of character_map.txt (via utils/generate_embedded_map.lua)
40-
docs = GitHub Pages web demo (symlinks to root) + design plans
41-
docs/character_map.txt = Symlink -> ../character_map.txt
42-
docs/index.html = Symlink -> ../index.html
43-
docs/plans/2026-02-03-double-encode-detection-design.md = Design doc: double-encode detection
44-
docs/printable_binary.js = Symlink -> ../js/printable_binary.js
45-
flake.lock = Pinned flake input revisions
46-
flake.nix = Nix packages (native/zig/wasm/ape) + Garnix checks (test-c/zig/js/ffi/leak/...)
47-
index.html = Root web demo page (served by GitHub Pages from /)
48-
js = Shared JavaScript module used by the Node CLI and the browser demo
49-
js/printable_binary.js = Shared JS module (encode/decode/parseMap) for the Node CLI and browser
50-
lib = Vendored Lua helper libraries
51-
lib/xxhash32.lua = xxHash32 implementation in Lua
52-
package.json = Node ESM manifest (type:module) for the JS implementation
53-
printable_binary.js = Symlink -> js/printable_binary.js so the root web page can import it
54-
src = Core implementations: standalone C, the C FFI header/CLI, and the Zig core+CLI
55-
src/printable_binary.c = Standalone, dependency-free C implementation + CLI (portable build)
56-
src/printable_binary.h = Public C FFI header -- the real cross-language API (pb_encode/decode/hexlike/...)
57-
src/printable_binary_ffi_main.c = C CLI that dogfoods the FFI by calling the Zig lib through the C ABI
58-
src/zig = Zig core library (logic + pb_* C ABI exports) and the Zig CLI
59-
src/zig/main.zig = Zig CLI (argument parsing + I/O around the Zig core)
60-
src/zig/printable_binary.zig = Zig core: encode/decode/hexlike logic + comptime map + pb_* C ABI exports
61-
test = Test suites: functional, fuzz, benchmark, leak, cross-implementation, unit
62-
test/benchmark_test = Encode/decode throughput benchmark (Zig default, announces impl)
63-
test/fuzz_test = Seeded random-data roundtrip fuzzer (reproducible via --seed)
64-
test/js = JavaScript unit tests (Deno/Node)
65-
test/js/test_printable_binary.js = JS unit tests (Deno/Node)
66-
test/leak_harness.c = FFI-driven leak harness: loops pb_* alloc/free, self-reports RSS
67-
test/leak_test = Memory-leak suite: runs a target long-lived and fails on RSS growth
68-
test/printable-binary-ape-wrapper = Wrapper to invoke the APE binary in tests
69-
test/test = Main cross-implementation functional suite (IMPLEMENTATION_TO_TEST-selectable)
70-
test/test-all = Master runner that exercises every implementation
71-
test/test_ape_arch_slice = APE multi-arch slice verification
72-
test/test_cross_implementation.sh = Cross-implementation output comparison
73-
test/test_node_cli_stats = Node CLI stats/stderr behavior test
74-
test/test_validate.c = C unit tests for pb_validate
75-
test/test_wasm = WASM build validation (via WASI runtime)
76-
utils = Tooling: map generation/auditing, fuzz PRNG, EAW data, analysis scripts
77-
utils/audit_character_map.lua = Audits the character map for conflicts/issues
78-
utils/data = Reference data (Unicode East Asian Width table)
79-
utils/data/EastAsianWidth.txt = Unicode East Asian Width reference data
80-
utils/diagnose.sh = Diagnostic helper
81-
utils/dump-chars.sh = Dump the character map for inspection
82-
utils/generate_embedded_map.lua = Generates character_map_embedded.h from character_map.txt (## aware)
83-
utils/js = One-off JS scripts used while designing/auditing the character map
84-
utils/js/analyze_encoding_sizes.js = One-off: analyze encoded-size distribution
85-
utils/js/compare_all_maps.js = One-off: compare candidate maps
86-
utils/js/compare_maps.js = One-off: compare two maps
87-
utils/js/debug_maps.js = One-off: debug map building
88-
utils/js/debug_test.js = One-off: debugging scratch
89-
utils/js/find_available.js = One-off: find available glyphs
90-
utils/js/find_conflicts.js = One-off: find glyph conflicts
91-
utils/js/find_replacements.js = One-off: find replacement glyphs
92-
utils/prng = LuaJIT PRNG utility for deterministic fuzz data
93-
utils/update_eaw_data.sh = Refresh the Unicode East Asian Width data file
4+
src/zig/ffi.zig = C ABI (FFI) export surface: all 12 pb_* C functions; root of libprintable_binary.a; keeps C symbols OUT of the importable printable_binary module so static (musl) consumers don't collide
5+
test/module_consumer.zig = Test fixture: minimal downstream importer of the printable_binary Zig module (mirrors how difz/blip consume it) for the FFI-symbol-leak test
6+
test/test_module_no_ffi_symbols = Regression test (nm oracle): importing the module must emit 0 pb_* symbols while the FFI static lib keeps all 12 — guards the difz-blocking duplicate-symbol bug
947
]

build.zig

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,13 @@ pub fn build(b: *std.Build) void {
1717
// =========================================================================
1818
// Static Library with C ABI (for FFI consumers like Cosmopolitan)
1919
// =========================================================================
20+
// Root is ffi.zig — the FFI-only file that holds every `export fn pb_*`.
21+
// It imports printable_binary.zig (relative), so the C ABI is emitted ONLY
22+
// here, never in the importable `printable_binary` module above. This is
23+
// what keeps static (musl) downstreams from colliding on duplicate pb_*
24+
// symbols. See src/zig/ffi.zig and test/test_module_no_ffi_symbols.
2025
const static_lib_mod = b.createModule(.{
21-
.root_source_file = b.path("src/zig/printable_binary.zig"),
26+
.root_source_file = b.path("src/zig/ffi.zig"),
2227
.target = target,
2328
.optimize = optimize,
2429
// Position-independent so the static lib links into PIE executables
@@ -92,9 +97,45 @@ pub fn build(b: *std.Build) void {
9297
const test_step = b.step("test", "Run unit tests");
9398
test_step.dependOn(&run_lib_tests.step);
9499

100+
// FFI-surface tests (the C ABI wrappers now live in ffi.zig)
101+
const ffi_test_mod = b.createModule(.{
102+
.root_source_file = b.path("src/zig/ffi.zig"),
103+
.target = target,
104+
.optimize = optimize,
105+
});
106+
ffi_test_mod.addAnonymousImport("character_map.txt", .{ .root_source_file = b.path("character_map.txt") });
107+
const ffi_tests = b.addTest(.{
108+
.root_module = ffi_test_mod,
109+
});
110+
test_step.dependOn(&b.addRunArtifact(ffi_tests).step);
111+
95112
// =========================================================================
96113
// Library-only step (for Makefile integration)
97114
// =========================================================================
98115
const lib_step = b.step("lib", "Build only the static library");
99116
lib_step.dependOn(&static_lib.step);
117+
118+
// =========================================================================
119+
// Module-consumer static lib (FFI-symbol-leak regression test)
120+
// =========================================================================
121+
// Builds a downstream consumer that imports the `printable_binary` module
122+
// the way sibling Zig projects do. test/test_module_no_ffi_symbols asserts
123+
// the resulting archive carries NO `pb_*` C symbols — they must originate
124+
// only in the FFI-root static library (libprintable_binary.a). See
125+
// test/module_consumer.zig.
126+
const consumer_mod = b.createModule(.{
127+
.root_source_file = b.path("test/module_consumer.zig"),
128+
.target = target,
129+
.optimize = optimize,
130+
.imports = &.{
131+
.{ .name = "printable_binary", .module = lib_mod },
132+
},
133+
});
134+
const consumer_lib = b.addLibrary(.{
135+
.name = "module_consumer",
136+
.linkage = .static,
137+
.root_module = consumer_mod,
138+
});
139+
const consumer_lib_step = b.step("consumer-lib", "Build the module-consumer static lib (FFI-symbol-leak test fixture)");
140+
consumer_lib_step.dependOn(&b.addInstallArtifact(consumer_lib, .{}).step);
100141
}

flake.nix

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,22 @@
347347
installPhase = "mkdir -p $out && touch $out/passed";
348348
};
349349

350+
# Architecture invariant: importing the `printable_binary` Zig module
351+
# must emit ZERO `pb_*` C symbols (the C ABI lives only in the FFI-root
352+
# static lib). Two static (musl) consumers of pb otherwise collide on
353+
# `duplicate symbol: pb_*` under ld.lld — this blocked difz. nm is an
354+
# independent oracle. See test/test_module_no_ffi_symbols.
355+
test-no-ffi-symbols = pkgs.stdenv.mkDerivation {
356+
name = "test-no-ffi-symbols";
357+
src = ./.;
358+
nativeBuildInputs = with pkgs; [ zig binutils ];
359+
buildPhase = ''
360+
export HOME=$TMPDIR
361+
bash ./test/test_module_no_ffi_symbols
362+
'';
363+
installPhase = "mkdir -p $out && touch $out/passed";
364+
};
365+
350366
# Guard against a stale generated C header: regenerating from
351367
# character_map.txt must reproduce the committed character_map_embedded.h.
352368
test-embedded-map-sync = pkgs.stdenv.mkDerivation {

0 commit comments

Comments
 (0)