Skip to content

Commit cd6d32d

Browse files
committed
feat(core): CRC-32/ISO-HDLC primitive + pb_crc32 FFI export (issue #1)
Checksum primitive backing the printable-binary-file.json container (issue #1): pure `crc32` (ISO-HDLC, vector-pinned to CRC32("123456789")=0xCBF43926) in printable_binary.zig, and the `pb_crc32` C-ABI export in ffi.zig (delegates to pb.crc32; null-safe). Adds the container design spec + PLAN.md, and gitignores inbox/ (transient LLMsend coordination).
1 parent 3a6682d commit cd6d32d

6 files changed

Lines changed: 240 additions & 0 deletions

File tree

.dirtree-state

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Dirtree: Stateful directory trees in the CLI for humans and LLMs. https://github.com/pmarreck/dirtree
22
ver=1.2
33
annotate=[
4+
PLAN.md = Issue #1 plan: decode workflow + printable-binary-file.json container
5+
docs/plans/2026-06-26-printable-binary-file-container-design.md = Design spec: printable-binary-file.json container + web decode workflow (issue #1)
46
src/zig/ffi.zig = C ABI (FFI) export surface: all 12 pb_* C functions; root of libprintable_binary.a; keeps C symbols OUT of the importable printable_binary module so static (musl) consumers don't collide
57
test/module_consumer.zig = Test fixture: minimal downstream importer of the printable_binary Zig module (mirrors how difz/blip consume it) for the FFI-symbol-leak test
68
test/test_module_no_ffi_symbols = Regression test (nm oracle): importing the module must emit 0 pb_* symbols while the FFI static lib keeps all 12 — guards the difz-blocking duplicate-symbol bug

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,3 +42,6 @@ CLAUDE*.md
4242
# Local convenience symlink to ~/dotfiles (external target — must not be committed
4343
# into a published repo; broke the GitHub Pages build as a dangling symlink).
4444
jj_cheatsheet.md
45+
46+
# Inter-LLM coordination notes (LLMsend) — transient/local, never tracked
47+
inbox/

PLAN.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
purpose: Track work on GitHub issue #1 — web-demo decode workflow + printable-binary-file.json metadata container
3+
audience: both
4+
maintained_by: agent
5+
---
6+
7+
# PLAN — issue #1: decode workflow + printable-binary-file.json
8+
9+
GitHub: https://github.com/pmarreck/printable_binary/issues/1
10+
Dispatch: inbox/2026-06-26-handle-issue-1.md (from Einstein)
11+
Spec: docs/plans/2026-06-26-printable-binary-file-container-design.md
12+
Ruling: inbox/processed/2026-06-26-RULING-issue-1-forks.md
13+
14+
## Forks RESOLVED (Einstein ruling, on Peter's behalf, 2026-06-26)
15+
- [x] FORK 1 — Architecture: **A2** (schema-as-spec + shared crc32 in core via FFI;
16+
consumers assemble the trivial JSON envelope natively; cross-impl differential
17+
+ CRC vector-pinning guard drift; web stays pure-JS).
18+
- [x] FORK 2 — Checksum: **CRC-32/ISO-HDLC** (vector-pinned).
19+
- [x] FORK 3 — Naming: **<name>.pbf.json**.
20+
- [~] FORK 4 — Web UX: NON-BLOCKING. Mock BOTH (explicit Encode/Decode AND
21+
auto-detect) for Peter at the web gate; bias toward making decode obvious.
22+
- Minors: snake_case keys; lowercase 8-hex crc; container is ADDITIVE (raw .pbt stays).
23+
24+
## SCM NOTE (resolved 2026-06-26 ~3:52pm EDT) — see Einstein ping
25+
Session began with the working copy on a STALE base (dd0f8014, Jun 10), which
26+
PREDATES the Jun 24 "refactor(ffi): move pb_* exports into ffi.zig" + Jun 25 CI.
27+
First crc32 pass landed there (export wrongly in printable_binary.zig). Re-homed
28+
NON-DESTRUCTIVELY onto yolo (3a6682dc): `jj new yolo` + restored docs + re-applied
29+
code in the CORRECT files. Old work preserved at ef7496c2 (change pqzmoppq/0,
30+
divergent) — recoverable via jj oplog. NOT pushed; stale divergent commit NOT
31+
abandoned — both await Peter's ok.
32+
33+
## DONE (core-first TDD, on yolo base, all green)
34+
- [x] `pub fn crc32` (src/zig/printable_binary.zig) — CRC-32/ISO-HDLC, bitwise,
35+
vector test (CRC32("")=0, "123456789"=0xCBF43926, "a"=0xE8B7BE43). RED→GREEN.
36+
- [x] `export fn pb_crc32` (src/zig/ffi.zig) — delegates to pb.crc32; FFI test
37+
(vector + null-safety). RED→GREEN.
38+
- [x] Green on: test-zig-unit, test-ffi-cli, test-zig (ReleaseFast + cross-impl).
39+
no-ffi-symbols invariant safe by construction (Linux CI enforces on push).
40+
41+
## NEXT (after Peter spec sign-off; web held for Peter mockup)
42+
- [ ] C header decl `uint32_t pb_crc32(const char*, size_t);` (src/printable_binary.h)
43+
— add when the CLI container verb consumes it (TDD-driven).
44+
- [ ] Container codec — round-trip oracle FIRST: decode(encode(bytes,meta)) ==
45+
(bytes,meta), byte+metadata identical. JS side (js/printable_binary.js) +
46+
C CLI side.
47+
- [ ] JS CRC-32 (vector-pinned to same constant) for the web demo.
48+
- [ ] CLI verb to emit/consume <name>.pbf.json (-/@stdin/@stdout + JSON I/O conv).
49+
- [ ] Cross-impl differential: C-built container decodes in JS & vice versa.
50+
- [ ] Web demo: decode UI (paste + drop) + metadata restore. MOCK BOTH for Peter
51+
FIRST (can't see rendered output), then build.
52+
- [ ] Docs: README section, dirtree notes, schema doc finalize.
53+
54+
## Discipline
55+
jj-only (never raw git); rm-safe (mv ~/.Trash, never rm); per-unit ./test green;
56+
./build + full suite before push; commit per logical unit; ping Einstein at milestones.
Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Design: printable-binary-file.json container + web decode workflow
2+
3+
**Date:** 2026-06-26
4+
**Issue:** https://github.com/pmarreck/printable_binary/issues/1
5+
**Status:** DRAFT — awaiting Peter's spec sign-off (forks 1–4 ruled by Einstein
6+
2026-06-26; see inbox/processed/2026-06-26-RULING-issue-1-forks.md)
7+
8+
## Problem
9+
The browser demo bills itself "Encoder/Decoder" but (a) offers **no clear
10+
DECODE workflow** — you can only decode by drag-dropping a `.pbt` file; there is
11+
no way to paste a printable-binary block and get a binary back — and (b) has
12+
**no way to preserve/restore file metadata** (filename, perms, owner/group,
13+
timestamps). Encoding `foo.png` yields `foo.png.pbt` and discards everything
14+
else.
15+
16+
## Resolved decisions (Einstein ruling, on Peter's behalf)
17+
1. **Architecture = A2.** The JSON envelope is a documented *schema* (the single
18+
source of truth). The only *algorithms* — the existing bytes↔glyphs codec and
19+
the new CRC-32 — live in the Zig core and are reached through the C FFI. Each
20+
consumer (C CLI in C, web demo in JS) assembles/parses the trivial flat-dict
21+
envelope with its native JSON facilities. Cross-impl agreement is guaranteed
22+
by a **differential test** (container built by C decodes in JS and vice
23+
versa) plus **CRC vector-pinning** — not by routing JSON through WASM.
24+
2. **Checksum = CRC-32/ISO-HDLC** (the zip/gzip/png CRC; reflected, poly
25+
`0xEDB88320`, init/xorout `0xFFFFFFFF`). Vector-pinned:
26+
`CRC32("123456789") == 0xCBF43926`.
27+
3. **On-disk name = `<originalname>.pbf.json`** (e.g. `photo.png.pbf.json`).
28+
Keeps the name+ext visible, gives editors/jq a `.json` affordance, `.pbf`
29+
marks it. The authoritative filename round-trips inside the `filename` key.
30+
4. **Web UX = deferred, non-blocking.** Does not gate core/CLI. At the web gate,
31+
mock BOTH an explicit Encode/Decode affordance AND an auto-detect+paste-box,
32+
and let Peter pick — biasing toward making decode *obvious* (the issue's real
33+
complaint is discoverability).
34+
35+
Minors: `snake_case` keys; CRC fields are lowercase 8-hex strings; the container
36+
is **additive** — the raw `.pbt` encode/decode flow stays; the container is a
37+
NEW option, not a replacement.
38+
39+
## Container schema v1 (`printable-binary-file`)
40+
```json
41+
{
42+
"format": "printable-binary-file",
43+
"version": 1,
44+
"filename": "photo.png",
45+
"data": "<printable-binary-encoded string>",
46+
"byte_length": 12345,
47+
"crc32": "cbf43926",
48+
"crc32_encoded": "1a2b3c4d",
49+
"modified_ms": 1719430000000,
50+
"created_ms": 1719420000000,
51+
"mode": "0644",
52+
"owner": "pmarreck",
53+
"group": "staff"
54+
}
55+
```
56+
57+
### Field semantics
58+
| key | type | required | meaning |
59+
|----------------|---------|----------|---------|
60+
| `format` | string | yes | literal `"printable-binary-file"` (format discriminator) |
61+
| `version` | number | yes | schema version (1) |
62+
| `filename` | string | yes | original base filename (authoritative; restored on decode) |
63+
| `data` | string | yes | the printable-binary-encoded original bytes |
64+
| `byte_length` | number | yes | length in bytes of the ORIGINAL (decoded) data |
65+
| `crc32` | string | yes | CRC-32 of the ORIGINAL bytes, lowercase 8-hex |
66+
| `crc32_encoded`| string | opt | CRC-32 of the UTF-8 bytes of the `data` string |
67+
| `modified_ms` | number | opt | mtime, integer ms since Unix epoch (UTC) |
68+
| `created_ms` | number | opt | birthtime, ms since epoch (omit if unavailable) |
69+
| `mode` | string | opt | POSIX permission bits as octal string, e.g. `"0644"` |
70+
| `owner` | string | opt | owner username (POSIX) |
71+
| `group` | string | opt | group name (POSIX) |
72+
73+
### Cross-platform rule
74+
POSIX-only fields (`mode`, `owner`, `group`) and `created_ms` are **omitted when
75+
unavailable** (Windows, browser). **Decode MUST tolerate any missing optional
76+
field and never fail because one is absent.** The browser File API yields only
77+
`name`, `size`, and `lastModified` (ms) — so the web demo populates `filename`,
78+
`byte_length`, `modified_ms`, `crc32` (and `crc32_encoded`), omitting the rest.
79+
80+
### Self-verification semantics (decode)
81+
- If `crc32_encoded` is present, verify it against the UTF-8 bytes of `data`
82+
BEFORE decoding (catches transport corruption of the JSON/`data` itself).
83+
- After decoding `data` → bytes, recompute CRC-32 and compare to `crc32`;
84+
recompute length and compare to `byte_length`.
85+
- A mismatch is a hard error in the CLI (non-zero exit, message to stderr) and a
86+
clear error in the web UI. (No silent acceptance of corrupted round-trips.)
87+
88+
## Architecture / where each piece lives
89+
- **Zig core (`src/zig/printable_binary.zig`)**
90+
- Existing: `pb_encode` / `pb_decode` (the codec) — unchanged, reused.
91+
- NEW `pb_crc32(input: ?[*]const u8, len: usize) -> u32` exported
92+
`callconv(.c)`. Pure, no I/O. Table-driven CRC-32/ISO-HDLC.
93+
- **C FFI header (`src/printable_binary.h`)**: declare `uint32_t pb_crc32(const
94+
char*, size_t);`.
95+
- **C CLI (`src/printable_binary.c` + FFI CLI)**: a verb/flag to EMIT a container
96+
(`--to-file-json` / read file+stat → assemble JSON) and to CONSUME one
97+
(`--from-file-json` / parse JSON → write bytes + restore metadata). Honors
98+
`-`/`@stdin`/`@stdout`. Names exact flags during implementation (kept Unix +
99+
Windows-alias-friendly per CLI conventions).
100+
- **JS (`js/printable_binary.js`)**: container assemble/parse methods + a JS
101+
CRC-32 (vector-pinned to the same constant) used by the web demo.
102+
- **No new format logic in WASM**; the web demo stays pure-JS.
103+
104+
## CRC-32 detail (to avoid impl drift)
105+
CRC-32/ISO-HDLC, reflected input/output, polynomial `0xEDB88320` (reflected
106+
`0x04C11DB7`), init `0xFFFFFFFF`, final xor `0xFFFFFFFF`. Both the Zig and JS
107+
implementations are unit-tested against published vectors:
108+
`""``0x00000000`, `"123456789"``0xCBF43926`, `"a"``0xE8B7BE43`. Because
109+
both are pinned to the SAME published oracle, authorship is irrelevant and they
110+
cannot silently disagree (MFIC: external oracle).
111+
112+
## TDD oracle / test plan (write tests FIRST)
113+
1. **CRC-32 vectors** (Zig unit + JS): the published vectors above. RED first.
114+
2. **Container round-trip** (the primary MFIC inverse-pair oracle):
115+
`decode(encode(bytes, meta)) == (bytes, meta)` — byte-identical `data` AND
116+
identical metadata (modulo platform-omitted fields). RED first.
117+
3. **Self-verify negative tests**: flip a byte in `data`, or corrupt a CRC
118+
field, → decode errors (proves the check isn't vacuous).
119+
4. **Cross-impl differential** (added to `test/test_cross_implementation.sh`):
120+
a container emitted by the C CLI decodes correctly in JS, and a
121+
JS/Node-emitted container decodes correctly via the C CLI — byte + metadata
122+
identical. External oracle against envelope drift.
123+
5. **Missing-optional-field tolerance**: a container with only the required
124+
fields decodes cleanly (no POSIX fields) — proves cross-platform tolerance.
125+
126+
## Non-goals (YAGNI)
127+
- No cryptographic/tamper-evidence hashing (CRC-32 is transport-integrity only).
128+
- No compression (the codec is the codec; container just wraps it).
129+
- No multi-file/archive container (single file per container in v1).
130+
- No restoration of owner/group/mode by the *web* demo (browser can't set them);
131+
it restores filename + mtime where the download mechanism allows.
132+
133+
## Build order
134+
core CRC-32 (+FFI) → container codec + round-trip oracle → CLI verb → cross-impl
135+
differential → web UI (after Peter picks the mockup). Ping Einstein at the core
136+
and CLI milestones.

src/zig/ffi.zig

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,3 +224,16 @@ test "FFI: pb_hexlike_encode/decode roundtrip + detect + null safety" {
224224
try std.testing.expect(bad.error_code != 0);
225225
try std.testing.expect(pb_detect_hexlike(null, 100) == 0);
226226
}
227+
228+
/// FFI: CRC-32/ISO-HDLC of `len` bytes at `input`. Null ptr (or len 0) yields
229+
/// the CRC of empty input (0). Delegates to the pure core `pb.crc32`.
230+
export fn pb_crc32(input: ?[*]const u8, len: usize) callconv(.c) u32 {
231+
if (input == null or len == 0) return pb.crc32("");
232+
return pb.crc32(input.?[0..len]);
233+
}
234+
235+
test "FFI: pb_crc32 matches vectors + null safety" {
236+
const s = "123456789";
237+
try std.testing.expectEqual(@as(u32, 0xCBF43926), pb_crc32(s.ptr, s.len));
238+
try std.testing.expectEqual(@as(u32, 0), pb_crc32(null, 100));
239+
}

src/zig/printable_binary.zig

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1314,3 +1314,33 @@ test "parseGlyphLine: filters comments/blank, strips trailing comment, takes fir
13141314
// CRLF tolerance
13151315
try std.testing.expectEqualStrings("·", parseGlyphLine(\r").?);
13161316
}
1317+
1318+
// =========================================================================
1319+
// CRC-32 (issue #1: printable-binary-file.json container integrity)
1320+
// =========================================================================
1321+
1322+
/// CRC-32/ISO-HDLC (the zip/gzip/png CRC): reflected input/output, polynomial
1323+
/// 0xEDB88320, init 0xFFFFFFFF, final xor 0xFFFFFFFF. Pure helper backing the
1324+
/// container's transport-integrity fields. Bitwise (table-free); ample for
1325+
/// container-sized payloads. `pub` so the FFI surface (ffi.zig) can delegate.
1326+
pub fn crc32(data: []const u8) u32 {
1327+
var crc: u32 = 0xFFFFFFFF;
1328+
for (data) |byte| {
1329+
crc ^= @as(u32, byte);
1330+
var k: usize = 0;
1331+
while (k < 8) : (k += 1) {
1332+
if (crc & 1 != 0) {
1333+
crc = (crc >> 1) ^ 0xEDB88320;
1334+
} else {
1335+
crc >>= 1;
1336+
}
1337+
}
1338+
}
1339+
return ~crc;
1340+
}
1341+
1342+
test "crc32: published vectors (ISO-HDLC)" {
1343+
try std.testing.expectEqual(@as(u32, 0x00000000), crc32(""));
1344+
try std.testing.expectEqual(@as(u32, 0xCBF43926), crc32("123456789"));
1345+
try std.testing.expectEqual(@as(u32, 0xE8B7BE43), crc32("a"));
1346+
}

0 commit comments

Comments
 (0)