|
| 1 | +# Design: printable-binary-file.json container + web decode workflow |
| 2 | + |
| 3 | +**Date:** 2026-06-26 |
| 4 | +**Issue:** https://github.com/pmarreck/printable_binary/issues/1 |
| 5 | +**Status:** DRAFT — awaiting Peter's spec sign-off (forks 1–4 ruled by Einstein |
| 6 | +2026-06-26; see inbox/processed/2026-06-26-RULING-issue-1-forks.md) |
| 7 | + |
| 8 | +## Problem |
| 9 | +The browser demo bills itself "Encoder/Decoder" but (a) offers **no clear |
| 10 | +DECODE workflow** — you can only decode by drag-dropping a `.pbt` file; there is |
| 11 | +no way to paste a printable-binary block and get a binary back — and (b) has |
| 12 | +**no way to preserve/restore file metadata** (filename, perms, owner/group, |
| 13 | +timestamps). Encoding `foo.png` yields `foo.png.pbt` and discards everything |
| 14 | +else. |
| 15 | + |
| 16 | +## Resolved decisions (Einstein ruling, on Peter's behalf) |
| 17 | +1. **Architecture = A2.** The JSON envelope is a documented *schema* (the single |
| 18 | + source of truth). The only *algorithms* — the existing bytes↔glyphs codec and |
| 19 | + the new CRC-32 — live in the Zig core and are reached through the C FFI. Each |
| 20 | + consumer (C CLI in C, web demo in JS) assembles/parses the trivial flat-dict |
| 21 | + envelope with its native JSON facilities. Cross-impl agreement is guaranteed |
| 22 | + by a **differential test** (container built by C decodes in JS and vice |
| 23 | + versa) plus **CRC vector-pinning** — not by routing JSON through WASM. |
| 24 | +2. **Checksum = CRC-32/ISO-HDLC** (the zip/gzip/png CRC; reflected, poly |
| 25 | + `0xEDB88320`, init/xorout `0xFFFFFFFF`). Vector-pinned: |
| 26 | + `CRC32("123456789") == 0xCBF43926`. |
| 27 | +3. **On-disk name = `<originalname>.pbf.json`** (e.g. `photo.png.pbf.json`). |
| 28 | + Keeps the name+ext visible, gives editors/jq a `.json` affordance, `.pbf` |
| 29 | + marks it. The authoritative filename round-trips inside the `filename` key. |
| 30 | +4. **Web UX = deferred, non-blocking.** Does not gate core/CLI. At the web gate, |
| 31 | + mock BOTH an explicit Encode/Decode affordance AND an auto-detect+paste-box, |
| 32 | + and let Peter pick — biasing toward making decode *obvious* (the issue's real |
| 33 | + complaint is discoverability). |
| 34 | + |
| 35 | +Minors: `snake_case` keys; CRC fields are lowercase 8-hex strings; the container |
| 36 | +is **additive** — the raw `.pbt` encode/decode flow stays; the container is a |
| 37 | +NEW option, not a replacement. |
| 38 | + |
| 39 | +## Container schema v1 (`printable-binary-file`) |
| 40 | +```json |
| 41 | +{ |
| 42 | + "format": "printable-binary-file", |
| 43 | + "version": 1, |
| 44 | + "filename": "photo.png", |
| 45 | + "data": "<printable-binary-encoded string>", |
| 46 | + "byte_length": 12345, |
| 47 | + "crc32": "cbf43926", |
| 48 | + "crc32_encoded": "1a2b3c4d", |
| 49 | + "modified_ms": 1719430000000, |
| 50 | + "created_ms": 1719420000000, |
| 51 | + "mode": "0644", |
| 52 | + "owner": "pmarreck", |
| 53 | + "group": "staff" |
| 54 | +} |
| 55 | +``` |
| 56 | + |
| 57 | +### Field semantics |
| 58 | +| key | type | required | meaning | |
| 59 | +|----------------|---------|----------|---------| |
| 60 | +| `format` | string | yes | literal `"printable-binary-file"` (format discriminator) | |
| 61 | +| `version` | number | yes | schema version (1) | |
| 62 | +| `filename` | string | yes | original base filename (authoritative; restored on decode) | |
| 63 | +| `data` | string | yes | the printable-binary-encoded original bytes | |
| 64 | +| `byte_length` | number | yes | length in bytes of the ORIGINAL (decoded) data | |
| 65 | +| `crc32` | string | yes | CRC-32 of the ORIGINAL bytes, lowercase 8-hex | |
| 66 | +| `crc32_encoded`| string | opt | CRC-32 of the UTF-8 bytes of the `data` string | |
| 67 | +| `modified_ms` | number | opt | mtime, integer ms since Unix epoch (UTC) | |
| 68 | +| `created_ms` | number | opt | birthtime, ms since epoch (omit if unavailable) | |
| 69 | +| `mode` | string | opt | POSIX permission bits as octal string, e.g. `"0644"` | |
| 70 | +| `owner` | string | opt | owner username (POSIX) | |
| 71 | +| `group` | string | opt | group name (POSIX) | |
| 72 | + |
| 73 | +### Cross-platform rule |
| 74 | +POSIX-only fields (`mode`, `owner`, `group`) and `created_ms` are **omitted when |
| 75 | +unavailable** (Windows, browser). **Decode MUST tolerate any missing optional |
| 76 | +field and never fail because one is absent.** The browser File API yields only |
| 77 | +`name`, `size`, and `lastModified` (ms) — so the web demo populates `filename`, |
| 78 | +`byte_length`, `modified_ms`, `crc32` (and `crc32_encoded`), omitting the rest. |
| 79 | + |
| 80 | +### Self-verification semantics (decode) |
| 81 | +- If `crc32_encoded` is present, verify it against the UTF-8 bytes of `data` |
| 82 | + BEFORE decoding (catches transport corruption of the JSON/`data` itself). |
| 83 | +- After decoding `data` → bytes, recompute CRC-32 and compare to `crc32`; |
| 84 | + recompute length and compare to `byte_length`. |
| 85 | +- A mismatch is a hard error in the CLI (non-zero exit, message to stderr) and a |
| 86 | + clear error in the web UI. (No silent acceptance of corrupted round-trips.) |
| 87 | + |
| 88 | +## Architecture / where each piece lives |
| 89 | +- **Zig core (`src/zig/printable_binary.zig`)** |
| 90 | + - Existing: `pb_encode` / `pb_decode` (the codec) — unchanged, reused. |
| 91 | + - NEW `pb_crc32(input: ?[*]const u8, len: usize) -> u32` exported |
| 92 | + `callconv(.c)`. Pure, no I/O. Table-driven CRC-32/ISO-HDLC. |
| 93 | +- **C FFI header (`src/printable_binary.h`)**: declare `uint32_t pb_crc32(const |
| 94 | + char*, size_t);`. |
| 95 | +- **C CLI (`src/printable_binary.c` + FFI CLI)**: a verb/flag to EMIT a container |
| 96 | + (`--to-file-json` / read file+stat → assemble JSON) and to CONSUME one |
| 97 | + (`--from-file-json` / parse JSON → write bytes + restore metadata). Honors |
| 98 | + `-`/`@stdin`/`@stdout`. Names exact flags during implementation (kept Unix + |
| 99 | + Windows-alias-friendly per CLI conventions). |
| 100 | +- **JS (`js/printable_binary.js`)**: container assemble/parse methods + a JS |
| 101 | + CRC-32 (vector-pinned to the same constant) used by the web demo. |
| 102 | +- **No new format logic in WASM**; the web demo stays pure-JS. |
| 103 | + |
| 104 | +## CRC-32 detail (to avoid impl drift) |
| 105 | +CRC-32/ISO-HDLC, reflected input/output, polynomial `0xEDB88320` (reflected |
| 106 | +`0x04C11DB7`), init `0xFFFFFFFF`, final xor `0xFFFFFFFF`. Both the Zig and JS |
| 107 | +implementations are unit-tested against published vectors: |
| 108 | +`""` → `0x00000000`, `"123456789"` → `0xCBF43926`, `"a"` → `0xE8B7BE43`. Because |
| 109 | +both are pinned to the SAME published oracle, authorship is irrelevant and they |
| 110 | +cannot silently disagree (MFIC: external oracle). |
| 111 | + |
| 112 | +## TDD oracle / test plan (write tests FIRST) |
| 113 | +1. **CRC-32 vectors** (Zig unit + JS): the published vectors above. RED first. |
| 114 | +2. **Container round-trip** (the primary MFIC inverse-pair oracle): |
| 115 | + `decode(encode(bytes, meta)) == (bytes, meta)` — byte-identical `data` AND |
| 116 | + identical metadata (modulo platform-omitted fields). RED first. |
| 117 | +3. **Self-verify negative tests**: flip a byte in `data`, or corrupt a CRC |
| 118 | + field, → decode errors (proves the check isn't vacuous). |
| 119 | +4. **Cross-impl differential** (added to `test/test_cross_implementation.sh`): |
| 120 | + a container emitted by the C CLI decodes correctly in JS, and a |
| 121 | + JS/Node-emitted container decodes correctly via the C CLI — byte + metadata |
| 122 | + identical. External oracle against envelope drift. |
| 123 | +5. **Missing-optional-field tolerance**: a container with only the required |
| 124 | + fields decodes cleanly (no POSIX fields) — proves cross-platform tolerance. |
| 125 | + |
| 126 | +## Non-goals (YAGNI) |
| 127 | +- No cryptographic/tamper-evidence hashing (CRC-32 is transport-integrity only). |
| 128 | +- No compression (the codec is the codec; container just wraps it). |
| 129 | +- No multi-file/archive container (single file per container in v1). |
| 130 | +- No restoration of owner/group/mode by the *web* demo (browser can't set them); |
| 131 | + it restores filename + mtime where the download mechanism allows. |
| 132 | + |
| 133 | +## Build order |
| 134 | +core CRC-32 (+FFI) → container codec + round-trip oracle → CLI verb → cross-impl |
| 135 | +differential → web UI (after Peter picks the mockup). Ping Einstein at the core |
| 136 | +and CLI milestones. |
0 commit comments