|
| 1 | +// SPDX-License-Identifier: MPL-2.0 |
| 2 | +// SPDX-FileCopyrightText: 2025-2026 hyperpolymath |
| 3 | += Phase F — string-wall slice 1: string indexing (evidence) |
| 4 | +:toc: macro |
| 5 | + |
| 6 | +[IMPORTANT] |
| 7 | +==== |
| 8 | +*Slice landed: read-side string indexing in the wasm backend.* The |
| 9 | +`string_char_code_at(s, i)` and `char_to_int(c)` builtins — already present |
| 10 | +in resolve/typecheck/interp — gained wasm-backend lowerings on the |
| 11 | +`[len: i32 LE][utf8 bytes...]` ABI. A previously STRING-GATED shape (byte |
| 12 | +indexing into a string literal) now compiles to wasm *and* parity-greens |
| 13 | +against the interpreter oracle. |
| 14 | +
|
| 15 | +This is the first slice of the *variable-string backend* wall |
| 16 | +(`proposals/MIGRATION-PLAN.adoc` §"The two walls", Phase F). It removes the |
| 17 | +wall for the *indexing/scan* family; allocation-returning string ops |
| 18 | +(`string_from_char_code`, `string_sub`, `slice`, concat, case-folding) remain |
| 19 | +for later slices. |
| 20 | +==== |
| 21 | + |
| 22 | +toc::[] |
| 23 | + |
| 24 | +== What was missing |
| 25 | + |
| 26 | +The string-builtin surface was wired end-to-end *except* the wasm backend: |
| 27 | + |
| 28 | +[cols="2,1,1,1,1",options="header"] |
| 29 | +|=== |
| 30 | +| Builtin | resolve.ml | typecheck.ml | interp.ml | codegen.ml (wasm) |
| 31 | +| `string_length` | ✓ | ✓ | ✓ | ✓ (pre-existing, `I32Load (2,0)`) |
| 32 | +| `string_char_code_at` | ✓ | ✓ | ✓ | *was missing -> added* |
| 33 | +| `char_to_int` | ✓ | ✓ | ✓ | *was missing -> added* |
| 34 | +|=== |
| 35 | + |
| 36 | +Before this slice, compiling `string_char_code_at(s, i)` failed at codegen: |
| 37 | + |
| 38 | +---- |
| 39 | +Code generation error: (Codegen.UnboundVariable |
| 40 | + "Function or variable not found: string_char_code_at") |
| 41 | +---- |
| 42 | + |
| 43 | +The builtin fell through the `gen_call` dispatch to the generic |
| 44 | +named-function path, which only knows user functions + the already-handled |
| 45 | +builtins — hence "unbound". The fix is a dispatch arm that emits the load |
| 46 | +sequence, exactly as `string_length` already does for length. |
| 47 | + |
| 48 | +== The lowering (lib/codegen.ml) |
| 49 | + |
| 50 | +String layout is `[len: i32 LE]` at the base pointer, with utf8 bytes |
| 51 | +starting at offset 4. Byte `i` therefore lives at `base + 4 + i`. |
| 52 | + |
| 53 | +`string_char_code_at(s, i)` — total function, `-1` on out-of-bounds (the |
| 54 | +shared absent-byte sentinel; the interp oracle returns the same): |
| 55 | + |
| 56 | +---- |
| 57 | +s_code; LocalSet base |
| 58 | +i_code; LocalSet idx |
| 59 | +;; condition: (idx >= 0) && (idx < len) |
| 60 | +LocalGet idx; I32Const 0; I32GeS |
| 61 | +LocalGet idx; LocalGet base; I32Load; I32LtS |
| 62 | +I32And |
| 63 | +If (result i32) |
| 64 | + ;; in-bounds: zero-extend byte at base + idx + 4 |
| 65 | + then: LocalGet base; LocalGet idx; I32Add; I32Load8U (offset=4) |
| 66 | + ;; out-of-bounds sentinel |
| 67 | + else: I32Const -1 |
| 68 | +---- |
| 69 | + |
| 70 | +The bounds test *guards* the load (`If`, not `Select`): an out-of-bounds |
| 71 | +`I32Load8U` could trap or read foreign linear memory, so the load must not |
| 72 | +execute unless the index is in range. `Select` would evaluate both arms. |
| 73 | + |
| 74 | +`char_to_int(c)` — identity on the wasm value lattice. A `Char` is already |
| 75 | +an i32 byte code (`LitChar` lowers to `I32Const (Char.code c)`), so the arm |
| 76 | +just evaluates the argument and leaves it; no conversion instruction. |
| 77 | + |
| 78 | +== Gate evidence |
| 79 | + |
| 80 | +=== Gate 1 — builds |
| 81 | + |
| 82 | +`dune build bin/main.exe` exit 0. The previously-failing probe now compiles: |
| 83 | + |
| 84 | +---- |
| 85 | +$ affinescript compile idx.affine -o idx.wasm # fn main = string_char_code_at("ABC",1) |
| 86 | +Compiled idx.affine -> idx.wasm (WASM) # exit 0 |
| 87 | +---- |
| 88 | + |
| 89 | +=== Gate 2 — parity (wasm vs interpreter oracle) |
| 90 | + |
| 91 | +Same inputs through both backends agree. Interp oracle confirmed against the |
| 92 | +real library API (`Parse_driver.parse_string` -> `Interp.eval_program` -> |
| 93 | +`Value.lookup_env "f"` -> `apply_function`); wasm executed under Node. |
| 94 | + |
| 95 | +[cols="3,1,1",options="header"] |
| 96 | +|=== |
| 97 | +| Expression | interp | wasm |
| 98 | +| `string_char_code_at("ABC", 0)` | 65 | 65 |
| 99 | +| `string_char_code_at("ABC", 1)` | 66 | 66 |
| 100 | +| `string_char_code_at("ABC", 2)` | 67 | 67 |
| 101 | +| `string_char_code_at("ABC", -1)` | -1 | -1 |
| 102 | +| `string_char_code_at("ABC", 9)` | -1 | -1 |
| 103 | +| `string_char_code_at("", 0)` | -1 | -1 |
| 104 | +| `char_to_int('Z')` | 90 | 90 |
| 105 | +|=== |
| 106 | + |
| 107 | +The packed fixture `tests/codegen/string_char_code_at.affine` returns |
| 108 | +`4276896` (positional base-256 pack of the three in-bounds bytes, minus the |
| 109 | +three `-1` sentinels, plus `char_to_int('Z')`); both backends produce it. |
| 110 | + |
| 111 | +=== Type safety preserved |
| 112 | + |
| 113 | +`char_to_int(65)` (Int where Char is required) is still rejected: |
| 114 | +`Unification error: (Unify.TypeMismatch (Char, Int))`. The wasm identity |
| 115 | +lowering does not loosen the front-end Char/Int distinction. |
| 116 | + |
| 117 | +== Tests added |
| 118 | + |
| 119 | +* `test/test_e2e.ml` — group *"E2E String-wall slice 1 (indexing)"*: seven |
| 120 | + interp-oracle cases (in-bounds bytes, both OOB directions, empty string, |
| 121 | + `char_to_int`). Runs under `dune runtest`. This is the interp consumer |
| 122 | + coverage mandated by `.claude/CLAUDE.md` §"Test-fixture hygiene". |
| 123 | +* `tests/codegen/string_char_code_at.affine` + |
| 124 | + `tests/codegen/test_string_char_code_at.mjs` — executable wasm parity, |
| 125 | + run by `tools/run_codegen_wasm_tests.sh` (CI). Asserts `main()` against the |
| 126 | + oracle constant; the same constants are pinned interp-side above. |
| 127 | + |
| 128 | +Full `tools/run_codegen_wasm_tests.sh` run: *all* codegen WASM tests pass, |
| 129 | +including the new harness, no sibling regressions. |
| 130 | + |
| 131 | +== Corpus impact |
| 132 | + |
| 133 | +This slice unblocks the *indexing/scan* shape — code that reads bytes out of |
| 134 | +a string and compares/branches on them (first-byte dispatch, manual scanning, |
| 135 | +char-class checks). It does *not* yet unblock string-gated kernels whose work |
| 136 | +is dominated by *building new strings* (`*ToString`/`*FromString` |
| 137 | +serialisers, `String.replaceAll` interpolation, translation-dict lookups |
| 138 | +keyed by string), which need the allocation-returning ops in later slices. |
| 139 | + |
| 140 | +== Next slices (variable-string backend wall) |
| 141 | + |
| 142 | +. *Allocation-returning ABI* — a shared `gen_string_alloc` helper writing the |
| 143 | + `[len][bytes]` shape into the heap, then `string_from_char_code(n)` as its |
| 144 | + first consumer (one-byte string `[1][n & 0xff]`). |
| 145 | +. *`string_sub` / `slice`* — bounded copy loops over the byte range. |
| 146 | +. *`startsWith` / `string_find`* — prefix/substring scans (read-side; can |
| 147 | + build on this slice's indexing without allocation). |
| 148 | +. *Concat + case-folding* — `to_lowercase` / `to_uppercase` / `++` on strings. |
| 149 | + |
| 150 | +Each slice: add the codegen arm, an interp-parity e2e group, and a |
| 151 | +`tests/codegen/*.mjs` executable check, then re-run the census and drop the |
| 152 | +gated count. |
0 commit comments