fix(lazy): type predicates + comprehensive audit document (v0.5.212)

Three correctness fixes for type predicates against lazy JSON.parse
results, plus a ~1400-line external-auditor document covering the
entire tape-based lazy JSON system.

Type predicate fixes:

1. Array.isArray(parsed) returned false on lazy arrays. Root cause
   was actually broader than the lazy path: Expr::ArrayIsArray in
   perry-codegen/src/expr.rs was a pure compile-time check that
   emitted TAG_FALSE whenever the operand's static HIR type wasn't
   definitively Array/Tuple. For JSON.parse results (typed `any`)
   this always failed, so Array.isArray(anyTyped) returned false
   even on eager arrays at runtime. Fixed by routing indeterminate
   static types (Any, Unknown, no annotation) through a new
   js_array_is_array runtime dispatch; the fast-path for
   definitively-array statics still emits TAG_TRUE, and
   definitively-scalar types (Number/String/Boolean/Null/Void/BigInt/
   Symbol) still emit TAG_FALSE as compile-time constants.

2. parsed instanceof Array returned false on lazy arrays. Root cause:
   js_instanceof in perry-runtime/src/object.rs (CLASS_ID_ARRAY
   branch) only matched GC_TYPE_ARRAY, not GC_TYPE_LAZY_ARRAY.
   Added the lazy type to the OR.

3. Array.isArray runtime (js_array_is_array) already had the same
   eager-only check; extended to accept GC_TYPE_LAZY_ARRAY too.
   Now reachable via the new codegen dispatch path.

Pre-existing limitation parsed.constructor === Array (returns false
on Perry regardless of lazy vs eager — property-lookup limitation in
Perry's class system) remains unchanged. Not in scope for this fix.

New audit document docs/audit-lazy-json.md covers:
 - Terminology + scope of the lazy path
 - Data structure layouts (TapeEntry 12 bytes, LazyArrayHeader
   offset invariants, arena layout)
 - Dispatch tables for every user-observable array operation
 - Garbage collection: trace_lazy_array + the cache zero-on-alloc
   invariant (a real bug caught during v0.5.208 development)
 - Correctness proof via 14-row case analysis against every JS
   array operation
 - Thread safety + GC safety
 - Performance characteristics: measured numbers vs Node + Bun on
   all three main benches + two new synthetic benches (sequential
   + random iteration)
 - 15 edge cases with handling (out-of-bounds, empty, deeply nested,
   unicode, mid-GC, etc.)
 - Test coverage across 6 suites (unit, integration, parity, gap,
   fastify, benches-as-correctness-gates)
 - Known limitations + scope boundaries (tape is top-level arrays
   only, 1KB threshold, etc.)
 - 20-item audit checklist with file:line references
 - Change log across v0.5.203 → v0.5.212

Written so a reviewer with no prior Perry knowledge can verify the
entire system against source — every claim has a file:line ref.

New test test_json_lazy_predicates.ts pins the three predicate fixes:
 - Array.isArray(lazyArr) → true
 - lazyArr instanceof Array → true
 - typeof lazyArr → "object"
 - Array.isArray(scalar) → false (guards against overeager fix)
 - isArray after .length / indexed / map all → true
 - eager [1,2,3] arrays still pass both predicates

10/10 existing JSON regressions (test_json_*) match Node byte-for-byte
under direct + lazy paths. Runtime tests 130/130.

Author: proggeramlug

CLAUDE.md

@@ -8,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Perry is a native TypeScript compiler written in Rust that compiles TypeScript source code directly to native executables. It uses SWC for TypeScript parsing and LLVM for code generation.
 
-**Current Version:** 0.5.211
+**Current Version:** 0.5.212
 
 ## TypeScript Parity Status
 
@@ -148,6 +148,7 @@ First-resolved directory cached in `compile_package_dirs`; subsequent imports re
 Keep entries to 1-2 lines max. Full details in CHANGELOG.md.
 
 - **v0.5.205** — Fix #183: `perry compile --target web` on a real-world app (Bloom Jump built on the Bloom engine) produced a WASM binary the browser refused to load — `Compiling function #687 failed: expected 0 elements on the stack for fallthru, found 103` (count varies with engine state). Root cause in `crates/perry-codegen-wasm/src/emit.rs`: the four direct-`Call`-instruction code paths — `Expr::Call` FuncRef arm (~4302), `Expr::Call` ExternFuncRef arm (~4324), `Expr::New` user-class ctor (~5844), `Expr::SuperCall` parent-ctor (~5894), `Expr::StaticMethodCall` direct-static path (~5979) — each emit `emit_expr(arg)` per source arg and pad up with `TAG_UNDEFINED` when `args.len() < expected`, but had no matching drop-excess branch when `args.len() > expected`. WASM `call` consumes exactly the callee's declared param count, so when JS's "extra args evaluated for side effects, then silently ignored" semantics met Perry's WASM codegen, every extra evaluated arg leaked past the call and accumulated on the enclosing block's operand stack — 103 values by the time `_start`'s final `end` hit the validator. The shape that triggered it in jump/bloom was `bloom/src/core/colors.ts`'s `Colors = new __AnonShape_2(...24 PropertyGets...)` landing on a Phase-3-synthesized ctor with lower declared arity, multiplied across bloom's 10 submodules. Fix: after each existing `for _ in args.len()..expected { I64Const(TAG_UNDEFINED) }` pad-up loop, add the mirror `for _ in expected..args.len() { Drop }` — matches JS semantics (extras evaluated for side effects but discarded) and keeps the operand stack aligned with the callee's WASM type at every direct-Call site. Verified end-to-end against the exact issue repro cloned fresh from `github.com/Bloom-Engine/jump` + `github.com/Bloom-Engine/engine`: both path A `file:./vendor/bloom/` and path B `file:../engine/` now compile to a WebAssembly-validating `.wasm` (416,923 / 413,780 bytes respectively, 140 FFI imports intact, `WebAssembly.compile` resolves clean on node 20+); a synthetic `takesFive(mc(),mc(),1,2,3,4)` minimal case that previously failed `Compiling function #213 failed: ... found 1` also validates. `cargo test --release -p perry-runtime -p perry-hir -p perry-codegen-wasm -p perry`: 262/262 passed. Note: issue #183 also claimed path A found only 1 module and emitted 9 FFI imports — could not reproduce in a fresh clone (both paths find 10 modules identically); most likely an artifact of the reporter's local `vendor/bloom` snapshot predating the `exports` map, and the "runGame silently no-ops" symptom the user actually observed was the browser refusing to instantiate the invalid WASM with the surrounding JS glue swallowing the error — fixed here.
+- **v0.5.212** — Three lazy-correctness fixes + comprehensive external-auditor document. (1) `Array.isArray(parsed)` returned `false` on lazy `JSON.parse` results. Root cause: `Expr::ArrayIsArray` in `crates/perry-codegen/src/expr.rs` was a pure compile-time check that emitted `TAG_FALSE` whenever the operand's static HIR type wasn't definitively `Array`/`Tuple`. For `JSON.parse` results (typed `any`) this meant the static check always failed, so even a runtime array always returned false. Fixed by routing indeterminate static types (`Any`, `Unknown`, no annotation) through a new `js_array_is_array` runtime dispatch in the same codegen arm; the fast-path for definitively-array statics still emits `TAG_TRUE` without a runtime call, and definitively-scalar types (Number/String/Boolean/Null/Void/BigInt/Symbol) still emit `TAG_FALSE` statically. (2) `parsed instanceof Array` returned `false` on lazy arrays. Root cause: `js_instanceof` in `crates/perry-runtime/src/object.rs:2864-2890` (the `CLASS_ID_ARRAY` branch) only matched `GC_TYPE_ARRAY`, not `GC_TYPE_LAZY_ARRAY`. Added the lazy-array obj_type to the OR. (3) `Array.isArray` runtime (`crates/perry-runtime/src/array.rs:js_array_is_array`) already had the same eager-only check; extended to accept `GC_TYPE_LAZY_ARRAY` too (now reachable via the new codegen dispatch path). Net effect: every reasonable type-predicate against a lazy array now matches Node byte-for-byte. Pre-existing limitation `parsed.constructor === Array` (returns false on Perry regardless of lazy vs eager — it's a property-lookup limitation in Perry's class system) remains unchanged. The `Array.isArray` codegen change is also a bug-fix for ANY `any`-typed value — before v0.5.212 `Array.isArray(anyTyped)` returned false even when the runtime value was a real eager array; pre-existing latent bug unrelated to lazy work. Verified: `test_json_lazy_per_element.ts` and the new `test_json_lazy_predicates.ts` match Node; all other `test_json_*` regressions clean. New document `docs/audit-lazy-json.md` — **~1400-line comprehensive external-auditor reference** covering the tape-based lazy JSON system end-to-end: layout (TapeEntry, LazyArrayHeader offset invariants, arena layout), dispatch (every user-observable array operation mapped to its runtime entry point), garbage collection (trace_lazy_array, cache zero-on-alloc invariant, tracer safety), correctness proof (14-row case analysis against every JS array operation), performance characteristics (measured numbers vs Node + Bun on all three main benches), edge cases (15 distinct cases with handling), test coverage (every file + gate), known limitations, and a 20-item audit checklist with file:line references. Written so a reviewer with no prior Perry knowledge can verify the entire system against source.
 - **v0.5.211** — Fix pre-existing linker regression from v0.5.204: `js_json_stringify` in `crates/perry-runtime/src/json.rs` was missing `#[no_mangle]`. Rust compiled it as a mangled symbol (`__ZN13perry_runtime4json17js_json_stringify17h…E`) which `perry-stdlib::fastify::context::jsvalue_to_json_string` (and `perry-stdlib::fastify::server::build_response_body`) couldn't find when linking against a statically-built `libperry_runtime.a` — surfaced as `ld: Undefined symbols: _js_json_stringify` when running `scripts/run_fastify_tests.sh` or `cargo test -p perry-stdlib`. Tracking down via `git log -L "/pub unsafe extern \"C\" fn js_json_stringify(/,/^}/"` identified v0.5.204 (`feat(json): lazy parse + lazy stringify`) as the commit that inadvertently removed the attribute when inserting `try_stringify_lazy_array` directly above. All 7 fastify integration tests now pass (GET /hello, GET /users/:id, POST /echo status + body, GET /does-not-exist → 404) — this was blocking end-to-end Fastify coverage since v0.5.204. One-line fix. Also: comprehensive test sweep at v0.5.211: `cargo test --release --workspace` with CI-matching exclusions = **44/44 test runs, 0 failed**, gap tests 24/28 (unchanged), parity tests 106 pass / 12 fail (same 12 as pre-v0.5.208 baseline — no regressions), thread tests 4/4, cache tests PASS, fastify tests 5/5 (newly unblocked), doc tests 95/115 (7 tvos-simulator cross-compile failures are env-dependent — `libperry_ui_tvos.a` built for macOS host not tvOS-sim; pre-existing, unrelated to JSON work).
 - **v0.5.210** — Issue #179 Step 2 completion: **lazy JSON parse is now the default** for top-level array blobs ≥ 1024 bytes. After v0.5.208 per-element sparse materialization + v0.5.209 walk cursor + adaptive materialize threshold, there is no measured access pattern where lazy loses to direct on non-tiny blobs. The `PERRY_JSON_TAPE` env var changed from "opt-in" (`=1` to enable, unset = direct) to "escape hatch" (`=0`/`off`/`false` forces direct, `=1`/`on`/`true` forces tape-on even for small blobs, otherwise auto). Lookup cached via `OnceLock` so the env-var check is amortized to once per process (was per-parse; 100k tight-loop parses saw a ~5 ms difference just from the lookup on macOS). Size threshold 1024 bytes chosen because a small-array bench (`'[1,2,3,4,5,6,7,8,9,10]'` × 100k iters) measured tape overhead ~10% over direct when the tape build + LazyArrayHeader cache/bitmap allocation cost didn't amortize over enough lookups; above the threshold the overhead is swamped by the parse-time savings. `@perry-lazy` JSDoc pragma still forces tape at the codegen level (via `js_json_parse_lazy`), unconditional of size. **Measured impact of flip on the three main benches (best-of-5, macOS ARM64)**: `bench_json_roundtrip` 400 ms / 137 MB → **90 ms / 130 MB** (4.4× faster). `bench_json_readonly` 290 ms / 120 MB → **80 ms / 90 MB** (3.6× faster, 25% less RSS). `bench_json_readonly_indexed` 300 ms / 120 MB → **90 ms / 90 MB** (3.3× faster, 25% less RSS). **vs Node 25.8.0** on the same benches: roundtrip Perry 90 ms vs Node 520 ms (5.8×), readonly Perry 80 ms vs Node 450 ms (5.6×), indexed Perry 90 ms vs Node 450 ms (5.0×). **vs Bun 1.3.12**: roundtrip Perry 90 ms vs Bun 290 ms (3.2×), readonly Perry 80 ms vs Bun 200 ms (2.5×), indexed Perry 90 ms vs Bun 210 ms (2.3×). Perry now leads every measured JSON workload. RSS gap to Bun (~50%) remains — closing that requires tier-3 generational GC per `docs/memory-perf-roadmap.md`. Gap test sweep unchanged at 24/28 (same pre-existing non-lazy failures). All existing `test_json_*.ts` pass byte-for-byte vs Node under default + forced-off + forced-on modes. Runtime tests 130/130. The original plan to implement static HIR analysis for lazy-safety auto-detection (v0.5.209 task) is closed without implementation — the runtime adaptive handling makes compile-time analysis unnecessary since lazy is now a strict improvement in the measured workload matrix.
 - **v0.5.209** — Issue #179 follow-up to v0.5.208: walk cursor + adaptive materialize threshold. Eliminates two remaining performance cliffs in the per-element lazy path. **Cliff 1: sequential iteration was O(n²).** `for (let i = 0; i < parsed.length; i++) parsed[i].field` on a 10k-element blob walked the tape from root on every access (0 + 1 + 2 + ... + 9999 = ~50M link chases), measured 3250 ms vs direct 40 ms (80× regression). Fix: new `walk_idx` + `walk_tape_pos` fields on `LazyArrayHeader` track the most recent (element index, tape offset) pair; when `lazy_get(i)` is called with `i >= walk_idx`, resume walking from the cached cursor instead of root. Sequential access amortizes to O(1) per step — measured 3250 ms → **51 ms** on the same bench (63× fix), parity with direct (39 ms). Cursor resets on `i < walk_idx` (reverse or out-of-order access) to keep correctness simple. **Cliff 2: random-order access was still O(n) per element.** `parsed[perm[i]]` with a shuffled permutation averages n/2 walk steps per cold access; 50k accesses on a 10k array = 250M walks = measured 525 ms vs direct 7 ms (75× regression). Fix: new `cumulative_walk_steps: u64` counter on `LazyArrayHeader`, incremented by `(i - start_count)` on every cold-path call. When cumulative steps exceed `2 × cached_length`, trigger `force_materialize_lazy` — at that point future walks cost more than one full-materialize (which is O(cached_length)), so flipping to tree access is the strict win. Sequential access averages 1 step per element and never trips the threshold; random access averages n/2 per step and trips after ~4 accesses on a 10k array. Post-trip, `lazy_get`'s fast path at the top (`if materialized != null`) hits immediately on every subsequent call and reads directly from the `ArrayHeader`. Measured 525 ms → **10 ms** on the random bench (52× fix). The post-trip path still preserves identity: `force_materialize_lazy` consults the sparse cache and reuses already-materialized JSValues in the new tree, so `parsed[i] === savedRef` remains true across the threshold. New test `test_json_lazy_iteration.ts` covers sequential / reverse / random / stringify-after-iter / repeated-identity / threshold-trip patterns; all match Node byte-for-byte. Existing `test_json_lazy_*` / `test_json_typed_*` / `test_json_pragma_lazy` / `test_json_lazy_per_element` all green under direct + lazy. Runtime tests 130/130. The three main JSON benches are unaffected (sequential cursor not hit because they don't do full iteration): `bench_json_roundtrip` lazy 72 ms / 130 MB, `bench_json_readonly` lazy 67 ms / 90 MB, `bench_json_readonly_indexed` lazy 76 ms / 90 MB — all steady vs v0.5.208. **With this commit there is no measured workload shape where lazy loses to direct.** The static HIR auto-detection originally planned for v0.5.209 is downgraded in urgency: runtime adaptive handling now makes "always lazy" a safe default for every access pattern we measured.

Cargo.lock

@@ -109,7 +109,7 @@ opt-level = "s"       # Optimize for size in stdlib
 opt-level = 3
 
 [workspace.package]
-version = "0.5.211"
+version = "0.5.212"
 edition = "2021"
 license = "MIT"
 repository = "https://github.com/PerryTS/perry"

Cargo.toml

@@ -6125,15 +6125,42 @@ pub(crate) fn lower_expr(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result<String> {
             Ok(ctx.block().call(DOUBLE, "js_date_set_utc_month", &[(DOUBLE, &d), (DOUBLE, &v)]))
         }
         Expr::ArrayIsArray(o) => {
-            // Compile-time check: emit TAG_TRUE if the operand is
-            // statically an array, else TAG_FALSE. NaN-boxed booleans
-            // so console.log prints "true"/"false".
-            let _ = lower_expr(ctx, o)?;
+            // Fast path: static type is definitively array → emit
+            // TAG_TRUE at compile time. Slow path: indeterminate
+            // type (Any / Unknown / no annotation) → emit runtime
+            // call to `js_array_is_array`, which correctly handles
+            // JSON.parse results, closure-captured values, function
+            // returns typed `any`, and lazy arrays
+            // (GC_TYPE_LAZY_ARRAY). Emitting TAG_FALSE as a compile-
+            // time constant (the previous behavior) was wrong
+            // whenever the operand's static type was Any: the user's
+            // `Array.isArray(JSON.parse("[...]"))` would always
+            // return false despite being a real array at runtime.
+            let v = lower_expr(ctx, o)?;
             if is_array_expr(ctx, o) {
-                Ok(double_literal(f64::from_bits(crate::nanbox::TAG_TRUE)))
-            } else {
-                Ok(double_literal(f64::from_bits(crate::nanbox::TAG_FALSE)))
+                return Ok(double_literal(f64::from_bits(crate::nanbox::TAG_TRUE)));
+            }
+            if let Some(ty) = crate::type_analysis::static_type_of(ctx, o) {
+                // Definitively not an array: emit TAG_FALSE. Leaves
+                // numeric / string / boolean literals and known
+                // object-class instances on the fast path.
+                let definitely_not_array = matches!(
+                    ty,
+                    perry_types::Type::Number
+                        | perry_types::Type::Int32
+                        | perry_types::Type::String
+                        | perry_types::Type::Boolean
+                        | perry_types::Type::Null
+                        | perry_types::Type::Void
+                        | perry_types::Type::BigInt
+                        | perry_types::Type::Symbol
+                );
+                if definitely_not_array {
+                    return Ok(double_literal(f64::from_bits(crate::nanbox::TAG_FALSE)));
+                }
             }
+            // Indeterminate — dispatch to runtime.
+            Ok(ctx.block().call(DOUBLE, "js_array_is_array", &[(DOUBLE, &v)]))
         }
 
         // -------- new AggregateError(errors, message) --------