fix: graceful fallback in cvt_smloc for invalid SMLoc#17
fix: graceful fallback in cvt_smloc for invalid SMLoc#17YevheniiKotyrlo wants to merge 1 commit into
Conversation
Hermes can hand fft a null-pointer SMLoc for synthesized nodes (error recovery, EOF tokens, certain Flow constructs). The assert in cvt_smloc aborts the process for any such input — `hparser::tests::test1` (a 35- char valid JS string) reproduces the panic on Windows on `upstream/main`, and the Vite-plugin adapter hits it during React Native bundling. Replace the assert with an early return that uses the established `SourceLoc::invalid()` sentinel — already used as the initial `range.end` value in `generated_cvt.rs`, in the `parse_with_flags` error path, and in `crates/fft_ast/src/node_child.rs`. Mirrors the existing fallback that `generated_cvt.rs` already applies to `range.end` via `is_empty()`, making both ends of a SourceRange uniformly degrade when source location is missing. No behavior change for valid input — the early return only affects the previously-panicking path. The cache, find_line, and make_source_loc paths are byte-identical for any valid SMLoc.
|
Update on the diagnosis: while investigating the downstream Repro and full diagnosis with stack-level traces is in the new PRs:
With #18 applied, both the original repro from #16 ( I'll leave the disposition of this PR to your judgement — happy with either path:
Either way, #18 is the structural root-cause fix and the one that actually unblocks Windows users. |
…t via build-time patch file Mirror the same change being upstreamed at facebook/hermes#2012: `__declspec(empty_bases)` on `simple_ilist` so MSVC actually applies Empty Base Optimization across the class's two empty bases. Without the attribute, MSVC pads each empty base with one byte and pointer-aligns the result, shifting the embedded `Sentinel` member from offset 0 to offset 8. The Rust FFI in `crates/hermes/src/parser/node.rs` is one of the callers that depends on the coincident layout: it wraps a `*const NodeList` (`= *const simple_ilist<Node>`) and iterates by reading `next` pointers, comparing each one against the head pointer it was given. On Linux/Clang and macOS the comparison succeeds — `Sentinel` is at offset 0, so `&list == &list.Sentinel` — and iteration terminates. On Windows MSVC the C++ side stores `&list.Sentinel` (offset 8) in the linked-list pointers while Rust received `&list` (offset 0); the comparison never matches, the iterator walks past the end of the list, dereferences the sentinel as if it were a real `Node`, and trips a `STATUS_ACCESS_VIOLATION 0xC0000005` on the first field load. Reproducible in `cargo test --release -p fft --lib hparser::tests::test1` on Windows (35-character JS suffices because it forces the FFI to iterate the function's params `simple_ilist`). Approach: a unified-diff patch file lives under `patches/` and the two cmake-driven build scripts apply it to the bundled Hermes submodule via `git apply` before configuring. The hook is: - **Target-gated.** Skipped entirely when `is_msvc_target()` is false. GCC/Clang already collapse the empty bases without the attribute, so Linux and macOS builds are unaffected. - **Idempotent.** Returns early when `__declspec(empty_bases) simple_ilist` is already present in the file — true after the first run in a single build, and also true once the Hermes submodule is bumped past facebook/hermes#2012 so the patch becomes unnecessary and can be removed at the maintainer's leisure. - **Loud on conflict.** If `git apply` fails (Hermes refactored `simple_ilist.h` and the patch's context lines no longer match), the build script panics with a message that names the patch file, surfaces git's stderr, and points at the two resolution paths: drop the patch when upstream lands, or regenerate it. Silently skipping would produce an unpatched binary that AVs at runtime — worse than a build error. Validated end-to-end on Windows (host) and on Linux WSL Ubuntu 22.04 with Rust 1.88.0: - `cargo test --release -p fft --lib hparser::tests::test1` passes on Windows (was `STATUS_ACCESS_VIOLATION` before). - `cargo test --workspace --release` clean on Windows. - Full workspace test clean on Linux (the patcher is gated and correctly skipped — verified by greping the submodule file after the build). - `cargo fmt --all --check` clean. - Re-running the build after the patch already landed exits the hook in <1ms (idempotency check). - Simulated Hermes-bump-with-conflict (inserted a comment ahead of the `simple_ilist` template) panics with the documented message and exit code 101, never producing a binary. Closes-by: should make jbroma#17's `cvt_smloc` graceful fallback unnecessary once this lands; the original `assert!(loc.is_valid(), ...)` is correct because Hermes does not produce null `SMLoc`s in this code path. Resolves the runtime AV reported in jbroma#16.
…t via build-time patch file Mirror the same change being upstreamed at facebook/hermes#2012: `__declspec(empty_bases)` on `simple_ilist` so MSVC actually applies Empty Base Optimization across the class's two empty bases. Without the attribute, MSVC pads each empty base with one byte and pointer-aligns the result, shifting the embedded `Sentinel` member from offset 0 to offset 8. The Rust FFI in `crates/hermes/src/parser/node.rs` is one of the callers that depends on the coincident layout: it wraps a `*const NodeList` (`= *const simple_ilist<Node>`) and iterates by reading `next` pointers, comparing each one against the head pointer it was given. On Linux/Clang and macOS the comparison succeeds — `Sentinel` is at offset 0, so `&list == &list.Sentinel` — and iteration terminates. On Windows MSVC the C++ side stores `&list.Sentinel` (offset 8) in the linked-list pointers while Rust received `&list` (offset 0); the comparison never matches, the iterator walks past the end of the list, dereferences the sentinel as if it were a real `Node`, and trips a `STATUS_ACCESS_VIOLATION 0xC0000005` on the first field load. Reproducible in `cargo test --release -p fft --lib hparser::tests::test1` on Windows (35-character JS suffices because it forces the FFI to iterate the function's params `simple_ilist`). Approach: a unified-diff patch file lives under `patches/` and the two cmake-driven build scripts apply it to the bundled Hermes submodule via `git apply` before configuring. The hook is: - **Target-gated.** Skipped entirely when `is_msvc_target()` is false. GCC/Clang already collapse the empty bases without the attribute, so Linux and macOS builds are unaffected. - **Idempotent.** Returns early when `__declspec(empty_bases) simple_ilist` is already present in the file — true after the first run in a single build, and also true once the Hermes submodule is bumped past facebook/hermes#2012 so the patch becomes unnecessary and can be removed at the maintainer's leisure. - **Race-safe.** Cargo runs `crates/hermes/build.rs` and `crates/fft_support/build.rs` in parallel, so both can pass the idempotency check while a sibling is still mid-`git apply`. If our own `git apply` then fails, we re-read the file: if the attribute is now present, we skip silently (sibling won the race); only a real conflict still panics. - **Loud on conflict.** If `git apply` fails for a non-race reason (Hermes refactored `simple_ilist.h` and the patch's context lines no longer match), the build script panics with a message that names the patch file, surfaces git's stderr, and points at the two resolution paths: drop the patch when upstream lands, or regenerate it. Silently skipping would produce an unpatched binary that AVs at runtime — worse than a build error. Validated end-to-end on Windows MSVC (host) and Linux WSL Ubuntu 22.04 with Rust 1.88.0, plus eight edge-case scenarios: - Clean apply on Windows: passes. - Idempotent re-run: passes in <10s, no re-application. - Linux: workspace passes; patcher correctly skips (verified by inspecting the submodule file post-build). - Hermes-bump-with-upstream-fix scenario (pre-applied attribute): builds clean, no re-application. - Patch file missing: clean panic with "was patches/ pruned?". - Submodule target file missing: clean panic with "reading ... failed". - Patch file corrupt (not a valid diff): clean panic surfacing git's "No valid patches in input". - Partial pre-application (comment present but attribute absent): clean panic with "patch does not apply". - Three back-to-back parallel clean builds: all pass; race-safety holds under the parallel invocation. `cargo fmt --all --check` clean. End-to-end: rebuilt napi binding plugged into onejs/one's Vite-native bundler returns a complete ~5 MB RN bundle in ~1.2 s on Windows where it previously segfaulted on the first request. Closes-by: should make jbroma#17's `cvt_smloc` graceful fallback unnecessary once this lands; the original `assert!(loc.is_valid(), ...)` is correct because Hermes does not produce null `SMLoc`s in this code path. Resolves the runtime AV reported in jbroma#16.
|
FYI — the upstream architectural fix landed: This further reduces the defensive value of this PR — the original Disposition still your call:
No rush from my end. |
Important
Updated diagnosis (2026-05-08) — the rationale below ("Hermes can hand us an invalid SMLoc for synthesized nodes") is wrong. Hermes does not produce null
SMLocs in this path; the originalassert!(loc.is_valid(), …)atconvert.rs:120is correct. The apparent nullSMLocs were a side effect of an MSVC empty-base-optimization layout bug in the bundled llvhsimple_ilist. The real structural fix is at#18(build-time patch, open) andfacebook/hermes#2012— MERGED 2026-05-11 atfacebook/hermes@768bab2a(upstream architectural fix). This PR's graceful fallback is at most defense in depth — no longer required for correctness now that the EBO mismatch is fixed at the source and#18ships the same diff as a build-time patch. Disposition is the maintainer's call: merge as defense, or close superseded by#18(and#2012upstream). Full corrected analysis in this comment. Original PR body preserved below.Summary
cvt_smlocaborts the process viaassert!(loc.is_valid(), …)whenever Hermes hands fft anSMLocwith a null pointer. This panics on Windows for at least one synthesized node when bundling a React Native source tree (fast-flow-transform-win32-x64-msvc@0.0.3, via vxrn's Vite plugin adapter insideonejs/one's native bundler). Reproducible across machines on Windows; not reproducible on Linux Docker.Filed and reproduced in detail at #16.
Fix
Replace the
assert!with an early-return that uses the establishedSourceLoc::invalid()sentinel:pub fn cvt_smloc(&mut self, loc: SMLoc) -> ast::SourceLoc { - assert!( - loc.is_valid(), - "All source locations from Hermes parser must be valid" - ); + // Hermes can hand us an invalid (null-pointer) SMLoc for synthesized + // nodes (error recovery, EOF tokens, some Flow constructs). Mirror + // generated_cvt's existing fallback for `range.end` rather than + // aborting the process. + if !loc.is_valid() { + return ast::SourceLoc::invalid(); + }Why this is structural rather than a band-aid:
SourceLoc::invalid()({ line: 0, col: 0 }, defined incrates/fft_support/src/source_manager.rs) is already the established sentinel in this codebase. It's used as the initialrange.endvalue ingenerated_cvt.rsbefore it's filled in, as the fallback inparse_with_flagswhen there's no first-error coordinate, and atcrates/fft_ast/src/node_child.rs:64-65. This patch just letscvt_smlocreturn the same sentinel for null-pointer input.generated_cvt.rs:range.startis passed tocvt_smlocunconditionally, whilerange.endis guarded byis_empty(). With this patch, both ends uniformly fall back toinvalid()when source location is missing.find_linepath, andmake_source_locpath all stay byte-identical.gen_js, sourcemap emission, error reporting) already acceptinvalid()as a no-position marker. Worst case for an affected node: the line/col in error messages becomes0:0rather than aborting the bundler — far better than the current behavior.Why now (not earlier)
The same
assert!exists upstream in Juno (hermes/unsupported/juno/crates/juno/src/hparser/convert.rs) — fft inherited it on the 2026-03-05 rename commit. For Juno (a CLI tool with controlled input) the assert is reasonable. For fft (a library invoked by bundler adapters with arbitrary user input), the same assert is a denial-of-service surface. Worth softening here even if Juno keeps it stricter.Verification
hparser::tests::test1(parse("function foo(p1) { var x = (10 + p1); }")) hits the existing panic on Windows onupstream/main— see Rust panic in cvt_smloc on Windows: "All source locations from Hermes parser must be valid" #16 for repro detail.cvt_smloc. (It then hits a separate Windows-only access violation downstream —STATUS_ACCESS_VIOLATION 0xc0000005— which the panic was previously short-circuiting. Different bug; outside this PR's scope.)facebook/hermessubmodule) and dropped it into the original consuming project (OneJS Vite-native bundler withvite.config.ts: native.bundler: 'vite'). The cvt_smloc panic is gone from the bundling pipeline. First request emits a complete ~609 KB UMD bundle.Test plan
cargo fmt --all --checkclean (workspace)cargo test --workspace— clean on Linux CI (no null SMLoc → no early-return code path → behavior byte-identical for valid input). On Windows, the lib test hits a pre-existing access violation that this PR does not address (the same Windows-only downstream bug noted above, present onupstream/mainonce you remove the cvt_smloc panic).pnpm check/pnpm test/pnpm e2e— Linux CI runs these; structural soundness on the patch is high (single-function early-return, no API change, no new code paths for valid SMLoc).Notes
This PR is intentionally minimal — single-function, ~5 lines. Larger questions (why Hermes produces null
SMLoconly on Windows; whether the C++ source manager has a path-encoding bug) need a debug-build binding and a native debugger, and feel like they belong to a separate investigation rather than blocking this fix.Closes #16