Skip to content

Commit f34391c

Browse files
feat: Deno-ESM target + grammar-v2 build-out (Refs #122) (#123)
* feat(codegen): Deno-ESM target — direct AST→ES module backend (Refs #122) Adds `lib/codegen_deno.ml`, a direct AffineScript-AST → ES-module transpiler, plus a `--deno-esm` flag (and `.deno.js` extension) wired into both compile dispatch paths in `bin/main.ml`. Why a direct transpiler, not a wasm-wrapping ESM shim (the handoff's original Phase 1): the motivating consumer (ubicity storage.ts / wasm-bridge.ts) is pure JS-value orchestration — it JSON-stringifies opaque objects, JSON-parses file contents back into JS objects, and returns those to JS callers. AffineScript's wasm ABI is i32-only (Codegen: params/ret all I32); arbitrary JS objects / JSON.stringify cannot cross it. There is essentially no computation that belongs in wasm, so source-to-source emission is the correct tool. Backend behaviour: - `pub fn`/`pub const`/`pub enum` -> `export` (ESM, no require). - `extern fn` -> direct host calls via a lowering table (Deno.*Sync / JSON / WebAssembly / path helpers), inlined so output is genuinely drop-in (no extra package to resolve). - struct + receiver-first free functions -> `export class`: constructor synthesised from the struct-returning fn (record literal -> `this.f = e`), methods from fns whose first param is the struct (receiver -> `this`), cross-method calls rewritten to `this.m(...)`. This is necessary because the current grammar accepts neither inherent `impl Type {}` nor a `self` expression (SELF_KW has no expression production — even stdlib/traits.affine fails to parse), so an "instance method" must be a free function taking the struct first. - Methods emitted `async`: the consumer surface is entirely async and callers `await`; `await` on a synchronously-returned value is valid JS, so the exact API is preserved with no async-extern ABI (#103 not required for this consumer; remains documented future work). - js_reserved trimmed to real ECMAScript reserved words (drops the Java-ism primitives Js_codegen carries: double/int/boolean/... are valid JS identifiers; mangling them would corrupt a consumer's required ESM export surface). Regression: tests/codegen-deno/class_basic.{affine,harness.mjs} + tools/run_codegen_deno_tests.sh + a CI step. The harness runs under Node 20 (CI has no Deno; Phase 1 fixtures are pure logic and the generated module only touches the Deno global lazily in unused helpers). `dune build` clean; `dune runtest` unchanged (the 2 pre-existing E2E Node-CJS `--vscode-extension` failures, #116/#117 territory, exist identically on pristine HEAD and are out of #122 scope). Refs #122, #35, #103. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(codegen-deno): statement-position return + control flow (Refs #122) The Deno-ESM backend inherited js_codegen's expression-only lowering of `return`/`if`/`match`/`try`: a statement-position `return e;` became `(() => { return e; })();`, whose value is discarded — so any function using an explicit `return` (or early-return in an `if`, or `return` inside a `for`/`while`) actually returned `undefined`. Phase 1 dodged it only because every fixture used `= expr;` bodies. Add a distinct statement-position lowering (gen_stmt_expr / gen_branch / gen_stmt_seq / gen_match_stmt / gen_try_stmt): in statement context `return` and control flow emit real JS statements; gen_expr keeps the IIFE form for genuine expression positions. StmtExpr, and loop trailing exprs, route through it. Regression: tests/codegen-deno/control_flow.{affine,harness.mjs} — while+trailing return, early return in if, return inside for+if, fallthrough. Full Deno-ESM suite green; Phase 1 unaffected. Refs #122. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(grammar): self expression + inherent impl blocks (Refs #122) Two pre-existing AffineScript grammar gaps blocked idiomatic OO and made even stdlib/traits.affine unparseable: 1. SELF_KW had no expression production. The receiver-param productions already bind a parameter named "self", but the body could never reference it — `self.field` / `self.method(x)` was always a parse error. Add `| SELF_KW { ExprVar (mk_ident "self" ...) }` to expr_primary; it resolves/typechecks as the ordinary "self" parameter binding (no resolver/typechecker change needed). 2. Inherent `impl Type { ... }` was a parse error at the `{`: the rule pre-committed to `impl_trait_ref? self_ty` where both alternatives start with an ident, an unresolved ambiguity. Restructure to parse one type_expr unconditionally, then an optional `FOR type_expr`; when `FOR` is present the leading type was the trait (trait_ref_of_type_expr recovers its name/args). Conflict-free; trait impls (`impl Trait for Type`) still parse identically. Verified: `impl Counter { fn bumped(ref self, by: Int) -> Int { self.start + by } }` compiles; trait-impl + receiver-first probes still compile; `dune runtest` unchanged (same 2 pre-existing E2E Node-CJS vscode failures, no new regressions); Deno-ESM suite green. Residual (separate, tracked): `Self` as a *type annotation* in trait default methods (stdlib/traits.affine:12 `ref other: Self`) — a narrow grammar/typecheck item, distinct from `self`-the-expression. Refs #122. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(typecheck): auto-deref ref/own/mut for struct field access (Refs #122) `fn(c: ref Point) = c.x` failed with "Field 'x' not found in type ref {x: Int, ...}": ExprField unified the receiver type directly with the expected record, so a TRef/TMut/TOwn wrapper made projection fail. Strip TRef/TMut/TOwn (via repr) before record projection — Rust-like auto-deref for field access. The borrow checker still governs aliasing separately; this only affects the *type* of the projected field, which is identical through a reference. Trait-method fallback path unchanged. This unblocks receiver-first methods that take `ref/own/mut self|recv` (the idiomatic, allocation-free form) under the Deno-ESM class synth. Regression: tests/codegen-deno/ref_fields.{affine,harness.mjs} — ref/own/mut receivers synthesised as Point methods, field access verified. `dune runtest` unchanged (same 2 pre-existing E2E Node-CJS vscode failures, 214 tests, zero new regressions). Deno-ESM suite green. Refs #122. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(grammar): qualified Type::Variant patterns in match arms (Refs #122) `pattern_primary` accepted only bare `upper_ident` / `upper_ident(p..)` constructors, never the qualified `Type::Variant` form. So every qualified match arm was a parse error — `match c { Color::Red => .. }`, recursive ADTs (`IntList::Cons(h, t) => ..`), and stdlib/traits.affine's `Ordering::Less => ..`. This blocked both `= match {..};` expression bodies and block-body matches. Add `upper_ident COLONCOLON upper_ident` and the payload form to pattern_primary. The type qualifier is discarded — PatCon carries only the variant name, exactly matching ExprVariant and the codegen tag dispatch (`scrut.tag === "Variant"`, payload at `.values[i]`), so the qualified and bare forms are interchangeable and consistent with the enum factory emission. Regression: tests/codegen-deno/match_enum.{affine,harness.mjs} — nullary qualified patterns in an expression-body match, plus recursive `len`/`sum` over a payload ADT (verified len=3, sum=60). `dune runtest` unchanged (same 2 pre-existing E2E Node-CJS vscode failures, 214 tests, zero new regressions). Deno-ESM suite green. Refs #122. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(stdlib/typecheck): C-proper string primitives + polymorphic ++ (Refs #122) Option-C enabler: make AffineScript-level string logic compile through the static pipeline + Deno-ESM backend, resting on a thin honest host-primitive layer instead of bespoke per-op externs. typecheck: - `++` (OpConcat) is now polymorphic over String and [T] — dispatched on the synthesised lhs type. Was String-only, so `acc ++ [x]` (how stdlib/string.affine's split/join accumulate) failed `Unify(Array, String)`. - `len` broadened from `Array[t]->Int` to `'a->Int` (stdlib/string calls it on String and arrays; matches the interpreter + the `.length` lowering). - Bound the honest string/char primitives: string_get/string_sub/ string_find/to_lowercase/to_uppercase/trim/parse_int/parse_float/ char_to_int/int_to_char/show (resolve.ml already knew them). codegen_deno: - Lower those primitives + `++` (shape-dispatched `__as_concat`, so array concat isn't string-coerced) to JS intrinsics. The lowering table now applies to ANY matching call head (not only declared externs) so AffineScript-level stdlib resolves... - ...EXCEPT a same-named user definition shadows the intrinsic (new ctx.local_fns guard) — fixes a user `fn len(xs: IntList)` being hijacked into `.length` (NaN). Net: ends_with/strip_suffix/find/int_to_string/string-&-array `++` all work as real AffineScript over the primitive layer (tests/codegen-deno/string_prims.{affine,harness.mjs}). Full Deno-ESM suite green (5 harnesses); `dune runtest` unchanged (same 2 pre-existing E2E Node-CJS vscode failures, 214 tests, zero new regressions). Note: stdlib/string.affine itself still has a separate Resolution error (angle-generic `Option<Char>` + unimported Some/None) — that file's own issue, not the primitive layer; addressed in the consumer migration. Refs #122. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(codegen-deno/typecheck): consumer-migration primitives + ++ disambiguation (Refs #122) Primitives the ubicity#30 storage migration needs, added to the honest host-primitive layer (each a genuine host capability, not smuggled logic): jsonNull (`null`), jsonGet/jsonGetStr (opaque object/index read — the data boundary for arbitrary host JSON), orDefault (`x ?? d` — preserves a JS default parameter), kbString (`(n/1024).toFixed(2)` — runtime number formatting), panic (`throw new Error(...)`). Mirrored in stdlib/Deno.affine. typecheck: `++` disambiguation hardened — when the lhs type is not yet determined (e.g. `let mut acc = []` then `acc ++ [x]`), disambiguate on the rhs instead of defaulting to String (which mis-typed array accumulation as String). Regression-free: 5 Deno-ESM harnesses green; `dune runtest` unchanged (same 2 pre-existing E2E Node-CJS vscode failures, 214 tests). Refs #122 (consumer enablement for ubicity#30). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent bab0051 commit f34391c

19 files changed

Lines changed: 1497 additions & 15 deletions

.github/workflows/ci.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,12 @@ jobs:
4040
- name: Run codegen WASM tests
4141
run: opam exec -- ./tools/run_codegen_wasm_tests.sh
4242

43+
- name: Run codegen Deno-ESM tests (issue #122)
44+
# Compiles tests/codegen-deno/*.affine with the --deno-esm backend
45+
# and runs the *.harness.mjs under Node (CI has Node 20, not Deno;
46+
# the Phase 1 fixtures are pure logic so Node ESM exercises them).
47+
run: opam exec -- ./tools/run_codegen_deno_tests.sh
48+
4349
- name: Run face-transformer regression tests
4450
run: opam exec -- ./tools/run_face_transformer_tests.sh
4551

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,3 +89,9 @@ htmlcov/
8989
*.bak
9090
*.wasm
9191
/a.out
92+
93+
# issue #122: generated Deno-ESM regression outputs (compiled from the
94+
# committed *.affine fixtures by tools/run_codegen_deno_tests.sh).
95+
/tests/codegen-deno/*.deno.js
96+
# Local-only build workaround (see file header); never committed.
97+
/dune-workspace

bin/main.ml

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -478,7 +478,7 @@ let repl_cmd_fn () =
478478
compilation errors. With [--wasm-gc], targets the WebAssembly GC
479479
proposal instead of WASM 1.0 linear memory. *)
480480
let compile_file face json wasm_gc vscode_ext vscode_adapter vscode_no_lc
481-
path output =
481+
deno_esm path output =
482482
let face = resolve_face ~quiet:json face path in
483483
if json then begin
484484
let diags = ref [] in
@@ -512,8 +512,9 @@ let compile_file face json wasm_gc vscode_ext vscode_adapter vscode_no_lc
512512
use the original [prog] because they handle imports natively
513513
via Codegen.gen_imports / the import section. *)
514514
let flat_prog = Affinescript.Module_loader.flatten_imports loader prog in
515+
let is_deno = deno_esm || Filename.check_suffix output ".deno.js" in
515516
let is_julia = Filename.check_suffix output ".jl" in
516-
let is_js = Filename.check_suffix output ".js" in
517+
let is_js = (not is_deno) && Filename.check_suffix output ".js" in
517518
let is_c = Filename.check_suffix output ".c" in
518519
let is_wgsl = Filename.check_suffix output ".wgsl" in
519520
let is_faust = Filename.check_suffix output ".dsp" in
@@ -534,7 +535,17 @@ let compile_file face json wasm_gc vscode_ext vscode_adapter vscode_no_lc
534535
let is_why3 = Filename.check_suffix output ".mlw" in
535536
let is_lean = Filename.check_suffix output ".lean" in
536537
let is_spirv = Filename.check_suffix output ".spv" in
537-
if is_julia then begin
538+
if is_deno then begin
539+
match Affinescript.Codegen_deno.codegen_deno flat_prog resolve_ctx.symbols with
540+
| Error msg ->
541+
add { severity = Error; code = "E0824";
542+
message = Printf.sprintf "Deno-ESM codegen error: %s" msg;
543+
span = Affinescript.Span.dummy; help = None; labels = [] }
544+
| Ok esm_code ->
545+
let oc = open_out output in
546+
output_string oc esm_code;
547+
close_out oc
548+
end else if is_julia then begin
538549
match Affinescript.Julia_codegen.codegen_julia flat_prog resolve_ctx.symbols with
539550
| Error msg ->
540551
add { severity = Error; code = "E0800";
@@ -725,8 +736,9 @@ let compile_file face json wasm_gc vscode_ext vscode_adapter vscode_no_lc
725736
cross-module imports for backends that don't have native
726737
module-system support. Wasm/Wasm-GC keep the original [prog]. *)
727738
let flat_prog = Affinescript.Module_loader.flatten_imports loader prog in
739+
let is_deno = deno_esm || Filename.check_suffix output ".deno.js" in
728740
let is_julia = Filename.check_suffix output ".jl" in
729-
let is_js = Filename.check_suffix output ".js" in
741+
let is_js = (not is_deno) && Filename.check_suffix output ".js" in
730742
let is_c = Filename.check_suffix output ".c" in
731743
let is_wgsl = Filename.check_suffix output ".wgsl" in
732744
let is_faust = Filename.check_suffix output ".dsp" in
@@ -747,7 +759,18 @@ let compile_file face json wasm_gc vscode_ext vscode_adapter vscode_no_lc
747759
let is_why3 = Filename.check_suffix output ".mlw" in
748760
let is_lean = Filename.check_suffix output ".lean" in
749761
let is_spirv = Filename.check_suffix output ".spv" in
750-
if is_julia then
762+
if is_deno then
763+
(match Affinescript.Codegen_deno.codegen_deno flat_prog resolve_ctx.symbols with
764+
| Error e ->
765+
Format.eprintf "@[<v>Deno-ESM codegen error: %s@]@." e;
766+
`Error (false, "Deno-ESM codegen error")
767+
| Ok esm_code ->
768+
let oc = open_out output in
769+
output_string oc esm_code;
770+
close_out oc;
771+
Format.printf "Compiled %s -> %s (Deno-ESM)@." path output;
772+
`Ok ())
773+
else if is_julia then
751774
(match Affinescript.Julia_codegen.codegen_julia flat_prog resolve_ctx.symbols with
752775
| Error e ->
753776
Format.eprintf "@[<v>Julia codegen error: %s@]@." e;
@@ -1178,6 +1201,20 @@ let vscode_no_lc_arg =
11781201
dependency for extensions that ship no language client; the \
11791202
wiring passes null in its place.")
11801203

1204+
(* Issue #122: --deno-esm. Selects the direct AST -> ES-module backend
1205+
({!Affinescript.Codegen_deno}) regardless of output extension, so a
1206+
drop-in `.js` ES module can be produced (e.g. `-o src/storage.js
1207+
--deno-esm`). A `.deno.js` output extension also routes here without
1208+
the flag, as a convenience for the test corpus / ad-hoc use. *)
1209+
let deno_esm_arg =
1210+
Arg.(value & flag & info ["deno-esm"]
1211+
~doc:"Emit a standalone Deno/Node ES module directly from the AST \
1212+
(issue #122): `export class` for struct+impl, `export` for \
1213+
public fns/consts, and `extern fn` lowered to direct host \
1214+
calls (Deno.*Sync / JSON / WebAssembly). No wasm, no require, \
1215+
no handle table — the output is a drop-in importable ESM. A \
1216+
`.deno.js` output extension selects this backend implicitly.")
1217+
11811218
(** Shared --face flag: select the parser surface-syntax face. *)
11821219
let face_arg =
11831220
let faces = Arg.enum [
@@ -1524,7 +1561,7 @@ let compile_cmd =
15241561
let doc = "Compile a file to WebAssembly (1.0 or GC proposal), Julia (.jl), JavaScript (.js), C (.c), a WGSL compute kernel (.wgsl), a Faust DSP program (.dsp), or an ONNX model (.onnx)" in
15251562
let info = Cmd.info "compile" ~doc in
15261563
Cmd.v info Term.(ret (const compile_file $ face_arg $ json_arg $ wasm_gc_arg
1527-
$ vscode_ext_arg $ vscode_adapter_arg $ vscode_no_lc_arg
1564+
$ vscode_ext_arg $ vscode_adapter_arg $ vscode_no_lc_arg $ deno_esm_arg
15281565
$ path_arg $ output_arg))
15291566

15301567
let fmt_cmd =

0 commit comments

Comments
 (0)