Skip to content

Commit 7d32728

Browse files
tree-sitter: v1 grammar — parses examples/01-single-module.twasm end-to-end (zero ERROR nodes) (#62)
## Summary Extends the Phase 0 tree-sitter scaffold (PR #58, region-decls only) to **v1 grammar** that parses `examples/01-single-module.twasm` **end-to-end with zero parse errors**. ## Verification (local) ```bash cd tools/tree-sitter-twasm npm install tree-sitter generate # no conflicts; parser.c emitted tree-sitter parse ../../examples/01-single-module.twasm # → 0 ERROR / 0 MISSING nodes, exit 0, full AST for all 134 lines ``` ## What v1 now covers (beyond region declarations) - **`memory_decl`** — `memory Name { initial: N; maximum: N; place X at N; }` - **`function_decl`** — `fn name(params) -> ret effects { ... } { body }` - `parameter_list` with `region_handle_type` (`&`, `&mut`, `own`) - `effects_clause` — `Read / Write / Alloc / Free / ReadRegion(R) / WriteRegion(R)` - `return_type` including `void` - **Statements** — `region.get`, `region.set`, `region.scan` (with `where` + binding + body), `let [mut] name [: type] = expr;`, assignment, `if`/`else`, `return` - **Expressions** — literals (int/float/bool/null), `identifier_expr`, `region_var` (`$x`), `binary_expr` (full operator set with precedence), `unary_expr`, `is_null_expr`, `paren_expr` - **Field paths** — including nested access (`.pos.x`) - **`region_target`** — bare-identifier form for post-null-check references Precedence chain: `['unary', 'mul', 'add', 'cmp', 'and', 'or']` with unary at top. ## Still deferred (next Track A PR) - Imports / exports - Invariant declarations + proof statements - `const` declarations - Block-expression `if ... yield` - Match on union regions - L13–L16 surface (`isolated`, `session`, `capability`, `choreography`) - L11/L12 surface (`cost_bound`, `fresh`, `version_of`, `region.sync`) - Lifetime annotations on function decls - Striated region layout By production count v1 covers ~60–70% of `spec/grammar.ebnf`. ## Files | File | Change | |---|---| | `grammar.js` | ~80 → ~270 lines | | `corpus/functions.txt` | new — 6 corpus tests covering memory_decl, if/else, is_null guard with bare-ident target, scan, nested field paths | | `queries/highlights.scm` | extended with function/memory/if/effect/handle-mode/literal/builtin categories | | `README.md` | updated to reflect v1 scope | | `tree-sitter.json` | new — tree-sitter-cli metadata | | `package.json` | minor metadata tweaks | ## CI integration Still deferred to the next Track A PR. When the grammar reaches full EBNF parity, the install-and-test cost (~30MB tree-sitter-cli toolchain on every PR run) is justified by the regression value. Today the 6-test corpus + the local parse-of-example-01 covers the regression risk well enough. ## Phase 0 impact This PR plus the existing Track A scaffold (PR #58) moves the Phase 0 §Track A checklist from "scaffold landed" to "scaffold + v1 grammar landed, parses simplest example end-to-end". Remaining Track A items (full EBNF parity, Idris2 parser, ReScript cut, codegen v0) are still ahead. ## Test plan - [x] `tree-sitter generate` succeeds (no conflicts) - [x] `tree-sitter parse examples/01-single-module.twasm` returns 0 ERROR / 0 MISSING nodes - [ ] Corpus tests pass on a CI runner with tree-sitter-cli installed (deferred to next PR with CI wiring) - [ ] Smoke test, structural E2E, claim-envelope, property, security-envelope, proof regression — none touch tools/tree-sitter-twasm/ so no regressions expected --- _Generated by [Claude Code](https://claude.ai/code/session_01ExgUTJmU5UQQNLKynwxDjm)_
2 parents f53e693 + b9f5523 commit 7d32728

6 files changed

Lines changed: 576 additions & 69 deletions

File tree

tools/tree-sitter-twasm/README.md

Lines changed: 37 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -2,40 +2,45 @@
22

33
Tree-sitter grammar for `.twasm` (typed-wasm surface syntax).
44

5-
## Status: scaffold (Phase 0 kickoff)
5+
## Status: v1 grammar (Phase 0 grammar extension)
66

7-
This is the **scaffold** for the tree-sitter grammar that will eventually
8-
back the Idris2 parser, the LSP, Linguist registration, and editor
9-
plugins. See `docs/PRODUCTION-PATH.adoc` §Phase 0 for the strategic
10-
context.
7+
The grammar now covers enough of `examples/01-single-module.twasm` to
8+
parse the full file: region declarations, memory declarations,
9+
function declarations (parameters / return types / effects), and the
10+
statement and expression forms used in example 01.
1111

1212
### What works today
1313

14-
- Tree-sitter scaffold + corpus harness (`tree-sitter test` runs)
15-
- `grammar.js` v0 — covers **region declarations only**:
16-
- `region Name { field: type; ... }`
17-
- `region Name[N] { ... }` (array quantifier)
18-
- Primitive types (`i32`, `f32`, `u8`, `bool`, etc.)
19-
- Nested region references (`@OtherRegion`)
20-
- Optional types (`opt<@T>`)
21-
- Fixed-size array fields (`u8[24]`)
22-
- `align N;` clauses
23-
- `where` field constraints (range form)
24-
- Single-line `//` comments
25-
26-
### What does NOT work yet (deferred to subsequent Track A PRs)
27-
28-
- Function declarations (`fn name(...) { ... }`)
29-
- Memory declarations, imports/exports
30-
- Statements: `region.get`, `region.set`, `let`, `if`, `while`
31-
- Effects, lifetime, cost-bound, freshness clauses
32-
- L13–L16 surface syntax (isolated modules, sessions, capabilities, choreography)
33-
- Tropical / epistemic extensions (L11/L12)
34-
- The full v1.5 surface — see `spec/grammar.ebnf` (~695 lines) for what's still ahead
35-
36-
This deliberate v0 scope covers enough of `examples/01-single-module.twasm`
37-
to exercise the toolchain end-to-end without overcommitting to the
38-
multi-month full-grammar port.
14+
- **Region declarations**`region Name { ... }`, array quantifiers `[N]`,
15+
primitives, region refs (`@T`), `opt<T>`, fixed-size arrays (`u8[24]`),
16+
`align N;`, `where` range constraints
17+
- **Memory declarations**`memory Name { initial: N; maximum: N; place X at N; ... }`
18+
- **Function declarations**`fn name(params) -> ret effects { ... } { body }`
19+
- parameter modes: `&region<R>`, `&mut region<R>`, `own region<R>`, and bare typed parameters
20+
- effects: `Read`, `Write`, `Alloc`, `Free`, `ReadRegion(R)`, `WriteRegion(R)`
21+
- **Statements**`region.get`, `region.set`, `region.scan`, `let [mut] name [: type] = expr;`,
22+
assignment, `if ... else`, `return`
23+
- **Expressions** — literals (`int`, `float`, `true`, `false`, `null`),
24+
identifiers, `$region_var`, binary ops (`+ - * / %`, `== != < > <= >=`, `&& ||`),
25+
unary (`-`, `!`), `is_null(expr)`, parenthesised
26+
- **Field paths** — single and nested (`.pos.x`)
27+
- **`//` line comments**
28+
29+
### What does NOT work yet (next Track A PR)
30+
31+
- Imports / exports (`import region X from "mod"`, `export region X`)
32+
- Invariant declarations and `proof` statements
33+
- `const` declarations
34+
- Block-expression `if ... yield`
35+
- Match on union regions
36+
- L13–L16 surface (`isolated`, `session`, `capability`, `choreography`)
37+
- L11/L12 surface (`cost_bound`, `fresh`, `version_of`, `region.sync`)
38+
- Lifetime annotations on function decls
39+
- `striated` region layout
40+
41+
Scope-wise the v1 grammar covers maybe 60–70% of `spec/grammar.ebnf`
42+
by production count. The remainder is sequenced for the next Track A
43+
PR.
3944

4045
## Why in-tree first
4146

@@ -66,8 +71,8 @@ regenerate locally via `tree-sitter generate`.
6671

6772
| Phase | This grammar's role |
6873
|-------|---------------------|
69-
| **0** (now) | Scaffold + region-decl coverage; proves toolchain works |
70-
| **1** | Extend to full `spec/grammar.ebnf` parity; back the Idris2 parser |
74+
| **0** (current) | v1 grammar — parses `examples/01-single-module.twasm` fully |
75+
| **1** | Extend to remaining `spec/grammar.ebnf` productions (imports, L11-L16, match, proof); back the Idris2 parser |
7176
| **4** | Extract to `hyperpolymath/tree-sitter-twasm`; publish to npm; submit to Linguist |
7277

7378
Tracked under issue [#48 (Phase 0)](https://github.com/hyperpolymath/typed-wasm/issues/48).
Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
============================================
2+
Simple function with effects and return type
3+
============================================
4+
5+
fn get_player_hp(players: &region<Players>, idx: i32) -> i32
6+
effects { ReadRegion(Players) }
7+
{
8+
region.get $players[idx] .hp -> hp;
9+
return hp;
10+
}
11+
12+
---
13+
14+
(source_file
15+
(function_decl
16+
name: (identifier)
17+
(parameter_list
18+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
19+
(parameter name: (identifier) type: (primitive_type)))
20+
(return_type (primitive_type))
21+
(effects_clause (effect (identifier)))
22+
(region_get_stmt
23+
(region_target (identifier) (identifier_expr (identifier)))
24+
(field_path (identifier))
25+
binding: (identifier))
26+
(return_stmt (identifier_expr (identifier)))))
27+
28+
29+
============================================
30+
Memory declaration with placements
31+
============================================
32+
33+
memory game_memory {
34+
initial: 64;
35+
maximum: 256;
36+
place Players at 0;
37+
place Enemies at 4800;
38+
}
39+
40+
---
41+
42+
(source_file
43+
(memory_decl
44+
name: (identifier)
45+
(initial_clause)
46+
(maximum_clause)
47+
(place_clause (identifier))
48+
(place_clause (identifier))))
49+
50+
51+
============================================
52+
If/else with region.set in both branches
53+
============================================
54+
55+
fn damage_player(players: &mut region<Players>, idx: i32, amount: i32)
56+
effects { ReadRegion(Players), WriteRegion(Players) }
57+
{
58+
region.get $players[idx] .hp -> current_hp;
59+
let new_hp: i32 = current_hp - amount;
60+
61+
if new_hp < 0 {
62+
region.set $players[idx] .hp, 0;
63+
} else {
64+
region.set $players[idx] .hp, new_hp;
65+
}
66+
}
67+
68+
---
69+
70+
(source_file
71+
(function_decl
72+
name: (identifier)
73+
(parameter_list
74+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
75+
(parameter name: (identifier) type: (primitive_type))
76+
(parameter name: (identifier) type: (primitive_type)))
77+
(effects_clause (effect (identifier)) (effect (identifier)))
78+
(region_get_stmt
79+
(region_target (identifier) (identifier_expr (identifier)))
80+
(field_path (identifier))
81+
binding: (identifier))
82+
(let_stmt
83+
name: (identifier)
84+
(primitive_type)
85+
(binary_expr (identifier_expr (identifier)) (identifier_expr (identifier))))
86+
(if_stmt
87+
(binary_expr (identifier_expr (identifier)) (literal (integer_literal)))
88+
(region_set_stmt
89+
(region_target (identifier) (identifier_expr (identifier)))
90+
(field_path (identifier))
91+
(literal (integer_literal)))
92+
(region_set_stmt
93+
(region_target (identifier) (identifier_expr (identifier)))
94+
(field_path (identifier))
95+
(identifier_expr (identifier))))))
96+
97+
98+
============================================
99+
Null check via is_null and bare-identifier region_target
100+
============================================
101+
102+
fn get_enemy_target_hp(enemies: &region<Enemies>, enemy_idx: i32) -> i32
103+
effects { ReadRegion(Enemies) }
104+
{
105+
region.get $enemies[enemy_idx] .target -> maybe_target;
106+
107+
if is_null(maybe_target) {
108+
return -1;
109+
}
110+
111+
region.get maybe_target .hp -> target_hp;
112+
return target_hp;
113+
}
114+
115+
---
116+
117+
(source_file
118+
(function_decl
119+
name: (identifier)
120+
(parameter_list
121+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
122+
(parameter name: (identifier) type: (primitive_type)))
123+
(return_type (primitive_type))
124+
(effects_clause (effect (identifier)))
125+
(region_get_stmt
126+
(region_target (identifier) (identifier_expr (identifier)))
127+
(field_path (identifier))
128+
binding: (identifier))
129+
(if_stmt
130+
(is_null_expr (identifier_expr (identifier)))
131+
(return_stmt (literal (integer_literal))))
132+
(region_get_stmt
133+
(region_target (identifier))
134+
(field_path (identifier))
135+
binding: (identifier))
136+
(return_stmt (identifier_expr (identifier)))))
137+
138+
139+
============================================
140+
region.scan with where + binding + body
141+
============================================
142+
143+
fn count_active_enemies(enemies: &region<Enemies>) -> i32
144+
effects { ReadRegion(Enemies) }
145+
{
146+
let mut count: i32 = 0;
147+
148+
region.scan $enemies where is_active == true -> |enemy| {
149+
count = count + 1;
150+
}
151+
152+
return count;
153+
}
154+
155+
---
156+
157+
(source_file
158+
(function_decl
159+
name: (identifier)
160+
(parameter_list
161+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier))))
162+
(return_type (primitive_type))
163+
(effects_clause (effect (identifier)))
164+
(let_stmt name: (identifier) (primitive_type) (literal (integer_literal)))
165+
(region_scan_stmt
166+
(region_target (identifier))
167+
(binary_expr (identifier_expr (identifier)) (literal))
168+
binding: (identifier)
169+
(assign_stmt (identifier) (binary_expr (identifier_expr (identifier)) (literal (integer_literal)))))
170+
(return_stmt (identifier_expr (identifier)))))
171+
172+
173+
============================================
174+
Nested field path
175+
============================================
176+
177+
fn move_player(players: &mut region<Players>, idx: i32, dx: f32, dy: f32)
178+
effects { ReadRegion(Players), WriteRegion(Players) }
179+
{
180+
region.get $players[idx] .pos.x -> old_x;
181+
region.get $players[idx] .pos.y -> old_y;
182+
183+
region.set $players[idx] .pos.x, old_x + dx;
184+
region.set $players[idx] .pos.y, old_y + dy;
185+
}
186+
187+
---
188+
189+
(source_file
190+
(function_decl
191+
name: (identifier)
192+
(parameter_list
193+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
194+
(parameter name: (identifier) type: (primitive_type))
195+
(parameter name: (identifier) type: (primitive_type))
196+
(parameter name: (identifier) type: (primitive_type)))
197+
(effects_clause (effect (identifier)) (effect (identifier)))
198+
(region_get_stmt
199+
(region_target (identifier) (identifier_expr (identifier)))
200+
(field_path (identifier) (identifier))
201+
binding: (identifier))
202+
(region_get_stmt
203+
(region_target (identifier) (identifier_expr (identifier)))
204+
(field_path (identifier) (identifier))
205+
binding: (identifier))
206+
(region_set_stmt
207+
(region_target (identifier) (identifier_expr (identifier)))
208+
(field_path (identifier) (identifier))
209+
(binary_expr (identifier_expr (identifier)) (identifier_expr (identifier))))
210+
(region_set_stmt
211+
(region_target (identifier) (identifier_expr (identifier)))
212+
(field_path (identifier) (identifier))
213+
(binary_expr (identifier_expr (identifier)) (identifier_expr (identifier))))))

0 commit comments

Comments
 (0)