Skip to content

Commit b9f5523

Browse files
committed
tree-sitter: v1 grammar — parses examples/01-single-module.twasm end-to-end
Extends the Phase 0 scaffold (PR #58, region-decls only) to v1 coverage. The 134-line examples/01-single-module.twasm now parses with **zero ERROR or MISSING nodes** — the full file, including regions, memory declaration, 5 function declarations, parameter lists with handle-mode types (&region, &mut region), effects clauses (Read/Write/Alloc/Free/ReadRegion/WriteRegion), return types, and the statement and expression forms used in example 01. ## What the v1 grammar now covers Beyond the v0 scaffold's region declarations: - memory_decl with initial/maximum/place clauses - function_decl with parameter_list, return_type, effects_clause, body - parameter with optional region_handle_type (&, &mut, own) - statements: region.get, region.set, region.scan, let [mut], assignment, if/else, return - expressions: literals (int / float / true / false / null), identifier_expr, region_var ($x), binary_expr (* / % + - == != < > <= >= && ||), unary_expr (- !), is_null_expr, paren_expr - field_path with nested access (.pos.x) - region_target — bare identifier form for post-null-check refs Precedence chain: ['unary', 'mul', 'add', 'cmp', 'and', 'or'] with unary at top and standard left-associative binary precedences. ## Still deferred (next Track A PR) - Imports / exports - Invariant declarations and proof statements - const declarations - Block-expression if (yield) - Match on union regions - L13-L16 surface (isolated / session / capability / choreography) - L11/L12 surface (cost_bound / fresh / version_of / region.sync) - Lifetime annotations on function decls - Striated region layout By production count v1 covers roughly 60-70% of spec/grammar.ebnf. ## Files - grammar.js extended from ~80 to ~270 lines - corpus/functions.txt new — 6 corpus tests covering memory_decl, if/else, is_null guard with bare-identifier region_target, region.scan with where + binding + body, nested field paths - queries/highlights.scm extended with function/memory/if/effect/ handle-mode/literal/builtin keyword categories - README.md updated to reflect v1 scope; deferred-list trimmed - tree-sitter.json metadata file (auto-generated by tree-sitter-cli) - package.json metadata tweaks (auto by cli) ## Verification Local generate + parse: $ tree-sitter generate # no conflicts, parser.c emitted $ tree-sitter parse examples/01-single-module.twasm -> 0 ERROR nodes, exit 0 CI integration still deferred to next Track A PR (when grammar reaches full EBNF parity the install-+-test cost justifies the ~30MB tree-sitter-cli toolchain on every PR run).
1 parent 7084347 commit b9f5523

6 files changed

Lines changed: 576 additions & 69 deletions

File tree

tools/tree-sitter-twasm/README.md

Lines changed: 37 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -2,40 +2,45 @@
22

33
Tree-sitter grammar for `.twasm` (typed-wasm surface syntax).
44

5-
## Status: scaffold (Phase 0 kickoff)
5+
## Status: v1 grammar (Phase 0 grammar extension)
66

7-
This is the **scaffold** for the tree-sitter grammar that will eventually
8-
back the Idris2 parser, the LSP, Linguist registration, and editor
9-
plugins. See `docs/PRODUCTION-PATH.adoc` §Phase 0 for the strategic
10-
context.
7+
The grammar now covers enough of `examples/01-single-module.twasm` to
8+
parse the full file: region declarations, memory declarations,
9+
function declarations (parameters / return types / effects), and the
10+
statement and expression forms used in example 01.
1111

1212
### What works today
1313

14-
- Tree-sitter scaffold + corpus harness (`tree-sitter test` runs)
15-
- `grammar.js` v0 — covers **region declarations only**:
16-
- `region Name { field: type; ... }`
17-
- `region Name[N] { ... }` (array quantifier)
18-
- Primitive types (`i32`, `f32`, `u8`, `bool`, etc.)
19-
- Nested region references (`@OtherRegion`)
20-
- Optional types (`opt<@T>`)
21-
- Fixed-size array fields (`u8[24]`)
22-
- `align N;` clauses
23-
- `where` field constraints (range form)
24-
- Single-line `//` comments
25-
26-
### What does NOT work yet (deferred to subsequent Track A PRs)
27-
28-
- Function declarations (`fn name(...) { ... }`)
29-
- Memory declarations, imports/exports
30-
- Statements: `region.get`, `region.set`, `let`, `if`, `while`
31-
- Effects, lifetime, cost-bound, freshness clauses
32-
- L13–L16 surface syntax (isolated modules, sessions, capabilities, choreography)
33-
- Tropical / epistemic extensions (L11/L12)
34-
- The full v1.5 surface — see `spec/grammar.ebnf` (~695 lines) for what's still ahead
35-
36-
This deliberate v0 scope covers enough of `examples/01-single-module.twasm`
37-
to exercise the toolchain end-to-end without overcommitting to the
38-
multi-month full-grammar port.
14+
- **Region declarations**`region Name { ... }`, array quantifiers `[N]`,
15+
primitives, region refs (`@T`), `opt<T>`, fixed-size arrays (`u8[24]`),
16+
`align N;`, `where` range constraints
17+
- **Memory declarations**`memory Name { initial: N; maximum: N; place X at N; ... }`
18+
- **Function declarations**`fn name(params) -> ret effects { ... } { body }`
19+
- parameter modes: `&region<R>`, `&mut region<R>`, `own region<R>`, and bare typed parameters
20+
- effects: `Read`, `Write`, `Alloc`, `Free`, `ReadRegion(R)`, `WriteRegion(R)`
21+
- **Statements**`region.get`, `region.set`, `region.scan`, `let [mut] name [: type] = expr;`,
22+
assignment, `if ... else`, `return`
23+
- **Expressions** — literals (`int`, `float`, `true`, `false`, `null`),
24+
identifiers, `$region_var`, binary ops (`+ - * / %`, `== != < > <= >=`, `&& ||`),
25+
unary (`-`, `!`), `is_null(expr)`, parenthesised
26+
- **Field paths** — single and nested (`.pos.x`)
27+
- **`//` line comments**
28+
29+
### What does NOT work yet (next Track A PR)
30+
31+
- Imports / exports (`import region X from "mod"`, `export region X`)
32+
- Invariant declarations and `proof` statements
33+
- `const` declarations
34+
- Block-expression `if ... yield`
35+
- Match on union regions
36+
- L13–L16 surface (`isolated`, `session`, `capability`, `choreography`)
37+
- L11/L12 surface (`cost_bound`, `fresh`, `version_of`, `region.sync`)
38+
- Lifetime annotations on function decls
39+
- `striated` region layout
40+
41+
Scope-wise the v1 grammar covers maybe 60–70% of `spec/grammar.ebnf`
42+
by production count. The remainder is sequenced for the next Track A
43+
PR.
3944

4045
## Why in-tree first
4146

@@ -66,8 +71,8 @@ regenerate locally via `tree-sitter generate`.
6671

6772
| Phase | This grammar's role |
6873
|-------|---------------------|
69-
| **0** (now) | Scaffold + region-decl coverage; proves toolchain works |
70-
| **1** | Extend to full `spec/grammar.ebnf` parity; back the Idris2 parser |
74+
| **0** (current) | v1 grammar — parses `examples/01-single-module.twasm` fully |
75+
| **1** | Extend to remaining `spec/grammar.ebnf` productions (imports, L11-L16, match, proof); back the Idris2 parser |
7176
| **4** | Extract to `hyperpolymath/tree-sitter-twasm`; publish to npm; submit to Linguist |
7277

7378
Tracked under issue [#48 (Phase 0)](https://github.com/hyperpolymath/typed-wasm/issues/48).
Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
============================================
2+
Simple function with effects and return type
3+
============================================
4+
5+
fn get_player_hp(players: &region<Players>, idx: i32) -> i32
6+
effects { ReadRegion(Players) }
7+
{
8+
region.get $players[idx] .hp -> hp;
9+
return hp;
10+
}
11+
12+
---
13+
14+
(source_file
15+
(function_decl
16+
name: (identifier)
17+
(parameter_list
18+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
19+
(parameter name: (identifier) type: (primitive_type)))
20+
(return_type (primitive_type))
21+
(effects_clause (effect (identifier)))
22+
(region_get_stmt
23+
(region_target (identifier) (identifier_expr (identifier)))
24+
(field_path (identifier))
25+
binding: (identifier))
26+
(return_stmt (identifier_expr (identifier)))))
27+
28+
29+
============================================
30+
Memory declaration with placements
31+
============================================
32+
33+
memory game_memory {
34+
initial: 64;
35+
maximum: 256;
36+
place Players at 0;
37+
place Enemies at 4800;
38+
}
39+
40+
---
41+
42+
(source_file
43+
(memory_decl
44+
name: (identifier)
45+
(initial_clause)
46+
(maximum_clause)
47+
(place_clause (identifier))
48+
(place_clause (identifier))))
49+
50+
51+
============================================
52+
If/else with region.set in both branches
53+
============================================
54+
55+
fn damage_player(players: &mut region<Players>, idx: i32, amount: i32)
56+
effects { ReadRegion(Players), WriteRegion(Players) }
57+
{
58+
region.get $players[idx] .hp -> current_hp;
59+
let new_hp: i32 = current_hp - amount;
60+
61+
if new_hp < 0 {
62+
region.set $players[idx] .hp, 0;
63+
} else {
64+
region.set $players[idx] .hp, new_hp;
65+
}
66+
}
67+
68+
---
69+
70+
(source_file
71+
(function_decl
72+
name: (identifier)
73+
(parameter_list
74+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
75+
(parameter name: (identifier) type: (primitive_type))
76+
(parameter name: (identifier) type: (primitive_type)))
77+
(effects_clause (effect (identifier)) (effect (identifier)))
78+
(region_get_stmt
79+
(region_target (identifier) (identifier_expr (identifier)))
80+
(field_path (identifier))
81+
binding: (identifier))
82+
(let_stmt
83+
name: (identifier)
84+
(primitive_type)
85+
(binary_expr (identifier_expr (identifier)) (identifier_expr (identifier))))
86+
(if_stmt
87+
(binary_expr (identifier_expr (identifier)) (literal (integer_literal)))
88+
(region_set_stmt
89+
(region_target (identifier) (identifier_expr (identifier)))
90+
(field_path (identifier))
91+
(literal (integer_literal)))
92+
(region_set_stmt
93+
(region_target (identifier) (identifier_expr (identifier)))
94+
(field_path (identifier))
95+
(identifier_expr (identifier))))))
96+
97+
98+
============================================
99+
Null check via is_null and bare-identifier region_target
100+
============================================
101+
102+
fn get_enemy_target_hp(enemies: &region<Enemies>, enemy_idx: i32) -> i32
103+
effects { ReadRegion(Enemies) }
104+
{
105+
region.get $enemies[enemy_idx] .target -> maybe_target;
106+
107+
if is_null(maybe_target) {
108+
return -1;
109+
}
110+
111+
region.get maybe_target .hp -> target_hp;
112+
return target_hp;
113+
}
114+
115+
---
116+
117+
(source_file
118+
(function_decl
119+
name: (identifier)
120+
(parameter_list
121+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
122+
(parameter name: (identifier) type: (primitive_type)))
123+
(return_type (primitive_type))
124+
(effects_clause (effect (identifier)))
125+
(region_get_stmt
126+
(region_target (identifier) (identifier_expr (identifier)))
127+
(field_path (identifier))
128+
binding: (identifier))
129+
(if_stmt
130+
(is_null_expr (identifier_expr (identifier)))
131+
(return_stmt (literal (integer_literal))))
132+
(region_get_stmt
133+
(region_target (identifier))
134+
(field_path (identifier))
135+
binding: (identifier))
136+
(return_stmt (identifier_expr (identifier)))))
137+
138+
139+
============================================
140+
region.scan with where + binding + body
141+
============================================
142+
143+
fn count_active_enemies(enemies: &region<Enemies>) -> i32
144+
effects { ReadRegion(Enemies) }
145+
{
146+
let mut count: i32 = 0;
147+
148+
region.scan $enemies where is_active == true -> |enemy| {
149+
count = count + 1;
150+
}
151+
152+
return count;
153+
}
154+
155+
---
156+
157+
(source_file
158+
(function_decl
159+
name: (identifier)
160+
(parameter_list
161+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier))))
162+
(return_type (primitive_type))
163+
(effects_clause (effect (identifier)))
164+
(let_stmt name: (identifier) (primitive_type) (literal (integer_literal)))
165+
(region_scan_stmt
166+
(region_target (identifier))
167+
(binary_expr (identifier_expr (identifier)) (literal))
168+
binding: (identifier)
169+
(assign_stmt (identifier) (binary_expr (identifier_expr (identifier)) (literal (integer_literal)))))
170+
(return_stmt (identifier_expr (identifier)))))
171+
172+
173+
============================================
174+
Nested field path
175+
============================================
176+
177+
fn move_player(players: &mut region<Players>, idx: i32, dx: f32, dy: f32)
178+
effects { ReadRegion(Players), WriteRegion(Players) }
179+
{
180+
region.get $players[idx] .pos.x -> old_x;
181+
region.get $players[idx] .pos.y -> old_y;
182+
183+
region.set $players[idx] .pos.x, old_x + dx;
184+
region.set $players[idx] .pos.y, old_y + dy;
185+
}
186+
187+
---
188+
189+
(source_file
190+
(function_decl
191+
name: (identifier)
192+
(parameter_list
193+
(parameter name: (identifier) type: (region_handle_type (handle_mode) (identifier)))
194+
(parameter name: (identifier) type: (primitive_type))
195+
(parameter name: (identifier) type: (primitive_type))
196+
(parameter name: (identifier) type: (primitive_type)))
197+
(effects_clause (effect (identifier)) (effect (identifier)))
198+
(region_get_stmt
199+
(region_target (identifier) (identifier_expr (identifier)))
200+
(field_path (identifier) (identifier))
201+
binding: (identifier))
202+
(region_get_stmt
203+
(region_target (identifier) (identifier_expr (identifier)))
204+
(field_path (identifier) (identifier))
205+
binding: (identifier))
206+
(region_set_stmt
207+
(region_target (identifier) (identifier_expr (identifier)))
208+
(field_path (identifier) (identifier))
209+
(binary_expr (identifier_expr (identifier)) (identifier_expr (identifier))))
210+
(region_set_stmt
211+
(region_target (identifier) (identifier_expr (identifier)))
212+
(field_path (identifier) (identifier))
213+
(binary_expr (identifier_expr (identifier)) (identifier_expr (identifier))))))

0 commit comments

Comments
 (0)