Skip to content

tree-sitter: v1 grammar — parses examples/01-single-module.twasm end-to-end (zero ERROR nodes)#62

Merged
hyperpolymath merged 1 commit into
mainfrom
phase-0/tree-sitter-grammar-extension
May 25, 2026
Merged

tree-sitter: v1 grammar — parses examples/01-single-module.twasm end-to-end (zero ERROR nodes)#62
hyperpolymath merged 1 commit into
mainfrom
phase-0/tree-sitter-grammar-extension

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Summary

Extends the Phase 0 tree-sitter scaffold (PR #58, region-decls only) to v1 grammar that parses examples/01-single-module.twasm end-to-end with zero parse errors.

Verification (local)

cd tools/tree-sitter-twasm
npm install
tree-sitter generate    # no conflicts; parser.c emitted
tree-sitter parse ../../examples/01-single-module.twasm
# → 0 ERROR / 0 MISSING nodes, exit 0, full AST for all 134 lines

What v1 now covers (beyond region declarations)

  • memory_declmemory Name { initial: N; maximum: N; place X at N; }
  • function_declfn name(params) -> ret effects { ... } { body }
    • parameter_list with region_handle_type (&, &mut, own)
    • effects_clauseRead / Write / Alloc / Free / ReadRegion(R) / WriteRegion(R)
    • return_type including void
  • Statementsregion.get, region.set, region.scan (with where + binding + body), let [mut] name [: type] = expr;, assignment, if/else, return
  • Expressions — literals (int/float/bool/null), identifier_expr, region_var ($x), binary_expr (full operator set with precedence), unary_expr, is_null_expr, paren_expr
  • Field paths — including nested access (.pos.x)
  • region_target — bare-identifier form for post-null-check references

Precedence chain: ['unary', 'mul', 'add', 'cmp', 'and', 'or'] with unary at top.

Still deferred (next Track A PR)

  • Imports / exports
  • Invariant declarations + proof statements
  • const declarations
  • Block-expression if ... yield
  • Match on union regions
  • L13–L16 surface (isolated, session, capability, choreography)
  • L11/L12 surface (cost_bound, fresh, version_of, region.sync)
  • Lifetime annotations on function decls
  • Striated region layout

By production count v1 covers ~60–70% of spec/grammar.ebnf.

Files

File Change
grammar.js ~80 → ~270 lines
corpus/functions.txt new — 6 corpus tests covering memory_decl, if/else, is_null guard with bare-ident target, scan, nested field paths
queries/highlights.scm extended with function/memory/if/effect/handle-mode/literal/builtin categories
README.md updated to reflect v1 scope
tree-sitter.json new — tree-sitter-cli metadata
package.json minor metadata tweaks

CI integration

Still deferred to the next Track A PR. When the grammar reaches full EBNF parity, the install-and-test cost (~30MB tree-sitter-cli toolchain on every PR run) is justified by the regression value. Today the 6-test corpus + the local parse-of-example-01 covers the regression risk well enough.

Phase 0 impact

This PR plus the existing Track A scaffold (PR #58) moves the Phase 0 §Track A checklist from "scaffold landed" to "scaffold + v1 grammar landed, parses simplest example end-to-end". Remaining Track A items (full EBNF parity, Idris2 parser, ReScript cut, codegen v0) are still ahead.

Test plan

  • tree-sitter generate succeeds (no conflicts)
  • tree-sitter parse examples/01-single-module.twasm returns 0 ERROR / 0 MISSING nodes
  • Corpus tests pass on a CI runner with tree-sitter-cli installed (deferred to next PR with CI wiring)
  • Smoke test, structural E2E, claim-envelope, property, security-envelope, proof regression — none touch tools/tree-sitter-twasm/ so no regressions expected

Generated by Claude Code

…to-end

Extends the Phase 0 scaffold (PR #58, region-decls only) to v1
coverage. The 134-line examples/01-single-module.twasm now parses
with **zero ERROR or MISSING nodes** — the full file, including
regions, memory declaration, 5 function declarations, parameter
lists with handle-mode types (&region, &mut region), effects
clauses (Read/Write/Alloc/Free/ReadRegion/WriteRegion), return
types, and the statement and expression forms used in example 01.

## What the v1 grammar now covers

Beyond the v0 scaffold's region declarations:

- memory_decl with initial/maximum/place clauses
- function_decl with parameter_list, return_type, effects_clause, body
- parameter with optional region_handle_type (&, &mut, own)
- statements: region.get, region.set, region.scan, let [mut],
  assignment, if/else, return
- expressions: literals (int / float / true / false / null),
  identifier_expr, region_var ($x), binary_expr (* / % + -
  == != < > <= >= && ||), unary_expr (- !), is_null_expr,
  paren_expr
- field_path with nested access (.pos.x)
- region_target — bare identifier form for post-null-check refs

Precedence chain: ['unary', 'mul', 'add', 'cmp', 'and', 'or'] with
unary at top and standard left-associative binary precedences.

## Still deferred (next Track A PR)

- Imports / exports
- Invariant declarations and proof statements
- const declarations
- Block-expression if (yield)
- Match on union regions
- L13-L16 surface (isolated / session / capability / choreography)
- L11/L12 surface (cost_bound / fresh / version_of / region.sync)
- Lifetime annotations on function decls
- Striated region layout

By production count v1 covers roughly 60-70% of spec/grammar.ebnf.

## Files

- grammar.js extended from ~80 to ~270 lines
- corpus/functions.txt new — 6 corpus tests covering memory_decl,
  if/else, is_null guard with bare-identifier region_target,
  region.scan with where + binding + body, nested field paths
- queries/highlights.scm extended with function/memory/if/effect/
  handle-mode/literal/builtin keyword categories
- README.md updated to reflect v1 scope; deferred-list trimmed
- tree-sitter.json metadata file (auto-generated by tree-sitter-cli)
- package.json metadata tweaks (auto by cli)

## Verification

Local generate + parse:
  $ tree-sitter generate    # no conflicts, parser.c emitted
  $ tree-sitter parse examples/01-single-module.twasm
    -> 0 ERROR nodes, exit 0

CI integration still deferred to next Track A PR (when grammar
reaches full EBNF parity the install-+-test cost justifies the
~30MB tree-sitter-cli toolchain on every PR run).
@github-actions

Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 93 issues detected

Severity Count
🔴 Critical 6
🟠 High 36
🟡 Medium 51

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Issue in quality.yml",
    "type": "missing_workflow",
    "file": "quality.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "Issue in security-policy.yml",
    "type": "missing_workflow",
    "file": "security-policy.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "Action actions/upload-artifact@v4 needs attention",
    "type": "unpinned_action",
    "file": "release.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/download-artifact@v4 needs attention",
    "type": "unpinned_action",
    "file": "release.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
    "type": "believe_me",
    "file": "/home/runner/work/typed-wasm/typed-wasm/src/abi/TypedWasm/ABI/SessionProtocol.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "critical"
  },
  {
    "reason": "assert_total bypasses totality checker (1 occurrences, CWE-704)",
    "type": "assert_total",
    "file": "/home/runner/work/typed-wasm/typed-wasm/src/abi/TypedWasm/ABI/SessionProtocol.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "high"
  },
  {
    "reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
    "type": "believe_me",
    "file": "/home/runner/work/typed-wasm/typed-wasm/src/abi/TypedWasm/ABI/Echo.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "critical"
  },
  {
    "reason": "assert_total bypasses totality checker (1 occurrences, CWE-704)",
    "type": "assert_total",
    "file": "/home/runner/work/typed-wasm/typed-wasm/src/abi/TypedWasm/ABI/Echo.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "high"
  },
  {
    "reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
    "type": "believe_me",
    "file": "/home/runner/work/typed-wasm/typed-wasm/src/abi/TypedWasm/ABI/ResourceCapabilities.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "critical"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

hyperpolymath added a commit that referenced this pull request May 25, 2026
- Delete rescript.json, create affinescript.json
- Rename .res files to .affine (7 files)
- Convert SafeDOMExample.affine to AffineScript syntax
- Replace all ReScript references with AffineScript in:
  - Documentation files
  - Configuration files (Justfile, package.json, web-project-deno.json)
  - .hypatia-ignore exemptions
- Parser .affine files retain OCaml-style syntax pending full migration

Addresses Language / package anti-pattern policy failures in PRs #61 and #62.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
@hyperpolymath
hyperpolymath merged commit 7d32728 into main May 25, 2026
26 of 27 checks passed
@hyperpolymath
hyperpolymath deleted the phase-0/tree-sitter-grammar-extension branch May 25, 2026 15:29
hyperpolymath added a commit that referenced this pull request May 25, 2026
…dd audit/onboarding/status/proposals stubs (#63)

## Summary

Complete document tidy of the repo for humans and machines.
RSR-template-aligned taxonomy, deletions of worthless / template-residue
files, additions of missing taxonomy stubs, drift-detection enforcement
of the new structure.

## Deletions (8 files, ~all worthless)

| File | Why |
|---|---|
| `QUICKSTART-DEV.adoc` | Template residue — contained unfilled
`{{BUILD_CMD}}`, `{{TEST_CMD}}`, `{{LANG_STACK}}` placeholders |
| `QUICKSTART-USER.adoc` | Template residue — body literally said "Typed
Wasm — See README.adoc for details." |
| `QUICKSTART-MAINTAINER.adoc` | Template residue — same pattern |
| `README.adoc.invariants.md` | Auto-generated heuristic "invariant
suggestion" garbage; no purpose |
| `docs/WHITEPAPER.md.invariants.md` | Same pattern |
| `docs/wikis/0.2-AI-MANIFEST.a2ml` + `docs/wikis/README.adoc` | Empty
placeholder directory conflicting with `docs/wiki/` (from PR #61) —
that's where actual wiki source lives |
| `generated/abi/README.adoc` | Tracked file inside a `generated/`
directory that's supposed to be gitignored; content moved to
`docs/architecture/ABI-PIPELINE.adoc` |

The 3 deleted QUICKSTARTs are replaced by the single clean
`docs/QUICKSTART.adoc` that already exists (60-second build + test path;
no template residue).

## Additions (4 new files, RSR-template alignment)

| File | Purpose |
|---|---|
| `AUDIT.adoc` (root) | RSR-template hard-rules summary;
typed-wasm-specific audit posture; status pointers; bot rules |
| `docs/onboarding/README.adoc` | New-contributor entry point;
reading-order; by-role guidance (use / contribute / proofs / security) |
| `docs/status/README.adoc` | RSR taxonomy alignment — pointer to the
root status documents (LEVEL-STATUS / PROOF-NEEDS / TEST-NEEDS);
explains why they stay at root |
| `docs/proposals/README.adoc` | RSR taxonomy alignment — staging ground
for future design proposals; authoring rules |
| `docs/architecture/ABI-PIPELINE.adoc` | Content rescued from the
deleted `generated/abi/README.adoc`; now in the architectural layer it
belongs to |

## Updates

- **`README.adoc`** — Quick Tour rebuilt with current paths (uses
`src/abi/TypedWasm/ABI/*.idr`, includes tree-sitter grammar, verifier
crate, status documents, AUDIT.adoc, onboarding index)
- **`ffi/zig/src/main.zig`** — header comment updated to point at new
`docs/architecture/ABI-PIPELINE.adoc` instead of deleted
`generated/abi/README.adoc`
- **`.gitignore`** — gitignore
`tools/tree-sitter-twasm/package-lock.json` for consistency with the
existing "deno.lock is the lockfile of record" convention
- **`tests/aspect/claim-envelope.mjs`** — pinned new RSR taxonomy files
(`AUDIT.adoc`, `docs/onboarding/README.adoc`, `docs/status/README.adoc`,
`docs/PRODUCTION-PATH.adoc`) into the rsrFiles list so drift-detection
enforces the new structure going forward
- **`docs/wiki/Phase-0-Status.md`** — synced with PR #62 (tree-sitter
v1) merge + this tidy PR in flight

## RSR taxonomy alignment

Compared to `hyperpolymath/rsr-template-repo`:

| RSR template item | typed-wasm status |
|---|---|
| `AUDIT.adoc` (root) | ✅ added in this PR |
| `EXPLAINME.adoc` (root) | ✅ already present |
| `CHANGELOG.md` (root) | ✅ already present |
| Single `QUICKSTART.adoc` | ✅ at `docs/QUICKSTART.adoc`; 3 root
template-residue versions deleted |
| `docs/onboarding/` | ✅ added in this PR |
| `docs/status/` | ✅ added in this PR (root status docs remain for
visibility; this is a pointer) |
| `docs/proposals/` | ✅ added in this PR |
| `docs/decisions/` | ✅ already present |
| `docs/architecture/` | ✅ already present (+ new ABI-PIPELINE.adoc) |

## What's NOT changed (deliberate)

- **`LEVEL-STATUS.md`, `PROOF-NEEDS.md`, `TEST-NEEDS.md` stay at root.**
These are read every audit, every CI run, every reviewer's first-pass —
root visibility is load-bearing. The RSR alignment is achieved via
`docs/status/README.adoc` as a pointer rather than a move that would
break dozens of existing references in workflow files + aspect tests.
- **`SPARK-experiment/`** stays. It's lightweight (one README), clearly
self-marked as exploratory, costs nothing to keep, and represents a real
future direction.
- **`benchmarks/`** not renamed to RSR template's `benches/`. The plural
form is already integrated everywhere (Justfile, CI, aspect tests); the
rename cost exceeds the consistency benefit.
- **`docs/arxiv/`, `docs/governance/`, `docs/legal/`, `docs/practice/`,
etc.** — keeping them as-is; RSR template scaffolding that's already in
place.

## Verification

All test surfaces green after tidy:

| Surface | Result |
|---|---|
| `claim-envelope` aspect | **57 passed** (was 53; +4 from new RSR file
pins) |
| `security-envelope` aspect | 10 passed |
| `property_test` | 29 passed |
| `proof/regression` | 25 passed, 1 skipped (idris2 not on PATH) |

claim-envelope §8 (path-reference drift detection) confirms all 31 path
references across README/ROADMAP/EXPLAINME/CLAUDE.md resolve on disk
after the changes.

## Test plan

- [x] Test surfaces all green locally
- [x] No dangling references to deleted files (`git grep` confirms)
- [ ] CI on this PR matches expected pattern (cargo-audit / smoke /
structural / cargo-verify green; persistent-red advisories unchanged)
- [ ] Reviewer can read `docs/onboarding/README.adoc` and reach a
working build via `docs/QUICKSTART.adoc`

## Repo health summary after this PR lands

- ✅ Root directory: only load-bearing files (no template residue, no
stale heuristic artefacts)
- ✅ Documentation taxonomy: RSR-aligned without breaking existing
references
- ✅ Drift detection: enforces new structure (claim-envelope §3 + §8)
- ✅ Wiki source: tracked in `docs/wiki/` with sync workflow;
cross-referenced from `Phase-0-Status.md` page
- ✅ Machine-readable: `.machine_readable/6a2/STATE.a2ml` reflects
current state (post #61)
- ✅ Audit posture: explicit at `AUDIT.adoc`, machine + human readable
- ✅ Onboarding: `docs/onboarding/README.adoc` gives a 5-minute path for
any new role


---
_Generated by [Claude
Code](https://claude.ai/code/session_01ExgUTJmU5UQQNLKynwxDjm)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants