Use tree-sitter-multilingual for syntax highlighting

Author: johnsamuelwrites

.github/workflows/example-executability.yml

@@ -17,6 +17,12 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      - name: Check out tree-sitter-multilingual
+        uses: actions/checkout@v4
+        with:
+          repository: multilingualprogramming/tree-sitter-multilingual
+          path: tree-sitter-multilingual
+
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v5
         with:
@@ -30,6 +36,9 @@ jobs:
       - name: Validate release metadata
         run: python tools/check_release.py
 
+      - name: Verify Monarch tokenizer is in sync
+        run: python tools/check_monarch_sync.py --tree-sitter-root tree-sitter-multilingual --show-diff
+
       - name: Check local links
         run: python tools/check_links.py
 

.github/workflows/monitor-multilingual.yml

@@ -27,6 +27,12 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      - name: Check out tree-sitter-multilingual
+        uses: actions/checkout@v4
+        with:
+          repository: multilingualprogramming/tree-sitter-multilingual
+          path: tree-sitter-multilingual
+
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
@@ -67,6 +73,9 @@ jobs:
       - name: Validate release metadata
         run: python tools/check_release.py
 
+      - name: Verify Monarch tokenizer is in sync
+        run: python tools/check_monarch_sync.py --tree-sitter-root tree-sitter-multilingual --show-diff
+
       - name: Check local links
         run: python tools/check_links.py
 

CHANGELOG.md

@@ -34,3 +34,6 @@ First tagged release of the browser playground.
 ### Changed
 
 - Updated the playground's complete-feature example links and visible source-extension copy from `.ml` to `.multi` to match the renamed upstream example files.
+- Migrated the browser editor from CodeMirror 5 to Monaco Editor while preserving the existing Pyodide runtime flow, share links, theme toggle, keyboard shortcuts, and split-pane behavior.
+- Pinned browser syntax highlighting to `tree-sitter-multilingual` by shipping a generated `assets/monarch.json` tokenizer artifact from the grammar repository.
+- Updated the existing CI workflows to check out `multilingualprogramming/tree-sitter-multilingual` and fail if the pinned Monaco tokenizer drifts from the generated upstream artifact.

README.md

@@ -28,19 +28,22 @@ The playground runs entirely in the browser:
 
 - **Pyodide** - CPython 3.12 compiled to WASM; runs the full interpreter
 - **wabt.js** - WABT on WASM; compiles generated WAT to binary WASM in the browser
-- **CodeMirror 5** - editor with syntax highlighting and dark/light theme toggle
+- **Monaco Editor** - browser editor with generated multilingual syntax highlighting and dark/light theme toggle
 
 No server is required. The page prefers a local wheel from `assets/wheel_info.json` when present, then falls back to installing `multilingualprogramming` from PyPI via `micropip`.
 
 The main document is a thin shell. Playground styles live in `assets/playground.css`, and the application logic is split across `assets/main.js`, `assets/editor.js`, `assets/runtime.js`, `assets/ui.js`, `assets/theme.js`, `assets/i18n.js`, and `assets/examples.js`.
 
+The editor syntax layer is generated from the sibling `tree-sitter-multilingual` repository at `https://github.com/multilingualprogramming/tree-sitter-multilingual`. The playground ships a pinned `assets/monarch.json` derived from that grammar so browser highlighting stays aligned with upstream keyword and grammar updates.
+
 ## Release validation
 
 The pinned CI baseline lives in `requirements-build.txt`.
 
 This release line is checked in two ways:
 
 - `python tools/check_examples.py` executes all bundled example programs in each supported source language.
+- `python tools/check_monarch_sync.py --tree-sitter-root ../tree-sitter-multilingual` verifies that the pinned Monaco tokenizer still matches the generated grammar artifact from `https://github.com/multilingualprogramming/tree-sitter-multilingual`.
 - GitHub Actions runs `.github/workflows/example-executability.yml` on pushes and pull requests.
 - A scheduled compatibility workflow also checks the pinned baseline, the latest published package, and upstream `main`.
 

RELEASE.md

@@ -21,20 +21,24 @@ Prepare a playground release when at least one of these changes:
 3. Update visible release copy in [`index.html`](index.html), especially the backend note in the WAT / WASM tab.
 4. Refresh examples and language metadata in [`assets/examples.js`](assets/examples.js), [`assets/editor.js`](assets/editor.js), and related runtime modules if language coverage changed.
    Also verify that visible source-extension copy and upstream example links still match the current example filename convention, currently `.multi`.
+   If the grammar repo changed, refresh the pinned Monaco tokenizer asset in [`assets/monarch.json`](assets/monarch.json) from `../tree-sitter-multilingual/generated/monarch.json`.
+   The upstream grammar repository is `https://github.com/multilingualprogramming/tree-sitter-multilingual`.
 5. Verify metadata files:
    [`sitemap.xml`](sitemap.xml),
    [`robots.txt`](robots.txt),
    [`site.webmanifest`](site.webmanifest).
 6. Run:
    `python tools/check_release.py --expected-date YYYY-MM-DD`
 7. Run:
+   `python tools/check_monarch_sync.py --tree-sitter-root ../tree-sitter-multilingual`
+8. Run:
    `python tools/check_examples.py`
-8. Confirm GitHub Actions passes:
+9. Confirm GitHub Actions passes:
    [`.github/workflows/example-executability.yml`](.github/workflows/example-executability.yml)
-9. Open [`index.html`](index.html) in a browser and smoke-test:
+10. Open [`index.html`](index.html) in a browser and smoke-test:
    language switching, example switching, run/clear/share, output tabs, theme toggle, keyboard navigation, and narrow-screen layout.
-10. Update [`CHANGELOG.md`](CHANGELOG.md).
-11. Prepare outward-facing release notes using the template below.
+11. Update [`CHANGELOG.md`](CHANGELOG.md).
+12. Prepare outward-facing release notes using the template below.
 
 ## Release note template
 
@@ -73,6 +77,7 @@ Highlights in the current release line:
 - `with`, `try/except`, `lambda`, `match/case`, and `async/await`
 - `@property` accessors and bytes literals
 - modular browser assets centered on [`assets/playground.css`](assets/playground.css), [`assets/main.js`](assets/main.js), and [`assets/runtime.js`](assets/runtime.js)
+- Monaco-based browser editing with grammar-generated syntax highlighting
 - improved accessibility and mobile behavior
 - added release validation with [`tools/check_release.py`](tools/check_release.py)
 - added example executability validation with [`tools/check_examples.py`](tools/check_examples.py) and [`.github/workflows/example-executability.yml`](.github/workflows/example-executability.yml)