Skip to content

Commit 7e9f181

Browse files
feat(res-to-affine): Phase-1 migration assistant skeleton (Refs #57) (#314)
Lands the .res→.affine migration assistant proposed in #57 as a tight Phase-1 cut: an OCaml CLI under tools/res-to-affine/ that text-scans a ReScript source for four of the six anti-patterns from idaptik's Wave 3 pilot (side-effect imports, %raw blocks, untyped exceptions / Promise.catch, mutable globals via :=) and emits a .affine skeleton with migration markers + the quoted original. Architecture decision recorded in docs/MIGRATION-ASSISTANT.adoc: canonical .res grammar is rescript-lang/tree-sitter-rescript, vendored manifest-only under editors/tree-sitter-rescript/ (pinned commit 990214a, MIT, compatible with this repo's MPL-2.0). Phase 2 swaps the text scanner for a tree-sitter AST walker; Phase 3 does partial translation of pure-structural forms. See tools/res-to-affine/README.md for the full plan and tools/res-to-affine/test/ for snapshot tests. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent bb11638 commit 7e9f181

15 files changed

Lines changed: 869 additions & 0 deletions

File tree

docs/MIGRATION-ASSISTANT.adoc

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
// SPDX-License-Identifier: MPL-2.0
2+
// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell
3+
= ReScript → AffineScript migration assistant — architectural decision
4+
:toc: macro
5+
:toclevels: 2
6+
7+
Tracks issue #57 (parser + metaparser). Companion to
8+
link:RESCRIPT-ELIMINATION.adoc[RESCRIPT-ELIMINATION.adoc], which is the
9+
authoritative ledger for the broader estate ReScript-surface retirement.
10+
11+
toc::[]
12+
13+
== Context
14+
15+
Estate language policy retires ReScript in favour of AffineScript →
16+
typed-wasm. Per the inventories captured in
17+
link:RESCRIPT-ELIMINATION.adoc[RESCRIPT-ELIMINATION.adoc] and the
18+
upstream tracker `hyperpolymath/gitbot-fleet#148`, ~5k LOC of ReScript
19+
remains in `gitbot-fleet/bots/sustainabot/bot-integration/src/` alone;
20+
the idaptik tail is ~542 `.res` files plus ~80 `.ts`. By-hand
21+
translation is impractical, *and* a literal transliterator misses the
22+
point — the AffineScript answer to ReScript's anti-patterns is
23+
*re-decomposition*, not a token-level rewrite.
24+
25+
Issue #57 proposes a *migration assistant*: a tool that reads `.res`,
26+
recognises the anti-patterns surfaced by the idaptik Wave 3 pilot, and
27+
emits a `.affine` *skeleton* that surfaces the work the human migrator
28+
still owes.
29+
30+
== Decision
31+
32+
* The migration assistant lives at `tools/res-to-affine/` as an OCaml
33+
CLI built by the repo's existing `dune` toolchain.
34+
* The canonical source-of-truth grammar for `.res` parsing is
35+
https://github.com/rescript-lang/tree-sitter-rescript[`rescript-lang/tree-sitter-rescript`],
36+
vendored manifest-only at `editors/tree-sitter-rescript/` (pinned to
37+
commit `990214a83f25801dfe0226bd7e92bb71bba1970f`, version 6.0.0,
38+
MIT-licensed and compatible with this repo's MPL-2.0).
39+
* The tool ships in three phases:
40+
+
41+
[cols="1,3,2"]
42+
|===
43+
| Phase | What it does | Status
44+
45+
| 1
46+
| Text-scan emitter detecting 4 of the 6 anti-patterns, emitting a
47+
`.affine` skeleton with migration markers and the quoted original
48+
for reference.
49+
| this PR
50+
51+
| 2
52+
| Replaces the text scanner with a tree-sitter AST walker reading the
53+
vendored grammar. Adds the two deferred patterns. Same emitter
54+
interface.
55+
| follow-up
56+
57+
| 3
58+
| Partial *translation* of pure-structural forms (type aliases, sum
59+
decls, simple `let` bindings, `switch` → `match`). Effect-laden /
60+
exception-bearing / globally-mutating regions remain TODO islands.
61+
| follow-up
62+
|===
63+
* The Phase-1 deliverable is **deliberately small and useful in
64+
isolation**. It gates the architectural commitment to tree-sitter
65+
behind something that already pays its way against real estate
66+
`.res` files.
67+
68+
== Alternatives considered
69+
70+
=== Use the ReScript compiler's own AST (`bs-tools` / `rescript ast`)
71+
72+
The richest signal is in the ReScript compiler's typed AST. Rejected
73+
because:
74+
75+
* Adding `rescript` as a build-time dependency contradicts the estate
76+
language policy (which bans new ReScript code and treats ReScript as
77+
the artefact to be retired).
78+
* The ReScript compiler's AST changes across versions in
79+
non-backwards-compatible ways; pinning would create an ongoing
80+
compatibility burden in the wrong direction.
81+
82+
=== Write a hand-rolled `.res` lexer/parser in OCaml
83+
84+
We already have `lib/rescript_codegen.ml` going *affinescript → .res*,
85+
so the grammar is partly understood. Rejected because:
86+
87+
* ReScript's surface syntax is large; recreating it for a one-way
88+
migration tool is days-to-weeks of work that the canonical
89+
tree-sitter grammar has already done and maintains.
90+
* The community grammar is MIT-licensed and version-pinned; the cost
91+
of consuming it is a one-line manifest plus an install script.
92+
93+
=== Pattern-detector only (no AST in any phase)
94+
95+
Phase 1 *is* this — but committing to it permanently would leave the
96+
two structural anti-patterns (callback records, oversized functions)
97+
undetected forever, and would block Phase 3 (partial translation),
98+
which is what makes the tool earn its keep on idaptik's 542 files.
99+
100+
== Consequences
101+
102+
* `editors/tree-sitter-rescript/` exists for the migration pipeline,
103+
not as an editor binding. The editor binding for AffineScript itself
104+
remains `editors/tree-sitter-affinescript/`.
105+
* `tools/res-to-affine/` is the first OCaml tool under `tools/`
106+
(existing tools are shell scripts or Rust). The `dune` integration
107+
is local to the tool's own `dune` file; no workspace changes.
108+
* Phase 2 introduces `tree-sitter` CLI as a runtime dependency for the
109+
migration assistant. It is *not* a build-time dependency for the
110+
AffineScript compiler itself. CI for the migration tool's Phase-2
111+
tests will need to install `tree-sitter-cli`.
112+
* The Phase plan is recorded in
113+
link:../tools/res-to-affine/README.md[`tools/res-to-affine/README.md`];
114+
this document is the architectural decision, the README is the
115+
user/contributor surface.
116+
117+
== References
118+
119+
* `tools/res-to-affine/README.md` — tool usage, Phase plan, design rationale.
120+
* `editors/tree-sitter-rescript/README.md` — vendoring manifest details.
121+
* `affinescript#57` — parser + metaparser proposal.
122+
* `gitbot-fleet#148` — downstream tracker for the consumed ReScript subtree.
123+
* link:RESCRIPT-ELIMINATION.adoc[`RESCRIPT-ELIMINATION.adoc`] — estate-wide ledger.
124+
* https://github.com/hyperpolymath/idaptik/blob/main/migration/main/LESSONS.md[idaptik LESSONS.md]
125+
— six anti-patterns the assistant targets.
126+
* https://github.com/hyperpolymath/idaptik/blob/main/migration/main/PILOT.md[idaptik PILOT.md]
127+
— original Wave-3 pilot that surfaced the six patterns.
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
<!-- SPDX-License-Identifier: MPL-2.0 -->
2+
<!-- SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell -->
3+
4+
# tree-sitter-rescript (vendoring manifest)
5+
6+
This directory is a **manifest-only vendoring** of the canonical
7+
[`rescript-lang/tree-sitter-rescript`][upstream] grammar. The grammar
8+
itself is not copied into this repository — `package.json` declares it
9+
as a dependency, and `scripts/install.sh` fetches and builds it at the
10+
pinned commit.
11+
12+
The grammar is consumed by `tools/res-to-affine/`, the `.res → .affine`
13+
migration assistant (`affinescript#57`). It is **not** an editor binding
14+
for AffineScript; for that, see `editors/tree-sitter-affinescript/`.
15+
16+
## Pinned upstream
17+
18+
- **Repository:** <https://github.com/rescript-lang/tree-sitter-rescript>
19+
- **Commit:** `990214a83f25801dfe0226bd7e92bb71bba1970f`
20+
- **Version:** 6.0.0
21+
- **License:** MIT (preserved upstream; compatible with this repo's MPL-2.0)
22+
23+
When updating the pin, regenerate `tools/res-to-affine/test/expected/`
24+
snapshots, since AST shapes may shift.
25+
26+
## Install
27+
28+
```sh
29+
./scripts/install.sh
30+
```
31+
32+
This writes a `tree-sitter-rescript` directory under `tools/vendor/`
33+
(gitignored — same convention as the WASI adapter pinning), containing
34+
the generated parser. Requires `git` and `tree-sitter` CLI on PATH.
35+
36+
## Why manifest, not copy
37+
38+
The upstream grammar is ~10k lines of JS plus generated C. Copying it
39+
into this MPL-2.0 repo would (a) bloat the tree, (b) create an ongoing
40+
sync burden, and (c) duplicate MIT-licensed code we have no business
41+
modifying. The manifest+install approach keeps the dependency explicit
42+
and pinned without absorbing the source.
43+
44+
[upstream]: https://github.com/rescript-lang/tree-sitter-rescript
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"name": "@affinescript/tree-sitter-rescript-vendoring",
3+
"version": "0.1.0",
4+
"private": true,
5+
"description": "Manifest-only vendoring of rescript-lang/tree-sitter-rescript for the .res -> .affine migration assistant (affinescript#57).",
6+
"license": "MPL-2.0",
7+
"dependencies": {
8+
"tree-sitter-rescript": "github:rescript-lang/tree-sitter-rescript#990214a83f25801dfe0226bd7e92bb71bba1970f"
9+
},
10+
"devDependencies": {
11+
"tree-sitter-cli": "^0.25.0"
12+
},
13+
"scripts": {
14+
"install-grammar": "./scripts/install.sh"
15+
}
16+
}
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/usr/bin/env bash
2+
# SPDX-License-Identifier: MPL-2.0
3+
# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell
4+
#
5+
# Fetch and build the pinned tree-sitter-rescript grammar.
6+
# Output goes under ../../.build/tree-sitter-rescript/ (gitignored).
7+
8+
set -euo pipefail
9+
10+
UPSTREAM_COMMIT="990214a83f25801dfe0226bd7e92bb71bba1970f"
11+
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
12+
# tools/vendor/ is the repo's convention for fetched-not-committed deps
13+
# (see .gitignore line 103, mirrors the WASI adapter provisioning).
14+
BUILD_DIR="${REPO_ROOT}/tools/vendor/tree-sitter-rescript"
15+
16+
if ! command -v tree-sitter >/dev/null 2>&1; then
17+
echo "error: tree-sitter CLI not found on PATH" >&2
18+
echo " install via: npm install -g tree-sitter-cli" >&2
19+
exit 2
20+
fi
21+
22+
if ! command -v git >/dev/null 2>&1; then
23+
echo "error: git not found on PATH" >&2
24+
exit 2
25+
fi
26+
27+
mkdir -p "$(dirname "$BUILD_DIR")"
28+
29+
if [ -d "$BUILD_DIR/.git" ]; then
30+
git -C "$BUILD_DIR" fetch --quiet origin "$UPSTREAM_COMMIT" || true
31+
git -C "$BUILD_DIR" checkout --quiet "$UPSTREAM_COMMIT"
32+
else
33+
rm -rf "$BUILD_DIR"
34+
git clone --quiet https://github.com/rescript-lang/tree-sitter-rescript.git "$BUILD_DIR"
35+
git -C "$BUILD_DIR" checkout --quiet "$UPSTREAM_COMMIT"
36+
fi
37+
38+
cd "$BUILD_DIR"
39+
tree-sitter generate
40+
41+
echo "tree-sitter-rescript built at ${BUILD_DIR} (commit ${UPSTREAM_COMMIT})"

tools/res-to-affine/README.md

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
<!-- SPDX-License-Identifier: MPL-2.0 -->
2+
<!-- SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell -->
3+
4+
# `res-to-affine` — ReScript-to-AffineScript migration assistant
5+
6+
A small OCaml CLI that reads a `.res` file and emits a `.affine` skeleton
7+
with **migration markers** — comments that name each anti-pattern the
8+
scanner found, point at the source line, and propose the AffineScript
9+
answer the human migrator should consider before porting.
10+
11+
Tracks: [`affinescript#57`](https://github.com/hyperpolymath/affinescript/issues/57)
12+
(parser + metaparser).
13+
Consumed by: [`hyperpolymath/gitbot-fleet#148`](https://github.com/hyperpolymath/gitbot-fleet/issues/148)
14+
and the broader `idaptik` migration.
15+
16+
## Usage
17+
18+
```sh
19+
# print skeleton to stdout
20+
dune exec tools/res-to-affine/main.exe -- path/to/Foo.res
21+
22+
# or write to a file
23+
dune exec tools/res-to-affine/main.exe -- path/to/Foo.res -o Foo.affine
24+
```
25+
26+
The output is **not compilable**. It is a starting point for the human:
27+
a quoted copy of the original sits at the bottom; the top carries a
28+
migration-considerations block; the middle is a `module` stub with
29+
`TODO`s. The human picks the decomposition; the tool surfaces what
30+
needs re-decomposing.
31+
32+
## What gets flagged (Phase 1)
33+
34+
The six anti-patterns surfaced in the
35+
[idaptik Wave 3 pilot](https://github.com/hyperpolymath/idaptik/blob/main/migration/main/LESSONS.md),
36+
of which the line-based scanner reliably detects four:
37+
38+
| Tag | Detection | AffineScript answer |
39+
|---|---|---|
40+
| `side-effect-import` | `let _ = Mod.foo` at top level | Explicit registration call |
41+
| `raw-js` | `%raw(...)` or `[%bs.raw ...]` | Typed extern (`ABI-FFI-README.md`) |
42+
| `untyped-exception` | `Promise.catch`, `Js.Exn`, `raise`, `try` | `Result[E, A]` / `Validation[E, A]` |
43+
| `mutable-global` | `:=` operator | Affine record threaded through |
44+
45+
Deferred to Phase 2 (need real AST):
46+
47+
- **inline lambda callback record** — N ≥ 3 `~handler: (...) =>` lambdas
48+
inside one record literal (collapse to a row-polymorphic record).
49+
- **oversized function** — function body > ~50 LOC (decompose).
50+
51+
## Why a skeleton and not a transliteration
52+
53+
The Frontier Programming Guides' standing rule is **re-decompose, not
54+
transliterate**. A line-for-line port preserves the source's anti-patterns
55+
into the target language and produces `.affine` files that are technically
56+
parseable but architecturally still ReScript. The migration assistant's
57+
job is to *make the re-decomposition tractable*, not to skip it. So:
58+
59+
- The skeleton is **honest about being incomplete** — it does not
60+
compile, on purpose.
61+
- The original source is **quoted at the bottom** so the migrator
62+
doesn't tab between files while writing the port.
63+
- Each marker links a source line to the AffineScript pattern that
64+
replaces it, so the migrator's next action is clear.
65+
66+
## Phase plan
67+
68+
### Phase 1 — text-scan emitter (this PR)
69+
70+
- OCaml binary builds with the repo's existing `dune` toolchain.
71+
- `Scanner` walks lines with `str` regexes; cheap and dependency-free.
72+
- `Emitter` writes the migration-considerations block, a `module` stub,
73+
and the quoted source.
74+
- Snapshot tests under `test/` ensure stable output.
75+
76+
This phase is **deliberately small**. It is useful immediately — runs
77+
against any `.res` file, surfaces 4 of 6 anti-patterns, gives the
78+
migrator a starting document — and it gates the architectural commitment
79+
to tree-sitter in Phase 2 behind something that already pays its way.
80+
81+
### Phase 2 — tree-sitter AST walker
82+
83+
- Install the pinned grammar from
84+
`editors/tree-sitter-rescript/` (manifest-only vendoring of
85+
`rescript-lang/tree-sitter-rescript@990214a`).
86+
- Replace `Scanner` with a walker over the s-expression output of
87+
`tree-sitter parse --quiet`, parsed by the existing `sexplib0`
88+
dependency.
89+
- Adds the two deferred patterns (callback records, oversized
90+
functions) and unlocks **structural** translation of trivial forms
91+
(e.g. `option<X>``Option[X]`, `result<X, Y>``Result[Y, X]`,
92+
`switch x { | A => ... }``match x { A => ... }`).
93+
- The `Emitter` interface does not change: same skeleton shape, same
94+
marker schema, richer body.
95+
96+
### Phase 3 — partial translation
97+
98+
Once the AST walker exists, the emitter can do more than mark — it can
99+
**translate** the pure-structural parts (type aliases, sum decls,
100+
simple `let` bindings, switch-to-match) and leave only effect-laden,
101+
exception-bearing, or globally-mutating regions as TODO. The skeleton
102+
becomes a working port of ~60–80% of the input, with TODO islands
103+
where re-decomposition is genuinely required.
104+
105+
Phase 3 is when the tool earns its keep on idaptik's 542 files.
106+
107+
## Testing
108+
109+
```sh
110+
dune test tools/res-to-affine/
111+
```
112+
113+
To regenerate snapshots after an intentional emitter change:
114+
115+
```sh
116+
cd tools/res-to-affine/test
117+
../../../_build/default/tools/res-to-affine/main.exe \
118+
fixtures/sample.res > expected/sample.affine
119+
```
120+
121+
The fixture under `test/fixtures/sample.res` is synthetic and exercises
122+
every Phase-1 anti-pattern. Real `.res` files from the estate (e.g.
123+
`gitbot-fleet/bots/sustainabot/bot-integration/src/*.res`) can be run
124+
ad hoc through the CLI without changes to the test suite.
125+
126+
## Non-goals
127+
128+
- **Not a ReScript compiler.** The scanner does not parse ReScript;
129+
even Phase 2 only walks the tree-sitter CST, not the ReScript
130+
type-checker's AST. If a `.res` file is syntactically invalid the
131+
tool may still emit a (less useful) skeleton.
132+
- **Not a build-time dependency on ReScript.** The pinned grammar is a
133+
parser, not the ReScript compiler. The estate's language policy
134+
(CLAUDE.md) bans new ReScript code; this tool exists to **help retire
135+
the existing ReScript surface**, not to bring more in.
136+
- **Not for editor integration.** Editor tree-sitter bindings for
137+
AffineScript live at `editors/tree-sitter-affinescript/`; this tool's
138+
vendored grammar is for the migration pipeline only.

tools/res-to-affine/dune

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
; SPDX-License-Identifier: MPL-2.0
2+
3+
(executable
4+
(name main)
5+
(modules main)
6+
(public_name res-to-affine)
7+
(package affinescript)
8+
(libraries res_to_affine cmdliner fmt fmt.tty))
9+
10+
(library
11+
(name res_to_affine)
12+
(modules scanner emitter)
13+
(libraries str))

0 commit comments

Comments
 (0)