Skip to content

Commit 4032145

Browse files
committed
docs(nook): record measured tree-sitter subset size finding
1 parent 8de939c commit 4032145

1 file changed

Lines changed: 54 additions & 10 deletions

File tree

docs/nook/design/02-tree-sitter-highlighting.md

Lines changed: 54 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -95,15 +95,59 @@ primary highlighter with chroma as the fallback for unwired languages.
9595
Nothing here changes the render path until the size question is settled,
9696
so the first-paint rule and the binary-size north star both hold.
9797

98-
## Open questions for the build hour
99-
100-
- Does subset-embedding one grammar actually avoid the 542 init
101-
registrations, or is the registry all-or-nothing? (Determines slice 1
102-
vs. runtime-blob-loading vs. WASM.)
103-
- Is upstream willing to expose per-grammar subpackages / DCE-friendly
104-
registration? A lean single-language import would make this trivial and
105-
is worth an issue against odvcencio/gotreesitter once the spike pins the
106-
exact constraint.
98+
## Measured: the subset path holds (2026-07-04, v0.20.9)
99+
100+
The size-gate spike ran. A standalone `CGO_ENABLED=0` module that imports
101+
`grammars`, resolves Go via `DetectLanguage("main.go")`, loads the grammar
102+
through `entry.Language()`, and highlights a Go snippet with
103+
`ts.NewHighlighter(lang, entry.HighlightQuery)`:
104+
105+
| build | tags | size |
106+
| --- | --- | --- |
107+
| bare Go binary | (none, no ts import) | 2.29 MB |
108+
| subset, Go only | `grammar_subset grammar_subset_go` | **9.10 MB** |
109+
| subset, Go + Python | `grammar_subset grammar_subset_go grammar_subset_python` | 9.18 MB |
110+
| umbrella (all 206) | (none) | 30.83 MB |
111+
112+
All four highlight the same snippet to the same 14 spans (comment 1,
113+
keyword 4, property 1, string 2, type 2, variable 4). Findings:
114+
115+
- **Subset-embedding does escape the 542 inits.** Under `grammar_subset`,
116+
`registry_builtin_gen.go` is tag-excluded and `registry_builtin_subset.go`
117+
makes `registerBuiltinLanguages()` a no-op; only the wired
118+
`z_subset_registry_register_<lang>.go` init runs. 9.10 MB vs 30.83 MB is
119+
a 70% / 21.7 MB cut. The naive umbrella import is off the table; the
120+
build-tag subset is the path.
121+
- **The runtime is the fixed cost; grammars are nearly free.** Go-only over
122+
bare is ~6.8 MB (the pure-Go parser/query engine). A second grammar
123+
(Python) adds ~0.08 MB. Per-language cost is the ~hundreds-of-KB the
124+
design predicted, so wiring more languages later is cheap; the one-time
125+
~6.8 MB runtime is the number that has to clear the north star.
126+
- **No runtime blob dir, no WASM fallback needed.** `grammar_subset` embeds
127+
the chosen blobs into the binary, so there is no
128+
`GOTREESITTER_GRAMMAR_BLOB_DIR` to ship beside `nook` and no wazero
129+
dependency to pay for. Slice-1 succeeds; slices 2 (disk blobs) and the
130+
WASM family stay unused.
131+
132+
Net for nook: adding tree-sitter-go pushes the binary from ~15 MB toward
133+
~21-22 MB before any chroma weight is reclaimed. Whether that ~6.8 MB
134+
one-time add clears "lightest, fastest terminal IDE" is the real go/no-go,
135+
and it is now a decision on measured numbers rather than an unknown. The
136+
next slice measures nook-with-`tshl` minus nook-with-chroma to get the true
137+
net add, then wires `tshl` behind the capture-name -> style-class contract.
138+
139+
Repro: `/tmp/tshl-spike` (throwaway module, gotreesitter v0.20.9); nook's
140+
own build is untouched.
141+
142+
## Open questions (updated)
143+
144+
- ~~Does subset-embedding one grammar avoid the 542 init registrations?~~
145+
**Resolved: yes** (9.10 MB vs 30.83 MB, above).
146+
- ~~Is upstream willing to expose DCE-friendly per-grammar registration?~~
147+
**Moot: already shipped.** The `grammar_subset` + `grammar_subset_<lang>`
148+
build-tag matrix (upstream issue #88) is exactly that mechanism, with
149+
committed `z_subset_*` files per language. No upstream issue is needed.
107150
- Highlight query dialect: gotreesitter ships `.scm` queries per language;
108151
confirm they match nook's expected capture vocabulary or add a small
109-
capture-name remap in `tshl`.
152+
capture-name remap in `tshl`. (Still open; the Go captures above map
153+
cleanly onto nook's existing highlight classes, so the remap looks small.)

0 commit comments

Comments
 (0)