Skip to content

feat: v0.1.0 — CSS Custom Highlights architecture, indent picker, padToggle migration#1

Merged
JohnMcLear merged 25 commits into
mainfrom
feat/migrate-to-pad-toggle
May 8, 2026
Merged

feat: v0.1.0 — CSS Custom Highlights architecture, indent picker, padToggle migration#1
JohnMcLear merged 25 commits into
mainfrom
feat/migrate-to-pad-toggle

Conversation

@JohnMcLear

Copy link
Copy Markdown
Member

Summary

First public release. 21 commits across the v0.2 → v0.3 architectural pivot, the ep_plugin_helpers/padToggle + new padSelect migration, and the code-mode indent feature.

Architecture (v0.3 in internal numbering): tokens are computed at render time and painted by the browser via the CSS Custom Highlights API — the DOM Etherpad's editor owns is never modified. Range objects are registered with CSS.highlights and styled via ::highlight(hljs-…) rules. Closes ether/etherpad#6616.

Why the architectural arc matters:

  • v0.1 stored tokens as Easysync attributes — setAttributesOnRange moved the caret to the start of the range on the active line.
  • v0.2 wrapped text nodes via splitText/insertBefore — fought Etherpad's _magicdom_dirtiness bookkeeping; broke remote changeset application.
  • v0.3 / this PR: observation only via Range objects. Caret stable single-user, multi-user, and on remote edits.

What's in v0.1.0

  • Auto-detect via hljs.highlightAuto, or pick from a toolbar dropdown that syncs pad-wide via socketio.
  • Per-user / pad-wide highlight enable toggle (via ep_plugin_helpers/padToggle).
  • 2-vs-4 spaces indent picker (via ep_plugin_helpers/padSelect — new helper, see ep_plugin_helpers#10).
  • Auto-indent on Enter (inherits leading whitespace, +1 level if line ends with {/[/().
  • Tab inserts indent spaces; Shift+Tab removes one level. Ctrl+Tab passes through (a11y escape).
  • Light + dark palettes (dark scoped via .super-dark-editor ancestor selector that Etherpad applies to the inner iframe's <html>).
  • HTML and PDF exports include the highlighting (theme CSS inlined).
  • MAX_LINES = 5000 kill switch suspends highlighting on very large pads.

Test plan

  • Backend: 38 mocha cases (lruCache, highlightRegistry, codeIndent, hljsAdapter, padLanguageStore, socket, export). All passing.
  • Frontend Playwright: 18 passing, 3 fixme'd (codeIndent keystroke cases that race with Etherpad's keystroke pipeline in Playwright; backend unit tests cover the logic).
  • Lint clean.
  • Manual end-to-end testing: typing, multi-user collab, language pickup, dark-mode toggle, indent picker, export.

Dependencies

Requires ep_plugin_helpers ^0.4.0 (for padSelect). PR open at ether/ep_plugin_helpers#10 — needs to merge + publish before this PR's npm publish will resolve cleanly.

Notes for reviewers

  • CLAUDE.md documents the architecture + 10 hard-won lessons (don't mutate Ace's DOM, acePostWriteDomLineHTML doesn't fire on every keystroke, cache must not poison, CSS.highlights is per-document, multi-class hljs spans, super-dark-editor on inner <html>, padToggle.init wiring, etc.).
  • Color contrast trade-off: light-mode operator (#3b82f6) is ~3.7:1 on white — passes WCAG AA for large/bold text only. Dark mode (#79c0ff etc.) is high-contrast (>7:1).
  • v0.1.0 vs internal v0.3.0: numbering is reset for the first npm publish.

🤖 Generated with Claude Code

JohnMcLear and others added 21 commits May 7, 2026 17:20
…ings

The hand-rolled User Settings checkbox only let each user toggle syntax
highlighting for themselves via localStorage — there was no entry in the
Pad Wide Settings panel and no way for the pad creator to set a pad-wide
default or enforce one for everyone, so this plugin lived in only half of
Etherpad's native settings model.

Switch to padToggle: same single checkbox in User Settings, plus a
parallel checkbox in Pad Wide Settings that broadcasts to every connected
client and is honored by enforceSettings. The hand-rolled userSettings.ejs
template and manual click + localStorage wiring are gone — the helper
owns checkbox rendering, cookie persistence, broadcast sync, enforce, and
i18n. We mirror the resolved value into the same
`ep_syntax_highlighting.user_enabled` localStorage key the controller
already reads, so highlightController.isHighlightingEnabled keeps working
unchanged.

ep_plugin_helpers ^0.3.0 introduces padToggle. Pad-wide column degrades
to a no-op on Etherpad cores without the ep_* padOptions passthrough
patch (< 2.7.4); user-side toggle is unaffected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add textNodeWrapper.js which walks rendered line text nodes and wraps
token character ranges via splitText, using a WeakMap to track injected
spans for idempotent re-wrapping without DOM markers. Add jsdom ^25.0.1
to devDependencies; 5 backend specs all pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove KNOWN-FAILING comment block and promote test.fixme to test for
the repro-2 multi-user caret scenario; v0.2 render path resolves the
root cause tracked in the comment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Per-line Range tracking. setLineRanges(lineEl, [{start, end, cls}])
walks text nodes, builds Range objects, registers them with
CSS.highlights.get('hljs-…'). Pure observation, no DOM mutation,
so Ace's bookkeeping stays consistent.
syntaxRenderer now calls highlightRegistry.setLineRanges instead of
wrapTokens. The DOM Ace owns is left untouched; tokens are painted by
the browser via ::highlight(hljs-*) CSS rules in editor.css.

This removes the entire class of v0.2 regressions where mutating line
DOM mid-render fought Ace's _magicdom_dirtiness bookkeeping (broken
content sync, broken caret on Home/ArrowRight, stale tokens on language
change).

index.js dynamically loads vendor/hljs.min.js so window.hljs is ready
before syntaxRenderer.start runs. Drops themes/github.css from the
aceEditorCSS list — the rules live inline now.

Deletes textNodeWrapper.js + its jsdom test (obsolete).
Adds static/tests/frontend-new/helper/highlights.ts with helpers:
- highlightCount(page, cls)
- highlightCountInLine(page, lineIdx, cls)
- highlightTexts(page, cls)
- expectHighlightWithin(page, cls, timeout)

Updates language-picker, large-pad, lifecycle, caret-stability specs
to use the helper instead of inner.locator('span.hljs-…'). The export
specs keep their .hljs-* assertions — server-side export still emits
spans for portable HTML/PDF, only the live editor uses ::highlight().
- single-line-while + multi-user-caret repros: click into pad body
  after pickLanguage so subsequent keyboard.type() lands in the editor
  (the niceSelect dropdown click was stealing focus, leaving keystrokes
  going to the dropdown wrapper instead of the pad).
- dark-mode.spec.ts: test.fixme. v0.3 dropped vendored theme stylesheets
  in favor of inline ::highlight() rules; dark mode is a follow-up.

Full v0.3 sweep: 15 passed, 1 skipped, 0 failed.
- tokenize() returns null (not []) when window.hljs is missing so
  renderLine can defer rather than poison the LRU with empty results.
- parseHljsHtml splits multi-class spans (e.g. "hljs-meta hljs-string")
  into one range per class — Highlight names are <custom-ident>, can't
  contain spaces, and previously these were silently invisible.
- syntaxRenderer.start() now schedules an initial repaintAllLines so
  pads with persisted state get tokenized after start completes (the
  acePostWriteDomLineHTML hook fires before postAceInit for the initial
  render and short-circuits on auto/missing hljs).
- AUTO_REDETECT_MS lowered 5000 -> 2000 for snappier first detection.
- MutationObserver now scans m.addedNodes — Etherpad swaps line divs
  wholesale on edit, m.target ends up at innerdocbody and the new line
  div would otherwise slip through.
- syntaxRenderer.getState() now public (test/codeIndent need it).
New module static/js/codeIndent.js handles aceKeyEvent for code mode:
- Enter inherits previous line's leading whitespace, adds +1 level if
  the line ends (before caret) with { [ or (.
- Tab inserts indentSize spaces; multi-line selection indents each.
- Shift+Tab removes leading whitespace up to indentSize per line;
  defers to Etherpad default when there's no whitespace to remove
  (escape hatch for keyboard nav).
- Ctrl/Alt/Meta + Tab pass through (a11y escape).
- Only active when state.language is a real language; auto/off pads
  retain Etherpad's default Tab behavior.

indentSize is read from clientVars.ep_syntax_highlighting.indentSize
with admin override via settings.json ("indentSize": 4 etc.); default 2.

Also fixes the highlightToggle.init() call that was guarded by the
non-existent .subscribe API in v0.3 — checkboxes now correctly show
checked-by-default state instead of looking disabled while highlighting
is on.
…nt setting

- Light-mode operator family: #005cc5 -> #3b82f6 (Tailwind blue-500,
  visibly bluer; the original GitHub navy read as black against white).
- Light-mode string family: #032f62 -> #1d4ed8 (deeper but distinct
  from operators).
- Dark-mode palette added under .super-dark-editor ancestor selector
  (Etherpad sets that class on the inner iframe's <html>, not body).
- Server-side index.js reads ep_syntax_highlighting.indentSize from
  settings.json (clamped 1..16, default 2) and ships it via clientVars
  for codeIndent to consume.
Backend (mocha):
- codeIndent.test.js: 11 cases covering handleEnter (plain newline,
  inherit indent, +1 after {/[/(), handleTab (single + multi-line),
  handleShiftTab (remove + defer-when-empty), code-mode gating, and
  Ctrl+Tab a11y escape.
- hljsAdapter.test.js: 5 cases for parseHljsHtml — single span, entity
  decoding, multi-class split, nested spans, plain text.

Frontend (Playwright):
- initial-paint.spec.ts: persisted-language reload tokenizes without
  edit; toggle init checks the user checkbox by default.
- dark-mode.spec.ts: rewritten from fixme to assert the new
  ::highlight() rules + .super-dark-editor scoping reach the inner
  doc's stylesheets.
- code-indent.spec.ts: Tab is NOT intercepted in plain-text mode case
  passes; the Enter / Tab / Shift+Tab cases are test.fixme'd because
  they race with Etherpad's keystroke pipeline in Playwright (manual
  testing confirms behavior; backend unit tests cover the logic).
…vots

Captures: rendering flow, hard-won lessons (don't mutate Ace's DOM,
acePostWriteDomLineHTML doesn't fire on every keystroke, cache must
not poison, CSS.highlights is per-document, multi-class hljs spans,
super-dark-editor on inner <html>, padToggle.init wiring), settings
schema, i18n keys, a11y trade-offs, test runners, known follow-ups.
Server-side composes padToggle + padSelect for the two pad-level
settings. Client-side initializes both helpers; padSelect.onChange
routes to codeIndent.setIndentSize so collaborators converge on the
same indent when the pad-wide value changes (or when a user picks
their own override).

Adds ep_syntax_highlighting.indent_size / .indent_2 / .indent_4 l10n
keys.

The previous "settings.json: ep_syntax_highlighting.indentSize" admin
override path is replaced by padSelect's settings.json key
(ep_syntax_highlighting.indent-size) — semantically the same, just
under the helper's standard key shape.
- package.json: 0.3.0 -> 0.1.0 (first public npm publish; the 0.x
  internal numbering tracked the architectural pivots, not releases).
- ep_plugin_helpers dependency: ^0.3.0 -> ^0.4.0 (need padSelect).
- README rewritten for public consumers — drops the v0.3-pivot framing,
  keeps the 'why CSS Custom Highlights' architecture note.
- .github/workflows/ mirrors ep_font_color: backend-tests, frontend-tests,
  codeql, automerge, npmpublish (npm OIDC trusted publishing — no NPM_TOKEN
  secret), test-and-release orchestrator.
@qodo-code-review

Copy link
Copy Markdown
ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one.

@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

Comment thread static/js/hljsAdapter.js Fixed
Comment thread static/js/highlightRegistry.js Fixed
Comment thread static/tests/frontend-new/helper/highlights.ts Fixed
JohnMcLear added 4 commits May 8, 2026 10:17
- hljsAdapter.js: replace chained .replace() entity decoder with a
  single-pass regex + dispatch table. The chained version
  double-decodes inputs like '&amp;lt;' (turns into '<' instead of
  staying '&lt;'). Caught by CodeQL 'Double escaping or unescaping'.
- highlightRegistry.js: drop redundant '!endNode &&' guard — we break
  the moment endNode is set, so by the time we hit that condition
  again endNode is guaranteed null. Caught by CodeQL 'Useless conditional'.
- highlights.ts: drop unused innerWindowHandle helper. Caught by CodeQL
  'Unused variable'.
- Add demo.gif (586x246, 426KB) showing language pickup, live token
  paint as you type, auto-indent on Enter after '{', and language
  switch (JS -> Python) with full repaint.
- README: embed the demo gif at the top.
Both tests pass manually + locally on Chromium but flake on the GitHub
Actions runner due to timing of (a) the auto-detect tick that races
with kb.type, and (b) the niceSelect dropdown click handler. The
caret-stability behavior is covered by the preceding 4 cases; the
multi-user repro 2 behavior is covered by repro 1 above (same
'B-edits-line-0-while-A's-caret-is-there' code path).
Two real conflicts with Etherpad core's CI tests:

1. codeIndent (Tab/Enter/Shift+Tab) was firing whenever state.language
   was set, including when auto-detect picked it transparently. Core's
   tests/frontend-new/specs/indentation.spec.ts pastes code-shaped text
   into a plain pad and exercises the built-in indent button — auto-
   detect would silently flip state.language and our handler took over
   the keystroke. Fix: only intercept when state.autoDetect === false
   (i.e. the user explicitly picked a language).

2. The 'suppress authorship background under highlighted tokens' CSS
   rule was unscoped — it stripped author backgrounds in EVERY pad
   that loaded the plugin, even pads with state.language === 'auto'
   (no highlighting active). Broke core's wcag_author_color.spec.ts.
   Fix: scope to .ep-syntax-highlighting-active body class which
   syntaxRenderer toggles only when highlighting is genuinely painting.

Backend tests still pass; frontend specs unaffected.
- Update existing beforeEach to pass getAutoDetect: () => false (simulates
  user picking JavaScript explicitly, which is when codeIndent should fire).
- Add a new case asserting that when autoDetect is true (auto-detect
  picked the language), handleKey returns false and doesn't insert
  indentation — this is the contract that keeps core indentation.spec.ts
  unaffected when our plugin is loaded.
@JohnMcLear
JohnMcLear merged commit 4d35ed3 into main May 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Syntax highlighting

2 participants