Skip to content

feat(editor): keystroke overlay — render layer + persistence (Part 2/3)#1

Merged
starkdcc merged 8 commits into
feat/keystroke-overlay-foundationfrom
feat/keystroke-overlay-ui
Jul 11, 2026
Merged

feat(editor): keystroke overlay — render layer + persistence (Part 2/3)#1
starkdcc merged 8 commits into
feat/keystroke-overlay-foundationfrom
feat/keystroke-overlay-ui

Conversation

@starkdcc

Copy link
Copy Markdown
Owner

Keystroke overlay — Part 2 of 3 (render layer + persistence)

Stacked on webadderallorg#733 (capture + display foundation). Based on the feat/keystroke-overlay-foundation branch so this diff shows only Part 2's changes — review webadderallorg#733 first.

What's in this PR (all tsc + Biome clean; 25 pure-core tests pass)

  • Settings persistenceshowKeystrokes / keystrokePosition / keystrokeSize added to ProjectEditorState + editorPreferences, normalized and defaulted (off / bottom-center / 1×), mirroring the cursor prefs.
  • PixiKeystrokeOverlay — a self-contained PIXI render layer mirroring PixiCursorOverlay's construct/update/reset/destroy contract, driven entirely by the tested coalescing core. Pooled Graphics + Text keycaps, screen-anchored (bottom/top, L/C/R), size-scaled, per-group fade. Type-checked against PixiJS v8.
  • detectGlyphStyle() — platform-aware ⌘/Ctrl/Win/Super (guarded for Node).
  • Keystroke sidecar persistence primitivesgetKeystrokePathForVideo (.keystrokes.json) + normalize/write/persistPending/snapshot keystroke telemetry, exact mirrors of the cursor telemetry functions.

Remaining wiring (Part 3/3 — needs a local dev loop to verify)

The pieces below are pure integration/threading. I have exact anchors for each (from a code-map pass); they compile in isolation but must be verified in npm run dev:

  • Enable capture: thread a { showKeystrokes } option through preload.setRecordingState + electron-env.d.ts + the set-recording-state handler → call the (already-existing) setIsKeystrokeCaptureActive(...) in the start branch; setIsKeystrokeCaptureActive(false) + snapshotKeystrokeTelemetryForPersistence() in every stop branch (native-win @994/996, set-recording-state stop @1838, + any mac/ffmpeg stop paths — grep snapshotCursorTelemetryForPersistence).
  • Read path: add get-keystroke-telemetry IPC (mirror get-cursor-telemetry) + preload.getKeystrokeTelemetry + electron-env.d.ts type.
  • Renderer state: VideoEditorkeystrokeTelemetry state + a loadKeystrokeTelemetry effect (mirror the videoSourcePath-keyed cursor effect including its fresh-recording retry loop), then pass keystrokeTelemetry + showKeystrokes/position/size to VideoPlayback.
  • VideoPlayback: declare the props; create a keystrokeContainer sibling of cursorContainer; instantiate PixiKeystrokeOverlay; attach at both mount sites (~2197/2297); .update(samples, timeMs, baseMaskRef.current, showKeystrokes, freeze) per frame (~2547); .destroy() in cleanup.
  • Settings UI: SettingsPanel case "cursor" — a "Show Keystrokes" Switch + position Select + size SliderControl, 3 props + defaults, and effects.* i18n keys in all 10 locale settings.json files.
  • Sidecar cleanup: unlink the .keystrokes.json alongside the cursor sidecar in project.ts + prune.ts.
  • (optional) export parity: draw the overlay in frameRenderer.ts / modernFrameRenderer.ts so keystrokes appear in MP4/GIF exports.

Manual test plan (once Part 3 lands)

  1. Enable "Show Keystrokes"; record; type text + shortcuts (⌘C / Ctrl+Shift+P); stop.
  2. In the editor: keycaps appear at the chosen anchor, chords group, rapid typing merges into a word, groups fade after ~1.2s, and scrubbing reveals keys at the right times.
  3. Reopen the .recordly project → overlay replays (sidecar persisted).
  4. Export MP4/GIF → keystrokes burned in (if export parity added).
  5. Privacy: with the toggle off, confirm no .keystrokes.json is written.

Notes

  • Privacy by design: capture is gated on a default-off flag; nothing is recorded unless the overlay is enabled.
  • Developed with AI assistance (disclosed in commit trailers). Happy to reshape scope or fold parts together.

starkdcc and others added 8 commits July 11, 2026 13:15
…layer (2/3)

Builds on the capture + display foundation (webadderallorg#733):
- projectPersistence + editorPreferences: persist showKeystrokes /
  keystrokePosition / keystrokeSize (mirrors the cursor prefs), normalized and
  defaulted (off / bottom-center / 1x).
- PixiKeystrokeOverlay: a parent-agnostic PIXI layer mirroring PixiCursorOverlay's
  construct/update/reset/destroy contract, driven entirely by the tested
  coalescing core; pooled Graphics + Text keycaps, screen-anchored (bottom/top),
  size-scaled, with per-group fade.
- detectGlyphStyle(): platform-aware modifier glyphs (guarded for Node tests).

tsc clean (electron + renderer); 25 pure-core tests pass. Remaining wiring
(settings-panel controls + parent threading, VideoPlayback instantiation,
keystroke sidecar persistence + IPC, export parity) needs a local dev loop to
verify and is tracked as the follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the main-process persistence layer for keystroke telemetry, mirroring the
cursor .cursor.json sidecar exactly:
- utils: getKeystrokePathForVideo -> `${videoPath}.keystrokes.json`
- telemetry: normalizeKeystrokeTelemetrySamples, writeKeystrokeTelemetry,
  persistPendingKeystrokeTelemetry, snapshotKeystrokeTelemetryForPersistence
  (use KEYSTROKE_TELEMETRY_VERSION / MAX_KEYSTROKE_SAMPLES; skip/remove the
  sidecar when empty, matching the cursor path).

tsc (node) + biome clean. Wiring these into set-recording-state, the stop
paths, a get-keystroke-telemetry IPC channel, and the renderer load path is
the remaining follow-up (see PR description checklist).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e prefs

Adding showKeystrokes/keystrokePosition/keystrokeSize to PersistedEditorControls
left captureEditorPresetSnapshot's EditorPresetSnapshot literal missing them.
Add the three fields (placeholder defaults until VideoEditor manages keystroke
settings in part 3). Full-project tsc is now clean except one pre-existing
unused import (VideoPlayback scalePreviewBorderRadius) that predates this branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- recording.ts: enable keystroke capture at record-start when showKeystrokes is
  set (new set-recording-state options arg); disable + snapshot on all stop
  paths (native-win + set-recording-state); add get-keystroke-telemetry read
  handler mirroring the cursor one.
- preload + electron-env: getKeystrokeTelemetry bridge, setRecordingState
  options, KeystrokeTelemetryPoint type.

tsc (node) clean. Renderer wiring (overlay mount + settings toggle) is the
remaining follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…derer)

Completes the keystroke overlay end to end:
- VideoEditor: keystrokeTelemetry state + load effect (get-keystroke-telemetry),
  showKeystrokes/position/size state, props to VideoPlayback + SettingsPanel.
- VideoPlayback: mount PixiKeystrokeOverlay in a sibling container, per-frame
  update (outside the cursor-overlay branch so it renders independently), plus
  attach/destroy lifecycle; drop a pre-existing unused import.
- SettingsPanel: 'Show Keystrokes' toggle + size slider.
- useScreenRecorder: pass showKeystrokes to setRecordingState at record start.
- i18n (en): effects.showKeystrokes / effects.keystrokesSize.

Full-project tsc clean (both configs); 25 keystroke tests pass; Electron dev
build boots and renderer renders with zero console errors. Position Select and
the other 9 locale strings are trivial follow-ups.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ext recording

Add showKeystrokes/keystrokePosition/keystrokeSize to the saveEditorPreferences
bundle (+ effect deps). Without this, toggling 'Show Keystrokes' only affected
the in-editor overlay; capture is gated at record-start on the persisted pref
(loadEditorPreferences().showKeystrokes), so the toggle must persist to actually
capture keystrokes on the next recording. Closes the end-to-end loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move 'Show Keystrokes' out of the cramped 3-toggle header row into its own
full-width labeled row at the top of the cursor panel body, with a full-size
switch (was scale-75) so its on/off state is obvious. Header returns to
Show Cursor + Loop cursor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@starkdcc starkdcc marked this pull request as ready for review July 11, 2026 17:48
@starkdcc starkdcc merged commit 662731e into feat/keystroke-overlay-foundation Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant