Skip to content

Commit 1f91d35

Browse files
committed
Make FlashPaste a faster overlay-capable agent surface
Constraint: Preserve the bash fallback and existing clipboard intent behavior while adding overlay checks and reducing daemon hot-path process work. Rejected: Commit generated agent state or the local references checkout | They are runtime/reference artifacts, not project deliverables. Rejected: Per-paste tmux, screenshot, xclip, and trigger-log fan-out | It keeps avoidable subprocess and filesystem work on the hot path. Confidence: medium Scope-risk: broad Directive: Keep overlayd diagnostics structured and keep hot-path probes throttled or opt-in. Tested: cargo fmt --manifest-path rs/Cargo.toml --all --check; RUSTC_WRAPPER= cargo clippy --manifest-path rs/Cargo.toml --workspace --offline -- -D warnings; RUSTC_WRAPPER= cargo test --manifest-path rs/Cargo.toml --workspace --offline; bash -n bin/flashpaste-doctor.sh bin/tmux-paste-dispatch.sh bin/flashpaste-screenshot-preload.sh install.sh bootstrap.sh packaging/build-deb.sh packaging/build-rpm.sh; git diff --cached --check; bash bin/flashpaste-doctor.sh Not-tested: Live Wayland layer-shell draw on this non-Wayland shell; AGENTS-release-check still reports historical missing tags/releases for older v1.10-v1.27 history.
1 parent 39f8635 commit 1f91d35

71 files changed

Lines changed: 6427 additions & 304 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/release.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
- name: Install build dependencies
2020
run: |
2121
sudo apt-get update -qq
22-
sudo apt-get install -y dpkg-dev lintian libwayland-dev libx11-dev libdbus-1-dev pkg-config
22+
sudo apt-get install -y dpkg-dev lintian libwayland-dev libx11-dev libdbus-1-dev pkg-config libcairo2-dev libglib2.0-dev libpango1.0-dev
2323
2424
- name: Resolve version from tag
2525
id: ver
@@ -46,6 +46,9 @@ jobs:
4646
- name: Build Rust workspace
4747
run: cargo build --release --manifest-path rs/Cargo.toml
4848

49+
- name: Build overlay daemon
50+
run: cargo build --release --manifest-path rs/Cargo.toml -p flashpaste-overlayd --features wayland
51+
4952
- name: Build .deb
5053
run: VERSION=${{ steps.ver.outputs.v }} bash packaging/build-deb.sh
5154

CHANGELOG.md

Lines changed: 17 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,25 +6,23 @@ Release-tag policy: every `vX.Y` commit on `main` must be tagged and have a matc
66

77
## [Unreleased]
88

9+
## [1.32] - 2026-05-20
10+
911
### Added
1012

11-
- Unified `flashpaste` CLI wrapping the six binaries (`dispatch`, `trigger`, `shoot`, `doctor`, `trace`, `mcp`)
12-
- `flashpaste-shoot --ocr` and `--ocr-only` flags for portal screenshot capture
13-
- `flashpaste-shoot --annotate` for screenshot annotation
14-
- `flashpasted` auto-compresses staged screenshots on the inotify path
15-
- `flashpaste-common` image auto-compress module
16-
- Doctor + docs surface OCR / annotate / auto-compress to users
17-
- `docs/` tree (architecture, install, troubleshooting, FAQ, comparison, use-cases, glossary)
18-
- `llms.txt` (AI-crawler manifest per llmstxt.org)
19-
- `CITATION.cff`
20-
- Distribution packaging: AUR PKGBUILDs (stable + git), Homebrew tap formula, Nix flake
21-
- Project-health files: `CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md`, this `CHANGELOG.md`, issue + PR templates, `FUNDING.yml`
22-
- CI: shellcheck + markdownlint + link-check workflow
23-
- Architecture Decision Records under `docs/adr/`
24-
- Reproducible benchmark suite (`bin/flashpaste-bench.sh`, `make bench`)
25-
- `assets/og-image.svg` for social sharing
26-
- `assets/hero-flow-light.svg` light-mode variant
27-
- README badges, animated SVG hero, animated tier-comparison chart, Mermaid sequence diagram, AI-assistant TL;DR block, extended FAQ, alternatives comparison
13+
- `flashpaste-overlayd`, a Wayland screen-annotation daemon for agent-driven highlights, pointers, labels, and fade-out TTLs.
14+
- Three MCP overlay tools in `flashpaste-mcp`: `highlight_region`, `point_at`, and `clear_annotations`.
15+
- GNOME-compatible overlay fallback path for environments without layer-shell support.
16+
- Multi-monitor overlay support so annotations map to the visible output geometry instead of assuming a single screen.
17+
18+
### Changed
19+
20+
- `flashpaste-doctor` now runs 17 core checks, up from the previous 13, including overlay daemon, socket, compositor, and draw-path probes.
21+
22+
### Notes
23+
24+
- Compositor support matrix: wlroots compositors with layer-shell get the native layer-surface path; GNOME/Mutter uses the fallback path; other compositors should run `flashpaste-overlayd --probe` before relying on MCP annotations.
25+
- GNOME fallback caveat: annotations are still programmatic and click-through by design, but Mutter does not expose the same layer-shell semantics as wlroots, so placement and stacking should be treated as best-effort until the probe and quickstart workflow pass on the target desktop.
2826

2927
## [1.26] - 2026-05-19
3028

@@ -172,7 +170,8 @@ Release-tag policy: every `vX.Y` commit on `main` must be tagged and have a matc
172170

173171
Initial commit: sub-120 ms bash hot path for image-paste into GNOME Wayland TUIs.
174172

175-
[Unreleased]: https://github.com/NagyVikt/flashpaste/compare/v1.19...HEAD
173+
[Unreleased]: https://github.com/NagyVikt/flashpaste/compare/v1.32...HEAD
174+
[1.32]: https://github.com/NagyVikt/flashpaste/compare/v1.26...v1.32
176175
[1.19]: https://github.com/NagyVikt/flashpaste/releases/tag/v1.19
177176
[1.18]: https://github.com/NagyVikt/flashpaste/releases/tag/v1.18
178177
[1.17]: https://github.com/NagyVikt/flashpaste/releases/tag/v1.17

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ FlashPaste is a clipboard glue tool for **GNOME Wayland + kitty + tmux**. Patche
1111
```bash
1212
git clone https://github.com/NagyVikt/flashpaste.git
1313
cd flashpaste
14-
./bin/flashpaste-doctor.sh # 13-probe pre-flight; touches nothing
14+
./bin/flashpaste-doctor.sh # 17-check pre-flight; touches nothing
1515
./install.sh # symlinks into ~/.local/bin/, drops user systemd units
1616
```
1717

Makefile

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
#
33
# Useful targets:
44
# make deb build .deb package into dist/
5+
# make rpm build .rpm package into dist/
56
# make install run install.sh
67
# make doctor run environment check
78
# make uninstall remove symlinks (best-effort)
@@ -12,11 +13,12 @@ REPO_ROOT := $(shell pwd)
1213
VERSION ?= $(shell git describe --tags --abbrev=0 2>/dev/null | sed 's/^v//')
1314
VERSION := $(if $(VERSION),$(VERSION),0.1.0)
1415

15-
.PHONY: deb install doctor uninstall clean release-deb help bench bench-tier1 bench-tier3 bench-markdown
16+
.PHONY: deb rpm install doctor uninstall clean release-deb help bench bench-tier1 bench-tier3 bench-markdown
1617

1718
help:
1819
@echo "flashpaste — make targets:"
1920
@echo " make deb build .deb (VERSION=$(VERSION))"
21+
@echo " make rpm build .rpm (VERSION=$(VERSION))"
2022
@echo " make install run install.sh (symlinks into ~/.local/bin)"
2123
@echo " make doctor environment check"
2224
@echo " make uninstall remove symlinks"
@@ -30,6 +32,9 @@ help:
3032
deb:
3133
VERSION=$(VERSION) bash packaging/build-deb.sh
3234

35+
rpm:
36+
VERSION=$(VERSION) bash packaging/build-rpm.sh
37+
3338
install:
3439
bash install.sh
3540

@@ -41,14 +46,14 @@ uninstall:
4146
@for f in tmux-paste-dispatch.sh clipboard-set.sh clipboard-janitor.sh \
4247
get-clipboard-text.sh clip-pipeline-log.sh screenshot-to-clipboard \
4348
flashpaste-screenshot-preload.sh flashpaste-doctor.sh \
44-
flashpaste-trace.sh wl-paste; do \
49+
flashpaste-trace.sh wl-paste flashpaste-overlayd flashpaste-overlay; do \
4550
[ -L "$$HOME/.local/bin/$$f" ] && rm "$$HOME/.local/bin/$$f" && echo " removed ~/.local/bin/$$f" || true; \
4651
done
4752
@for f in wl-clipboard.desktop wl-paste.desktop wl-copy.desktop; do \
4853
[ -L "$$HOME/.local/share/applications/$$f" ] && rm "$$HOME/.local/share/applications/$$f" && echo " removed ~/.local/share/applications/$$f" || true; \
4954
done
5055
@[ -L "$$HOME/paste_image.sh" ] && rm "$$HOME/paste_image.sh" && echo " removed ~/paste_image.sh" || true
51-
@systemctl --user disable --now clipboard-janitor.service flashpaste-screenshot-watcher.path 2>/dev/null || true
56+
@systemctl --user disable --now clipboard-janitor.service flashpaste-screenshot-watcher.path flashpaste-overlayd.service 2>/dev/null || true
5257
@echo "uninstall done"
5358

5459
clean:

README.md

Lines changed: 44 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,17 @@ The missing clipboard glue between **GNOME · kitty · tmux** and your terminal
1313

1414
<img src="assets/hero-animated.svg" alt="Animated diagram: a screenshot leaves the PrtScr key, travels through a glowing pipe past the mutter, kitty, and tmux layers, and lands in a Claude Code terminal as an attached image in 15 milliseconds — visualizing how FlashPaste pastes screenshots into terminal LLM agents on GNOME Wayland" width="100%">
1515

16+
<!--
17+
overlay-demo.gif slot — see "Agent-driven screen annotation" section below
18+
and `docs/overlay-quickstart.md#record-the-demo-gif`. The asset lives at
19+
`assets/overlay-demo.gif` (≤5 MB, 10–15 s). Uncomment the <img> tag below
20+
once the file is committed; until then the placeholder keeps the README
21+
free of broken images.
22+
-->
23+
<!--
24+
<img src="assets/overlay-demo.gif" alt="Demo GIF: user pastes a screenshot into Claude Code, Claude replies 'the bug is in this function' and a red box appears around the relevant code in the user's editor, the user clicks straight to it — driven by flashpaste-mcp's highlight_region tool and flashpaste-overlayd" width="100%">
25+
-->
26+
1627
<br>
1728

1829
<!-- Badges, grouped: project · stack · ecosystem -->
@@ -130,22 +141,39 @@ FlashPaste is **MIT** — fork it, vendor it, ship it.
130141
## flashpaste-mcp — agents get a first-class API
131142

132143
<div align="center">
133-
<img src="assets/mcp-flow.svg" alt="Animated sequence diagram of flashpaste-mcp: a terminal LLM agent on the left (Claude Code, Codex CLI, Aider) sends four MCP tool calls take_screenshot, read_clipboard, copy_text, paste_to_pane along colored lanes to the flashpaste-mcp server on the right" width="100%">
144+
<img src="assets/mcp-flow.svg" alt="Animated sequence diagram of flashpaste-mcp: a terminal LLM agent on the left (Claude Code, Codex CLI, Aider) sends MCP tool calls including take_screenshot, read_clipboard, copy_text, paste_to_pane, highlight_region, point_at, and clear_annotations along colored lanes to the flashpaste-mcp server on the right" width="100%">
134145
</div>
135146

136-
`flashpaste-mcp` is a drop-in [Model Context Protocol](https://modelcontextprotocol.io) server. Any MCP-aware client ([Claude Code](https://www.anthropic.com/claude-code), [Codex CLI](https://github.com/openai/codex), [Aider](https://github.com/Aider-AI/aider), [Continue](https://github.com/continuedev/continue), ) picks it up automatically and gets four tools:
147+
`flashpaste-mcp` is a drop-in [Model Context Protocol](https://modelcontextprotocol.io) server. Any MCP-aware client ([Claude Code](https://www.anthropic.com/claude-code), [Codex CLI](https://github.com/openai/codex), [Aider](https://github.com/Aider-AI/aider), [Continue](https://github.com/continuedev/continue), ...) picks it up automatically and gets clipboard, screenshot, paste, and screen annotation tools:
137148

138149
| Tool | What it does |
139150
|---|---|
140151
| `take_screenshot` | Capture via `xdg-desktop-portal` → returns PNG bytes + MIME |
141152
| `read_clipboard` | Read the current Wayland selection (text or image) with MIME |
142153
| `copy_text` | Place text on the clipboard via the stable daemon owner |
143154
| `paste_to_pane` | Send a paste to a specific kitty/tmux pane by id, even unfocused |
155+
| `highlight_region` | Draw a temporary rectangle or circle highlight on the user's visible screen |
156+
| `point_at` | Draw a temporary pointer arrow toward a specific screen location |
157+
| `clear_annotations` | Clear active overlay highlights, pointers, and labels |
144158

145159
Drop it in your client config — see [`docs/architecture.md`](docs/architecture.md#mcp).
146160

147161
---
148162

163+
## Agent-driven screen annotation
164+
165+
Once the agent can *see* your screen (via `take_screenshot`), the next obvious move is to let it *point at things*. `flashpaste-overlayd` is a tiny Rust daemon that paints temporary boxes, arrows, and labels directly on your Wayland screen, driven by three MCP tools that ship in the same `flashpaste-mcp` server — `highlight_region`, `point_at`, `clear_annotations`. Claude reads your screenshot, decides "the bug is in this function," draws a red box around it, and you click straight there. Annotations are click-through by default, fade out on a TTL, and never block your input.
166+
167+
```jsonc
168+
// One MCP call from the agent → red box appears on screen for 4 seconds.
169+
{"name": "highlight_region",
170+
"arguments": {"shape": "rect", "x": 412, "y": 318, "w": 280, "h": 96, "color": "#ff3b30", "label": "the bug is in this function", "ttl_ms": 4000}}
171+
```
172+
173+
Setup, the GNOME caveat, and the recording recipe live in [`docs/overlay-quickstart.md`](docs/overlay-quickstart.md).
174+
175+
---
176+
149177
## How to install FlashPaste on Linux
150178

151179
### One-line bootstrap (any distro)
@@ -160,7 +188,7 @@ Then wire up your dotfiles, reload, and verify:
160188
cat ~/.local/share/flashpaste/examples/tmux.conf.snippet >> ~/.tmux.conf
161189
cat ~/.local/share/flashpaste/examples/kitty.conf.snippet >> ~/.config/kitty/kitty.conf
162190
tmux source-file ~/.tmux.conf # then restart kitty
163-
flashpaste-doctor # 13 green checks = ready
191+
flashpaste-doctor # 17 core checks = ready
164192
```
165193

166194
<details>
@@ -290,7 +318,7 @@ Yes — see [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`AGENTS.md`](AGENTS.md). T
290318
- [`CHANGELOG.md`](CHANGELOG.md) — release history (Keep-a-Changelog).
291319
- [`AGENTS.md`](AGENTS.md) — contributor + AI-agent guide; release workflow lives here.
292320
- [`llms.txt`](llms.txt) — AI-crawler manifest ([llmstxt.org](https://llmstxt.org) standard).
293-
- [`flashpaste-mcp`](docs/architecture.md) — MCP server: `take_screenshot`, `read_clipboard`, `copy_text`, `paste_to_pane`.
321+
- [`flashpaste-mcp`](docs/architecture.md) — MCP server: `take_screenshot`, `read_clipboard`, `copy_text`, `paste_to_pane`, `highlight_region`, `point_at`, `clear_annotations`.
294322

295323
MIT — see [LICENSE](LICENSE). Built by [@NagyVikt](https://github.com/NagyVikt).
296324

@@ -313,7 +341,7 @@ If FlashPaste saved you a Wayland headache, a [GitHub star](https://github.com/N
313341
"@type": "SoftwareApplication",
314342
"name": "FlashPaste",
315343
"alternateName": ["flashpaste", "flash-paste"],
316-
"description": "Sub-15 ms image-paste glue for terminal AI agents (Claude Code, Codex CLI, Aider, llm) on GNOME Wayland. Works around mutter's surfaceless-client clipboard refusal, kitty's map ctrl+v interception, and tmux's bind -n C-v recursion via three progressive performance tiers.",
344+
"description": "Sub-15 ms image-paste glue and agent overlay for terminal AI agents (Claude Code, Codex CLI, Aider, llm) on GNOME Wayland. Works around mutter's surfaceless-client clipboard refusal, kitty's map ctrl+v interception, and tmux's bind -n C-v recursion via three progressive performance tiers, then lets MCP clients annotate the visible screen with highlight_region, point_at, and clear_annotations.",
317345
"url": "https://github.com/NagyVikt/flashpaste",
318346
"codeRepository": "https://github.com/NagyVikt/flashpaste",
319347
"downloadUrl": "https://github.com/NagyVikt/flashpaste/releases/latest",
@@ -323,6 +351,14 @@ If FlashPaste saved you a Wayland headache, a [GitHub star](https://github.com/N
323351
"license": "https://spdx.org/licenses/MIT.html",
324352
"programmingLanguage": ["Rust", "Bash"],
325353
"softwareRequirements": ["kitty", "tmux", "wl-clipboard", "xclip", "ydotool"],
354+
"featureList": [
355+
"Sub-15 ms screenshot paste into terminal LLM agents on GNOME Wayland",
356+
"MCP server for screenshot capture, clipboard reads, text copy, and pane paste",
357+
"Agent overlay for screen annotation on visible Linux UI regions",
358+
"highlight_region draws temporary rectangles or circles on the user's screen",
359+
"point_at draws temporary screen pointer arrows for buttons, panels, and app regions",
360+
"clear_annotations removes active overlay highlights, pointers, and labels"
361+
],
326362
"offers": { "@type": "Offer", "price": "0", "priceCurrency": "USD" },
327363
"author": {
328364
"@type": "Person",
@@ -334,6 +370,8 @@ If FlashPaste saved you a Wayland headache, a [GitHub star](https://github.com/N
334370
"claude code", "codex cli", "aider", "llm cli",
335371
"image paste linux", "screenshot paste terminal",
336372
"mcp server", "model context protocol",
373+
"screen annotation", "agent overlay", "mcp screen pointer", "linux ai overlay",
374+
"highlight_region", "point_at", "clear_annotations",
337375
"terminal ai", "llm agent", "wayland clipboard fix",
338376
"paste screenshot claude code linux",
339377
"ubuntu 24.04 wayland clipboard bug"
@@ -404,7 +442,7 @@ If FlashPaste saved you a Wayland headache, a [GitHub star](https://github.com/N
404442
"name": "Does FlashPaste include an MCP (Model Context Protocol) server?",
405443
"acceptedAnswer": {
406444
"@type": "Answer",
407-
"text": "Yes. flashpaste-mcp is a drop-in MCP server that any MCP-aware client (Claude Code, Codex CLI, Aider, Continue) picks up automatically. It exposes four tools: take_screenshot (capture via xdg-desktop-portal), read_clipboard (current Wayland selection with MIME), copy_text (place text on the clipboard via the daemon owner), and paste_to_pane (send a paste to a specific kitty or tmux pane by id, even when unfocused)."
445+
"text": "Yes. flashpaste-mcp is a drop-in MCP server that any MCP-aware client (Claude Code, Codex CLI, Aider, Continue) picks up automatically. It exposes tools for take_screenshot (capture via xdg-desktop-portal), read_clipboard (current Wayland selection with MIME), copy_text (place text on the clipboard via the daemon owner), paste_to_pane (send a paste to a specific kitty or tmux pane by id, even when unfocused), and overlay screen annotation through highlight_region, point_at, and clear_annotations."
408446
}
409447
}
410448
]

THIRD_PARTY.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Third-Party Notices
2+
3+
This file records third-party projects directly referenced by the FlashPaste
4+
overlay work. FlashPaste itself is MIT licensed; see [`LICENSE`](LICENSE).
5+
6+
## Runtime Dependencies
7+
8+
### smithay-client-toolkit
9+
10+
- Project: [Smithay client-toolkit](https://github.com/Smithay/client-toolkit)
11+
- Use: Wayland client and layer-shell plumbing for `flashpaste-overlayd`.
12+
- License: MIT.
13+
- Authors credited by the crate metadata: Elinor Berger, i509VCB, and Ashley
14+
Wulber. The license text in the crate also carries Victor Berger's copyright.
15+
16+
### cairo-rs, cairo-sys-rs, and pangocairo
17+
18+
- Project: [gtk-rs core bindings](https://github.com/gtk-rs/gtk-rs-core)
19+
- Use: Rust bindings used by the optional overlay renderer.
20+
- License: MIT.
21+
- Authors credited by the crate metadata: The gtk-rs Project Developers.
22+
23+
### Cairo graphics library
24+
25+
- Project: [Cairo](https://www.cairographics.org/)
26+
- Use: Underlying 2D graphics library used through the Rust Cairo bindings.
27+
- License: LGPL-2.1 or MPL-1.1, at the user's option.
28+
- Credit: Cairo authors and contributors.
29+
30+
## Reference Projects
31+
32+
### Gromit-MPX
33+
34+
- Project: [Gromit-MPX](https://github.com/bk138/gromit-mpx)
35+
- Use: Reference for mature human-driven screen annotation behavior.
36+
- License: GPL-2.0.
37+
- Credit: Simon Budig, Christian Beier, Barak A. Pearlmutter, and the
38+
Gromit-MPX contributors listed in `references/gromit-mpx/AUTHORS`.
39+
- Note: FlashPaste does not copy or vendor Gromit-MPX code.
40+
41+
### wayscriber
42+
43+
- Project: [wayscriber](https://github.com/devmobasa/wayscriber)
44+
- Use: Reference for Wayland annotation UX, Smithay client-toolkit usage, Cairo
45+
rendering, and compositor-specific behavior.
46+
- License: MIT.
47+
- Credit: devmobasa and wayscriber contributors.
48+
- Note: FlashPaste does not copy or vendor wayscriber code.

0 commit comments

Comments
 (0)