Skip to content

feat: replace selected text in place via macOS Accessibility API#5

Open
antoncoding wants to merge 1 commit into
masterfrom
feat/replace-selection-in-place
Open

feat: replace selected text in place via macOS Accessibility API#5
antoncoding wants to merge 1 commit into
masterfrom
feat/replace-selection-in-place

Conversation

@antoncoding

Copy link
Copy Markdown
Owner

What

One shortcut (default Cmd+M) now transforms the currently selected text in the frontmost app, replacing it in place. No more copy → shortcut → paste round-trip.

Select text anywhere → press the shortcut → the selection is swapped for the AI-transformed version. A ✨ shows in the menu bar while it works; the same completion notification fires as before.

How

Uses the macOS Accessibility API (AXSelectedText on the focused UI element) to read and write the selection directly — the clipboard is never touched.

  • selection.rs (new) — thin AX wrapper: read/write AXSelectedText on the focused element, plus a first-run Accessibility permission prompt.
  • core::transform_selection — permission check → read selection → existing transform_text + tone prompt → write back → history → notification. Reuses all existing transform/history/notification plumbing.
  • lib.rs — global-shortcut handler now calls transform_selection; new command registered.
  • tray.rs — tray shows a ✨ indicator while transforming.

The tray "Transform" menu item keeps the old clipboard flow (clicking a menu steals focus, so a selection-based action can't work there).

Notes

  • New macOS-only deps: accessibility-sys, core-foundation, macos-accessibility-client.
  • Pinned time to 0.3.51 (0.3.52 breaks cookie 0.18.1) and committed Cargo.lock (it was excluded by a global gitignore) so the build is reproducible for everyone.
  • Requires granting Milo Accessibility permission on first use (the app prompts).

Test plan

  • cargo build links cleanly
  • Grant Accessibility, select text in another app, press the shortcut → selection replaced in place
  • Notification + ✨ indicator appear

🤖 Generated with Claude Code

One shortcut now transforms the current selection in the frontmost app
directly — select text, press the shortcut, and it's replaced in place.
No more copy/shortcut/paste round-trip.

- selection.rs: read/write AXSelectedText on the focused element, plus a
  first-run Accessibility permission prompt. Clipboard is untouched.
- core::transform_selection: permission check -> read selection -> existing
  transform_text + tone prompt -> write back -> history -> notification.
- shortcut handler now calls transform_selection; tray shows a ✨ while busy.
- pin time to 0.3.51 (0.3.52 breaks cookie 0.18.1) and commit Cargo.lock so
  the build is reproducible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a one-shortcut flow on macOS that reads and replaces the user's selected text in place using the macOS Accessibility API, bypassing the clipboard. It also adds a visual "transforming" indicator (✨) to the tray menu during execution. The review feedback highlights two critical issues: first, replacing the shortcut handler unconditionally with the macOS-only transform_selection breaks the global shortcut on Windows and Linux; second, the is_transforming state and tray indicator may not be reset if the operation fails or panics, which can be resolved using an RAII guard.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src-tauri/src/lib.rs
let app_handle = app.clone();
tauri::async_runtime::spawn(async move {
if let Err(e) = core::transform_clip_with_setting(app_handle.clone(), true).await {
if let Err(e) = core::transform_selection(app_handle.clone(), true).await {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Regression: Global Shortcut Broken on Windows and Linux

By replacing core::transform_clip_with_setting with core::transform_selection unconditionally in the global shortcut handler, the shortcut is now completely broken on non-macOS platforms (where transform_selection always returns an error).

To preserve cross-platform compatibility, we should conditionally fall back to the clipboard-based transformation (transform_clip_with_setting) on Windows and Linux.

                            #[cfg(target_os = "macos")]
                            let result = core::transform_selection(app_handle.clone(), true).await;
                            #[cfg(not(target_os = "macos"))]
                            let result = core::transform_clip_with_setting(app_handle.clone(), true).await;

                            if let Err(e) = result {

Comment thread src-tauri/src/core.rs
Comment on lines +103 to +116
// Avoid overlapping runs.
{
let mut transforming = state.is_transforming.lock().unwrap();
if *transforming {
return Ok(());
}
*transforming = true;
}
crate::tray::set_transforming(&handle, true);

let result = replace_selection(&handle).await;

crate::tray::set_transforming(&handle, false);
*state.is_transforming.lock().unwrap() = false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Robustness: Ensure Transforming State is Always Reset

If replace_selection panics or returns early, the is_transforming state and the tray icon "transforming" indicator () will never be reset, leaving the application permanently locked in a transforming state.

Using an RAII guard (TransformGuard) ensures that the state and tray icon are always cleaned up correctly when the guard goes out of scope, even in the event of a panic or early return.

        // Avoid overlapping runs.
        {
            let mut transforming = state.is_transforming.lock().unwrap();
            if *transforming {
                return Ok(());
            }
            *transforming = true;
        }

        struct TransformGuard(tauri::AppHandle);
        impl Drop for TransformGuard {
            fn drop(&mut self) {
                crate::tray::set_transforming(&self.0, false);
                if let Ok(mut transforming) = self.0.state::<crate::AppState>().is_transforming.lock() {
                    *transforming = false;
                }
            }
        }
        let _guard = TransformGuard(handle.clone());
        crate::tray::set_transforming(&handle, true);

        let result = replace_selection(&handle).await;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant