Skip to content

Commit d4f5b9a

Browse files
oxoxDevclaude
oxoxDev
and
claude
authored
fix(overlay): fullscreen visibility, voice server reliability, and resize (tinyhumansai#528) (tinyhumansai#585)
* fix(overlay): shrink initial overlay window to match idle orb dimensions (tinyhumansai#528) The Tauri overlay window was 248×228 px while the idle orb renders at 50×50. The excess transparent area wasted compositing resources and created an invisible click-absorbing region. Reduce to 60×60 to tightly frame the idle orb with minimal padding. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(overlay): add native drag support with position persistence (tinyhumansai#528) - Mouse-down on the orb initiates Tauri startDragging() for native window drag - Dragged position is saved to localStorage and survives mode changes (idle ↔ active) so the orb stays where the user placed it - Double-click resets to the default bottom-right corner - Cursor changes to grab/grabbing for affordance - Skip default repositioning when a saved position exists Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): reclass NSWindow to NSPanel for fullscreen visibility (tinyhumansai#528) macOS fullscreen apps run in separate Spaces where standard NSWindow cannot follow. Use object_setClass() to reclass the Tauri overlay window from NSWindow to NSPanel at runtime, then configure it with NonactivatingPanel style mask and Transient collection behavior — matching the working Swift accessibility helper pattern. Key configuration that makes this work: - object_setClass(NSWindow → NSPanel) — in-place reclass, no reparenting - NSWindowStyleMask::NonactivatingPanel — critical for panel behavior - NSWindowCollectionBehavior::Transient (not Stationary) — follows Spaces - Window level 25 (NSStatusWindowLevel) — floats above fullscreen apps - setFloatingPanel(true), setHidesOnDeactivate(false) Previous approaches that failed: 1. CGShieldingWindowLevel + CanJoinAllSpaces — hidden (NSWindow limitation) 2. Window level i32::MAX-17 + Stationary — hidden (Space membership issue) 3. CGS private API CGSSetWindowTags sticky bit — blocked on Sonoma 4. object_setClass WITHOUT NonactivatingPanel mask — hidden 5. Create new NSPanel + reparent webview — CRASH (Tao delegate panic) Also removes unused objc2-core-graphics and objc2-foundation deps. Ref: tauri-apps/tauri#11488 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(sidecar): make dev signing non-fatal in stage script codesign failures no longer call process.exit(), preventing yarn tauri dev from hanging when the dev signing identity is missing or the keychain rejects the request. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): prevent race condition and fix restart after stop - Atomically transition Stopped → Idle at start of run() to prevent duplicate run() calls during slow globe listener compilation - Wrap CancellationToken in Mutex so run() creates a fresh token on each start — a cancelled token cannot be reused after stop() - Reset state to Stopped if hotkey listener fails to start Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): capture server errors from spawned run() task Store errors from the background server.run() task via set_last_error() so they surface in voice_server_status RPC responses instead of being silently lost. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): enable programmatic resize and shrink idle dimensions - Change overlay window from 60x60 to 50x50 to match idle orb size - Remove minWidth/minHeight constraints that blocked dynamic resize - Set resizable: true so setSize() calls work for bubble expansion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): simplify window resize and bubble rendering - Clear min/max constraints before resizing to avoid clamping - Replace CSS transition-based bubble visibility with conditional mount for more reliable rendering when mode changes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: fix fmt and remove unused bubbles variable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): enforce lock ordering to prevent race between run() and stop() Acquire cancel lock before state lock in run() — same order as stop() — so stop() cannot cancel a stale token between setting Idle and swapping. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): restore saved position on resize and format with prettier Parse and apply saved drag coordinates instead of just using their presence as a sentinel. Also reformats for prettier compliance. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(sidecar): fail-fast on dev signing failure in CI environments Add CI detection so signing failures abort the build in CI but remain non-fatal for local development. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: fix cargo fmt on Tauri shell import Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent f014058 commit d4f5b9a

9 files changed

Lines changed: 374 additions & 68 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎app/src-tauri/Cargo.lock‎

Lines changed: 2 additions & 16 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎app/src-tauri/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,8 @@ env_logger = "0.11"
4444
nix = { version = "0.29", default-features = false, features = ["signal"] }
4545

4646
[target.'cfg(target_os = "macos")'.dependencies]
47+
objc2 = "0.6"
4748
objc2-app-kit = "0.3.2"
48-
objc2-core-graphics = "0.3.2"
4949

5050
[features]
5151
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!

‎app/src-tauri/src/lib.rs‎

Lines changed: 74 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,11 @@ use tauri_plugin_global_shortcut::{GlobalShortcutExt, ShortcutState};
1717
use tauri_plugin_deep_link::DeepLinkExt;
1818

1919
#[cfg(target_os = "macos")]
20-
use objc2_app_kit::{NSWindow, NSWindowCollectionBehavior};
20+
use objc2::runtime::{AnyClass, AnyObject};
2121
#[cfg(target_os = "macos")]
22-
use objc2_core_graphics::CGShieldingWindowLevel;
22+
use objc2::ClassType;
23+
#[cfg(target_os = "macos")]
24+
use objc2_app_kit::{NSPanel, NSWindowCollectionBehavior, NSWindowStyleMask};
2325

2426
/// Tracks the currently registered dictation hotkey string so we can unregister it later.
2527
struct DictationHotkeyState(Mutex<Vec<String>>);
@@ -91,23 +93,79 @@ fn pin_overlay_bottom_right(window: &WebviewWindow) {
9193

9294
#[cfg(target_os = "macos")]
9395
fn configure_overlay_window_macos(window: &WebviewWindow) {
94-
if let Err(err) = window.set_always_on_top(true) {
95-
log::warn!("[overlay] failed to set always-on-top: {err}");
96-
}
97-
if let Err(err) = window.set_visible_on_all_workspaces(true) {
98-
log::warn!("[overlay] failed to set visible-on-all-workspaces: {err}");
99-
}
96+
// Standard NSWindow cannot float above fullscreen apps on macOS because
97+
// fullscreen apps run in a separate Space. Only NSPanel can do this.
98+
//
99+
// Tauri/tao hardcodes NSWindow as the window class, so we use
100+
// object_setClass() to reclass the existing NSWindow into an NSPanel
101+
// at runtime. This avoids creating a new window (which crashes because
102+
// Tao's window delegate is tightly coupled to the original NSWindow).
103+
//
104+
// After reclassing, we set the NonactivatingPanel style mask and
105+
// Transient collection behavior — matching the working Swift overlay
106+
// helper (accessibility/helper.rs OverlayController) which is confirmed
107+
// to float above fullscreen apps on macOS Sonoma.
108+
//
109+
// Previous attempts that FAILED:
110+
// 1. CGShieldingWindowLevel + CanJoinAllSpaces + FullScreenAuxiliary → hidden
111+
// 2. Window level i32::MAX-17 + Stationary → hidden
112+
// 3. CGS private API CGSSetWindowTags sticky bit → hidden
113+
// 4. object_setClass WITHOUT NonactivatingPanel style mask → hidden
114+
// 5. Create new NSPanel + reparent webview → CRASH (Tao delegate panic)
115+
//
116+
// See: https://github.com/tauri-apps/tauri/issues/11488
100117

101118
match window.ns_window() {
102-
Ok(ns_window) => unsafe {
103-
let window: &NSWindow = &*ns_window.cast();
104-
let mut behavior = window.collectionBehavior();
105-
behavior.insert(NSWindowCollectionBehavior::FullScreenAuxiliary);
106-
behavior.insert(NSWindowCollectionBehavior::CanJoinAllSpaces);
107-
window.setCollectionBehavior(behavior);
108-
window.setLevel((CGShieldingWindowLevel() + 1) as isize);
119+
Ok(ns_window_raw) => unsafe {
120+
let ns_window = ns_window_raw as *mut AnyObject;
121+
122+
// ── Reclass NSWindow → NSPanel ──────────────────────────
123+
let panel_class: *const AnyClass = NSPanel::class();
124+
objc2::ffi::object_setClass(ns_window, panel_class);
125+
log::info!("[overlay] reclassed NSWindow → NSPanel via object_setClass");
126+
127+
// Cast to NSPanel for method calls
128+
let panel: &NSPanel = &*(ns_window as *const NSPanel);
129+
130+
// ── Style mask: add NonactivatingPanel ──────────────────
131+
// This is the KEY piece the Swift helper uses. Without it,
132+
// the panel doesn't behave as a proper non-activating panel
133+
// and won't float above fullscreen Spaces.
134+
let current_style = panel.styleMask();
135+
panel.setStyleMask(current_style | NSWindowStyleMask::NonactivatingPanel);
136+
137+
// ── Collection behavior ─────────────────────────────────
138+
// The Swift helper uses .canJoinAllSpaces + .transient
139+
// (NOT .stationary or .fullScreenAuxiliary alone).
140+
// Transient means the panel follows the active Space and
141+
// appears above fullscreen apps.
142+
panel.setCollectionBehavior(
143+
NSWindowCollectionBehavior::CanJoinAllSpaces
144+
| NSWindowCollectionBehavior::Transient
145+
| NSWindowCollectionBehavior::FullScreenAuxiliary
146+
| NSWindowCollectionBehavior::IgnoresCycle,
147+
);
148+
149+
// ── Window level: status bar tier ───────────────────────
150+
// NSStatusWindowLevel = 25. The Swift helper uses .statusBar
151+
// which is the same value.
152+
panel.setLevel(25);
153+
154+
// ── Panel-specific properties ───────────────────────────
155+
panel.setFloatingPanel(true);
156+
panel.setHidesOnDeactivate(false);
157+
panel.setBecomesKeyOnlyIfNeeded(true);
158+
panel.setWorksWhenModal(true);
159+
160+
// Make sure it's ordered front
161+
panel.orderFrontRegardless();
162+
109163
log::info!(
110-
"[overlay] macOS overlay configured for all spaces/fullscreen auxiliary at shielding+1 level"
164+
"[overlay] NSPanel configured — level=25, \
165+
NonactivatingPanel+canJoinAllSpaces+transient, \
166+
floatingPanel={}, hidesOnDeactivate={}",
167+
panel.isFloatingPanel(),
168+
panel.hidesOnDeactivate(),
111169
);
112170
},
113171
Err(err) => {

‎app/src-tauri/tauri.conf.json‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,15 +24,13 @@
2424
{
2525
"label": "overlay",
2626
"title": "OpenHuman Overlay",
27-
"width": 248,
28-
"height": 228,
29-
"minWidth": 248,
30-
"minHeight": 228,
27+
"width": 50,
28+
"height": 50,
3129
"transparent": true,
3230
"decorations": false,
3331
"alwaysOnTop": true,
3432
"skipTaskbar": true,
35-
"resizable": false,
33+
"resizable": true,
3634
"visible": true,
3735
"center": false
3836
}

‎app/src/overlay/OverlayApp.tsx‎

Lines changed: 86 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@ const OVERLAY_IDLE_MARGIN = 10;
4545
const OVERLAY_ACTIVE_MARGIN = 20;
4646
const OVERLAY_IDLE_OPACITY = 0.6;
4747

48+
const OVERLAY_POSITION_KEY = 'overlay-position';
49+
4850
/** Default auto-dismiss for an attention bubble when no ttl is supplied. */
4951
const DEFAULT_ATTENTION_TTL_MS = 6000;
5052
/** Grace period after STT `released` before returning to idle, giving the
@@ -398,6 +400,43 @@ export default function OverlayApp() {
398400

399401
// ── Window framing: resize / reposition on mode change ────────────────
400402
const status: 'idle' | 'active' = mode === 'idle' ? 'idle' : 'active';
403+
const userDraggedRef = useRef(false);
404+
405+
/** Save the current window position to localStorage after a drag. */
406+
const persistPosition = useCallback(async () => {
407+
try {
408+
const appWindow = getCurrentWindow();
409+
const pos = await appWindow.outerPosition();
410+
localStorage.setItem(OVERLAY_POSITION_KEY, JSON.stringify({ x: pos.x, y: pos.y }));
411+
userDraggedRef.current = true;
412+
} catch {
413+
// position read failed — ignore
414+
}
415+
}, []);
416+
417+
/** Reset saved position so the overlay snaps back to the default corner. */
418+
const resetPosition = useCallback(() => {
419+
localStorage.removeItem(OVERLAY_POSITION_KEY);
420+
userDraggedRef.current = false;
421+
}, []);
422+
423+
/** Initiate native window drag on mouse-down. */
424+
const handleDragStart = useCallback(
425+
async (e: React.MouseEvent) => {
426+
// Only drag on primary button; ignore if a click handler should fire
427+
if (e.button !== 0) return;
428+
e.preventDefault();
429+
try {
430+
const appWindow = getCurrentWindow();
431+
await appWindow.startDragging();
432+
// After the drag completes, persist the new position
433+
void persistPosition();
434+
} catch {
435+
// startDragging can fail if not supported — fall through silently
436+
}
437+
},
438+
[persistPosition]
439+
);
401440

402441
useEffect(() => {
403442
const appWindow = getCurrentWindow();
@@ -408,24 +447,55 @@ export default function OverlayApp() {
408447
const size = new LogicalSize(width, height);
409448

410449
const updateWindowFrame = async () => {
450+
// Remove all size constraints first, then set the new size, then
451+
// re-apply constraints. This avoids the ordering problem where the
452+
// old min/max clamps the new size.
453+
try {
454+
await appWindow.setMinSize(null);
455+
} catch {
456+
/* ignore */
457+
}
458+
try {
459+
await appWindow.setMaxSize(null);
460+
} catch {
461+
/* ignore */
462+
}
411463
try {
412464
await appWindow.setSize(size);
413465
} catch (error) {
414466
console.warn('[overlay] failed to resize overlay window', error);
415467
}
416-
468+
console.debug(`[overlay] resized to ${width}x${height} (active=${isActive})`);
469+
// Lock to exact size so the user can't accidentally resize
417470
try {
418471
await appWindow.setMinSize(size);
419-
} catch (error) {
420-
console.warn('[overlay] failed to set overlay min size', error);
472+
} catch {
473+
/* ignore */
421474
}
422-
423475
try {
424476
await appWindow.setMaxSize(size);
425-
} catch (error) {
426-
console.warn('[overlay] failed to set overlay max size', error);
477+
} catch {
478+
/* ignore */
427479
}
428480

481+
// Restore saved position from a previous drag
482+
const saved = localStorage.getItem(OVERLAY_POSITION_KEY);
483+
if (saved) {
484+
try {
485+
const { x, y } = JSON.parse(saved) as { x: number; y: number };
486+
await appWindow.setPosition(new LogicalPosition(x, y));
487+
userDraggedRef.current = true;
488+
return;
489+
} catch {
490+
localStorage.removeItem(OVERLAY_POSITION_KEY);
491+
}
492+
}
493+
494+
if (userDraggedRef.current) {
495+
return;
496+
}
497+
498+
// Default: pin to bottom-right corner
429499
try {
430500
const monitor = await currentMonitor();
431501
if (!monitor) {
@@ -445,8 +515,6 @@ export default function OverlayApp() {
445515
}, [status]);
446516

447517
// ── Render ────────────────────────────────────────────────────────────
448-
const bubbles = useMemo<OverlayBubble[]>(() => (bubble ? [bubble] : []), [bubble]);
449-
450518
const orbClassName = useMemo(() => {
451519
if (status === 'active') {
452520
return 'border-blue-950 bg-blue-700';
@@ -463,15 +531,13 @@ export default function OverlayApp() {
463531
<div className="flex h-screen w-screen items-end justify-end bg-transparent px-0 py-0">
464532
<div
465533
className={`relative flex select-none flex-col items-end ${status === 'active' ? 'gap-3' : 'gap-0'}`}>
466-
<div
467-
className={`flex flex-col items-end gap-2 transition-all duration-200 ${status === 'active' ? 'max-w-[184px] opacity-100' : 'max-w-0 opacity-0'}`}>
468-
{bubbles.map(b => (
469-
<div key={b.id} className="animate-[overlay-bubble-in_220ms_ease-out]">
470-
{/* key on the chip itself remounts the typewriter for each new bubble */}
471-
<OverlayBubbleChip key={b.id} bubble={b} />
534+
{status === 'active' && bubble && (
535+
<div className="max-w-[184px]">
536+
<div className="animate-[overlay-bubble-in_220ms_ease-out]">
537+
<OverlayBubbleChip key={bubble.id} bubble={bubble} />
472538
</div>
473-
))}
474-
</div>
539+
</div>
540+
)}
475541

476542
<div className="relative">
477543
<button
@@ -484,15 +550,17 @@ export default function OverlayApp() {
484550
: 'OpenHuman overlay'
485551
}
486552
onClick={goIdle}
553+
onMouseDown={handleDragStart}
554+
onDoubleClick={resetPosition}
487555
onMouseEnter={() => {
488556
setIsHovered(true);
489557
}}
490558
onMouseLeave={() => {
491559
setIsHovered(false);
492560
}}
493-
className={`group relative flex cursor-pointer items-center justify-center overflow-hidden rounded-full border transition-all duration-200 ${orbClassName} ${orbSizeClassName}`}
561+
className={`group relative flex cursor-grab items-center justify-center overflow-hidden rounded-full border transition-all duration-200 active:cursor-grabbing ${orbClassName} ${orbSizeClassName}`}
494562
style={orbStyle}
495-
title="Click to dismiss">
563+
title="Drag to move · Double-click to reset position">
496564
<div
497565
className={`pointer-events-none opacity-95 transition-transform duration-300 group-hover:scale-105 ${orbCanvasClassName}`}>
498566
<RotatingTetrahedronCanvas inverted={tetrahedronInverted} />

‎scripts/stage-core-sidecar.mjs‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,8 +98,16 @@ if (process.platform === "darwin") {
9898
{ cwd: root, encoding: "utf8" },
9999
);
100100
if (check.stdout && check.stdout.includes(DEV_IDENTITY)) {
101-
run("codesign", ["--force", "--sign", DEV_IDENTITY, "--timestamp=none", dest]);
102-
console.log(`[core:stage] Signed sidecar with "${DEV_IDENTITY}"`);
101+
const signResult = spawnSync("codesign", ["--force", "--sign", DEV_IDENTITY, "--timestamp=none", dest], { cwd: root, stdio: "inherit", shell: false });
102+
const isCI = process.env.CI === "true" || process.env.CI === "1";
103+
if (signResult.status === 0) {
104+
console.log(`[core:stage] Signed sidecar with "${DEV_IDENTITY}"`);
105+
} else if (isCI) {
106+
console.error(`[core:stage] Dev signing failed (status ${signResult.status}) in CI — aborting.`);
107+
process.exit(signResult.status ?? 1);
108+
} else {
109+
console.warn(`[core:stage] Dev signing failed (status ${signResult.status}), continuing without stable signing.`);
110+
}
103111
} else {
104112
console.warn(
105113
`[core:stage] Dev signing identity "${DEV_IDENTITY}" not found.\n` +

‎src/openhuman/voice/schemas.rs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -357,10 +357,12 @@ fn handle_voice_server_start(params: Map<String, Value>) -> ControllerFuture {
357357

358358
let server = global_server(server_config);
359359
let config_clone = config.clone();
360+
let server_for_err = server.clone();
360361

361362
tokio::spawn(async move {
362363
if let Err(e) = server.run(&config_clone).await {
363364
log::error!("[voice_server] server exited with error: {e}");
365+
server_for_err.set_last_error(&e).await;
364366
}
365367
});
366368

0 commit comments

Comments
 (0)