feat: Computer Use — Windows 跨平台支持 + GUI 无障碍增强 + Python Bridge

三平台 Computer Use (macOS + Windows + Linux)，Windows 专项增强。

- MCP server: toolCalls/tools/executor/mcpServer 等 12 文件完整实现
- 平台抽象层: platforms/{win32,darwin,linux}.ts
- 跨平台 executor: executorCrossPlatform.ts
- CHICAGO_MCP + VOICE_MODE feature flags 启用

- windowMessage.ts: SendMessageW (WM_CHAR Unicode + 剪贴板粘贴)
- windowBorder.ts: 4 叠加窗口边框 (30fps 跟踪)
- uiAutomation.ts: UI Automation 元素树/点击/写值
- accessibilitySnapshot.ts: 无障碍快照 → 模型感知 GUI
- bridge.py + bridgeClient.ts: Python 长驻进程 (替代 per-call PS)

- window_management: min/max/restore/close/focus (Win32 API)
- click_element / type_into_element: 按名称操作 (无需坐标)
- 截图自动附带 Accessibility Snapshot

- 17 种方法, stdin/stdout JSON 通信
- 窗口枚举 1.5ms vs PS 500ms, 截图 360ms vs PS 800ms
- 依赖: mss + Pillow + pywinauto

Author: unraid

.gitignore

@@ -13,4 +13,16 @@ src/utils/vendor/
 # AI tool runtime directories
 .agents/
 .codex/
-.omx/
+.omx/
+
+# Binary / screenshot files (root only)
+/*.png
+*.bmp
+
+# Agent / tool state dirs
+.swarm/
+.agents/__pycache__/
+
+# Python bytecode
+__pycache__/
+*.pyc

DEV-LOG.md

@@ -39,6 +39,7 @@
 
 ## Computer Use Windows 增强：窗口绑定截图 + UI Automation + OCR (2026-04-03)
 
+
 在三平台基础实现之上，利用 Windows 原生 API 增强 Computer Use 的 Windows 专属能力。
 
 **新增文件：**
@@ -118,23 +119,6 @@ packages/@ant/computer-use-{input,swift}/src/
 | `vendor/audio-capture/{platform}/audio-capture.node` | 6 个平台的原生音频二进制（cpal，来自参考项目） |
 | `vendor/audio-capture-src/index.ts` | 原生模块加载器（按 `${arch}-${platform}` 动态 require `.node`） |
 
-**修改文件：**
-
-| 文件 | 变更 |
-|------|------|
-| `packages/audio-capture-napi/src/index.ts` | SoX 子进程 stub → 原生 `.node` 加载器（含 `process.cwd()` workspace 路径 fallback） |
-| `scripts/dev.ts` | `DEFAULT_FEATURES` 加 `"VOICE_MODE"` |
-| `build.ts` | `DEFAULT_BUILD_FEATURES` 加 `"VOICE_MODE"` |
-| `docs/features/voice-mode.md` | 追加恢复计划章节（第八节） |
-
-**验证结果：**
-
-- `isNativeAudioAvailable()` → `true`（Windows x64 原生 `.node` 加载成功）
-- `feature('VOICE_MODE')` → `ENABLED`
-- `bun run build` → voice 代码编入产物
-
-**运行时前置条件：** claude.ai OAuth 登录 + 麦克风权限
-
 ---
 
 ## Enable Claude in Chrome MCP (2026-04-03)

build.ts

@@ -2,62 +2,62 @@ import { readdir, readFile, writeFile, cp } from "fs/promises";
 import { join } from "path";
 import { getMacroDefines } from "./scripts/defines.ts";
 
-const outdir = "dist";
+const outdir = 'dist'
 
 // Step 1: Clean output directory
-const { rmSync } = await import("fs");
-rmSync(outdir, { recursive: true, force: true });
+const { rmSync } = await import('fs')
+rmSync(outdir, { recursive: true, force: true })
 
 // Default features that match the official CLI build.
 // Additional features can be enabled via FEATURE_<NAME>=1 env vars.
 const DEFAULT_BUILD_FEATURES = ["AGENT_TRIGGERS_REMOTE", "CHICAGO_MCP", "VOICE_MODE"];
 
 // Collect FEATURE_* env vars → Bun.build features
 const envFeatures = Object.keys(process.env)
-    .filter(k => k.startsWith("FEATURE_"))
-    .map(k => k.replace("FEATURE_", ""));
-const features = [...new Set([...DEFAULT_BUILD_FEATURES, ...envFeatures])];
+  .filter(k => k.startsWith('FEATURE_'))
+  .map(k => k.replace('FEATURE_', ''))
+const features = [...new Set([...DEFAULT_BUILD_FEATURES, ...envFeatures])]
 
 // Step 2: Bundle with splitting
 const result = await Bun.build({
-    entrypoints: ["src/entrypoints/cli.tsx"],
-    outdir,
-    target: "bun",
-    splitting: true,
-    define: getMacroDefines(),
-    features,
-});
+  entrypoints: ['src/entrypoints/cli.tsx'],
+  outdir,
+  target: 'bun',
+  splitting: true,
+  define: getMacroDefines(),
+  features,
+})
 
 if (!result.success) {
-    console.error("Build failed:");
-    for (const log of result.logs) {
-        console.error(log);
-    }
-    process.exit(1);
+  console.error('Build failed:')
+  for (const log of result.logs) {
+    console.error(log)
+  }
+  process.exit(1)
 }
 
 // Step 3: Post-process — replace Bun-only `import.meta.require` with Node.js compatible version
-const files = await readdir(outdir);
-const IMPORT_META_REQUIRE = "var __require = import.meta.require;";
-const COMPAT_REQUIRE = `var __require = typeof import.meta.require === "function" ? import.meta.require : (await import("module")).createRequire(import.meta.url);`;
+const files = await readdir(outdir)
+const IMPORT_META_REQUIRE = 'var __require = import.meta.require;'
+const COMPAT_REQUIRE = `var __require = typeof import.meta.require === "function" ? import.meta.require : (await import("module")).createRequire(import.meta.url);`
 
-let patched = 0;
+let patched = 0
 for (const file of files) {
-    if (!file.endsWith(".js")) continue;
-    const filePath = join(outdir, file);
-    const content = await readFile(filePath, "utf-8");
-    if (content.includes(IMPORT_META_REQUIRE)) {
-        await writeFile(
-            filePath,
-            content.replace(IMPORT_META_REQUIRE, COMPAT_REQUIRE),
-        );
-        patched++;
-    }
+  if (!file.endsWith('.js')) continue
+  const filePath = join(outdir, file)
+  const content = await readFile(filePath, 'utf-8')
+  if (content.includes(IMPORT_META_REQUIRE)) {
+    await writeFile(
+      filePath,
+      content.replace(IMPORT_META_REQUIRE, COMPAT_REQUIRE),
+    )
+    patched++
+  }
 }
 
 console.log(
-    `Bundled ${result.outputs.length} files to ${outdir}/ (patched ${patched} for Node.js compat)`,
-);
+  `Bundled ${result.outputs.length} files to ${outdir}/ (patched ${patched} for Node.js compat)`,
+)
 
 // Step 4: Copy native .node addon files (audio-capture)
 const vendorDir = join(outdir, "vendor", "audio-capture");
@@ -66,16 +66,16 @@ console.log(`Copied vendor/audio-capture/ → ${vendorDir}/`);
 
 // Step 5: Bundle download-ripgrep script as standalone JS for postinstall
 const rgScript = await Bun.build({
-    entrypoints: ["scripts/download-ripgrep.ts"],
-    outdir,
-    target: "node",
-});
+  entrypoints: ['scripts/download-ripgrep.ts'],
+  outdir,
+  target: 'node',
+})
 if (!rgScript.success) {
-    console.error("Failed to bundle download-ripgrep script:");
-    for (const log of rgScript.logs) {
-        console.error(log);
-    }
-    // Non-fatal — postinstall fallback to bun run scripts/download-ripgrep.ts
+  console.error('Failed to bundle download-ripgrep script:')
+  for (const log of rgScript.logs) {
+    console.error(log)
+  }
+  // Non-fatal — postinstall fallback to bun run scripts/download-ripgrep.ts
 } else {
-    console.log(`Bundled download-ripgrep script to ${outdir}/`);
+  console.log(`Bundled download-ripgrep script to ${outdir}/`)
 }