diff --git a/docs/en_us/2.2-IntegratedInterfaceOverview.md b/docs/en_us/2.2-IntegratedInterfaceOverview.md index d4020f853d..ca691844c4 100644 --- a/docs/en_us/2.2-IntegratedInterfaceOverview.md +++ b/docs/en_us/2.2-IntegratedInterfaceOverview.md @@ -359,6 +359,9 @@ Set controller options. Will be split into specific options in bindings. - ScreenshotResizeMethod Set screenshot resize interpolation method (0=INTER_NEAREST, 1=INTER_LINEAR, 2=INTER_CUBIC, 3=INTER_AREA, 4=INTER_LANCZOS4), default is INTER_AREA(3) +- BackgroundManagedKeys + Declare Win32 background managed key domain. After setting, matching `MaaControllerPostClickKey`, `MaaControllerPostKeyDown`, `MaaControllerPostKeyUp` and pipeline `ClickKey`, `LongPressKey`, `KeyDown`, `KeyUp` operations automatically route through the background guardian path. Only supported by Win32 controllers; other controllers will fail. + ### MaaControllerPostConnection Asynchronously connect device. This is an asynchronous operation that immediately returns an operation id. You can query the status via `MaaControllerStatus` and `MaaControllerWait`. diff --git a/docs/en_us/2.4-ControlMethods.md b/docs/en_us/2.4-ControlMethods.md index 3de9b98df3..cbd87bb306 100644 --- a/docs/en_us/2.4-ControlMethods.md +++ b/docs/en_us/2.4-ControlMethods.md @@ -99,6 +99,7 @@ Different programs on Win32 handle input differently, so there is no universal m > - The `WithCursorPos` methods briefly move the cursor to the target position, then restore it after sending the message, hence "Brief" cursor seizure, but it does not block user operations. > - The `WithWindowPos` methods briefly move the window so the target position aligns with the current cursor position, then restore the window position after sending the message. The cursor is not moved, so there is no mouse seizure, but the window may briefly flicker. > - Win32 also provides a **Mouse Lock Follow** mode: enable with `MaaControllerSetOption(ctrl, MaaCtrlOption_MouseLockFollow, &enabled, sizeof(bool))` (set `enabled` to `true` to enable, `false` to disable). Designed for TPS/FPS games that lock the mouse to the window in the background. When enabled, the window continuously follows the mouse cursor, and RawInput counter-moves prevent the game from sensing hardware mouse movement. Use `MaaControllerPostRelativeMove` to inject intentional camera rotation while this mode is active. **Note:** On Win32, `MaaControllerPostRelativeMove` requires mouse-lock-follow mode to be active; calling it without this mode will fail. Only supported with MessageInput-based input methods (SendMessage / PostMessage variants). +> - Win32 also provides a **Background Managed Keys** guardian: declare the managed key domain through `MaaControllerSetOption(ctrl, MaaCtrlOption_BackgroundManagedKeys, keycodes, sizeof(int32_t) * count)` with an array of virtual key codes to manage. After setting, matching key operations automatically route through the background guardian path and continuously correct key state while the controller is idle. ### Win32 Screencap diff --git "a/docs/zh_cn/2.2-\351\233\206\346\210\220\346\216\245\345\217\243\344\270\200\350\247\210.md" "b/docs/zh_cn/2.2-\351\233\206\346\210\220\346\216\245\345\217\243\344\270\200\350\247\210.md" index 5986a34923..a954fb314a 100644 --- "a/docs/zh_cn/2.2-\351\233\206\346\210\220\346\216\245\345\217\243\344\270\200\350\247\210.md" +++ "b/docs/zh_cn/2.2-\351\233\206\346\210\220\346\216\245\345\217\243\344\270\200\350\247\210.md" @@ -359,6 +359,9 @@ - ScreenshotResizeMethod 设置截图缩放插值方法（0=INTER_NEAREST, 1=INTER_LINEAR, 2=INTER_CUBIC, 3=INTER_AREA, 4=INTER_LANCZOS4），默认为 INTER_AREA(3) +- BackgroundManagedKeys + 声明 Win32 后台受管键域。声明成功后，现有 `MaaControllerPostClickKey`、`MaaControllerPostKeyDown`、`MaaControllerPostKeyUp` 以及 pipeline 中的 `ClickKey`、`LongPressKey`、`KeyDown`、`KeyUp` 会对命中的键自动走后台守护路径。该选项仅支持 Win32 控制器；其他控制器设置会失败。 + ### MaaControllerPostConnection 异步连接设备。这是一个异步操作，会立即返回一个操作 id，可通过 `MaaControllerStatus` 和 `MaaControllerWait` 查询状态。 diff --git "a/docs/zh_cn/2.4-\346\216\247\345\210\266\346\226\271\345\274\217\350\257\264\346\230\216.md" "b/docs/zh_cn/2.4-\346\216\247\345\210\266\346\226\271\345\274\217\350\257\264\346\230\216.md" index 17c429f48b..a303427b16 100644 --- "a/docs/zh_cn/2.4-\346\216\247\345\210\266\346\226\271\345\274\217\350\257\264\346\230\216.md" +++ "b/docs/zh_cn/2.4-\346\216\247\345\210\266\346\226\271\345\274\217\350\257\264\346\230\216.md" @@ -99,6 +99,7 @@ Win32 下不同程序处理输入的方法不同，不存在一个通用方式 > - `WithCursorPos` 系列方式会短暂移动光标到目标位置，发送完消息后会将光标移回原位置，因此会“短暂”抢占鼠标，但不会阻止用户操作。 > - `WithWindowPos` 系列方式会短暂移动窗口，使目标位置与当前光标位置重合，发送完消息后会将窗口移回原位置。不会移动光标，因此不抢占鼠标，但窗口会短暂闪烁。 > - Win32 还提供了**鼠标锁定跟随模式**（Mouse Lock Follow）：通过 `MaaControllerSetOption(ctrl, MaaCtrlOption_MouseLockFollow, &enabled, sizeof(bool))` 开启（`enabled` 为 `true` 开启，`false` 关闭），适用于 TPS/FPS 等在后台将鼠标锁定到窗口内的游戏。开启后窗口会始终跟随鼠标移动，同时通过 RawInput 对冲阻止游戏感知硬件鼠标位移。配合 `MaaControllerPostRelativeMove` 可在此模式下注入视角旋转。**注意：** Win32 平台的 `MaaControllerPostRelativeMove` 需要先开启鼠标锁定跟随模式，否则调用将失败。仅支持 MessageInput 系列输入方式（SendMessage / PostMessage 及其变体）。 +> - Win32 还提供了**后台受管键守护**（Background Managed Keys）：通过 `MaaControllerSetOption(ctrl, MaaCtrlOption_BackgroundManagedKeys, keycodes, sizeof(int32_t) * count)` 声明需要接管的虚拟键码数组。声明成功后，命中的按键操作自动走后台守护路径，并在控制器空闲时持续修正按键状态。 ### Win32 Screencap diff --git a/include/MaaControlUnit/ControlUnitAPI.h b/include/MaaControlUnit/ControlUnitAPI.h index 992d54057c..81959cdf64 100644 --- a/include/MaaControlUnit/ControlUnitAPI.h +++ b/include/MaaControlUnit/ControlUnitAPI.h @@ -101,6 +101,7 @@ class Win32ControlUnitAPI virtual ~Win32ControlUnitAPI() = default; virtual bool set_mouse_lock_follow(bool /*enabled*/) { return false; } + virtual bool set_background_managed_keys_option(const int32_t* /*keycodes*/, size_t /*count*/) { return false; } }; class MacOSControlUnitAPI diff --git a/include/MaaFramework/MaaDef.h b/include/MaaFramework/MaaDef.h index 87ab07e8f3..fbc1628ecc 100644 --- a/include/MaaFramework/MaaDef.h +++ b/include/MaaFramework/MaaDef.h @@ -234,6 +234,14 @@ enum MaaCtrlOptionEnum /// /// value: int, eg: 3; val_size: sizeof(int) MaaCtrlOption_ScreenshotResizeMethod = 6, + + /// Configure background managed key domain for Win32 controllers. + /// Must be set before connection. After setting, matching ClickKey / LongPressKey / KeyDown / KeyUp + /// operations automatically route through the background guardian path. + /// Only supported by Win32 controllers; other controllers will fail. + /// + /// value: int32_t array of virtual key codes; val_size: sizeof(int32_t) * count + MaaCtrlOption_BackgroundManagedKeys = 7, }; typedef MaaOption MaaTaskerOption; diff --git a/source/MaaAgentClient/Client/AgentClient.cpp b/source/MaaAgentClient/Client/AgentClient.cpp index fd968e3c5e..aa3b1df8f0 100644 --- a/source/MaaAgentClient/Client/AgentClient.cpp +++ b/source/MaaAgentClient/Client/AgentClient.cpp @@ -2460,6 +2460,27 @@ bool AgentClient::handle_controller_set_option(const json::value& j) ret = controller->set_option(key, &v, sizeof(v)); break; } + case MaaCtrlOption_BackgroundManagedKeys: { + if (!req.value.is_array()) { + LogError << "BackgroundManagedKeys value must be an array" << VAR(req.value.type_name()); + ret = false; + break; + } + std::vector keys; + for (const auto& item : req.value.as_array()) { + if (!item.is_number()) { + LogError << "BackgroundManagedKeys array element must be a number" << VAR(item.type_name()); + ret = false; + break; + } + keys.push_back(static_cast(item.as_integer())); + } + if (!ret) { + break; + } + ret = controller->set_option(key, keys.data(), sizeof(int32_t) * keys.size()); + break; + } default: LogError << "unknown key" << VAR(req.key); break; diff --git a/source/MaaAgentServer/RemoteInstance/RemoteController.cpp b/source/MaaAgentServer/RemoteInstance/RemoteController.cpp index 8597b0a1e7..cdebe18d77 100644 --- a/source/MaaAgentServer/RemoteInstance/RemoteController.cpp +++ b/source/MaaAgentServer/RemoteInstance/RemoteController.cpp @@ -42,6 +42,21 @@ bool RemoteController::set_option(MaaCtrlOption key, MaaOptionValue value, MaaOp jvalue = *reinterpret_cast(value); break; + case MaaCtrlOption_BackgroundManagedKeys: { + if (val_size != 0 && val_size % sizeof(int32_t) != 0) { + LogError << "invalid val_size for int32_t[] option" << VAR(key) << VAR(val_size); + return false; + } + size_t count = val_size / sizeof(int32_t); + auto keycodes = reinterpret_cast(value); + json::array arr; + for (size_t i = 0; i < count; ++i) { + arr.emplace_back(keycodes[i]); + } + jvalue = std::move(arr); + break; + } + default: LogError << "unknown key" << VAR(key); return false; diff --git a/source/MaaFramework/Controller/ControllerAgent.cpp b/source/MaaFramework/Controller/ControllerAgent.cpp index a926f2a591..7268d858b1 100644 --- a/source/MaaFramework/Controller/ControllerAgent.cpp +++ b/source/MaaFramework/Controller/ControllerAgent.cpp @@ -45,6 +45,8 @@ bool ControllerAgent::set_option(MaaCtrlOption key, MaaOptionValue value, MaaOpt return set_mouse_lock_follow_option(value, val_size); case MaaCtrlOption_ScreenshotResizeMethod: return set_screenshot_resize_method(value, val_size); + case MaaCtrlOption_BackgroundManagedKeys: + return set_background_managed_keys_option(value, val_size); default: LogError << "Unknown key" << VAR(key) << VAR(value); @@ -1250,4 +1252,30 @@ bool ControllerAgent::set_screenshot_resize_method(MaaOptionValue value, MaaOpti return true; } +bool ControllerAgent::set_background_managed_keys_option(MaaOptionValue value, MaaOptionValueSize val_size) +{ + LogDebug; + + if (val_size != 0 && val_size % sizeof(int32_t) != 0) { + LogError << "invalid value size: " << val_size; + return false; + } + + if (!control_unit_) { + LogError << "control_unit_ is nullptr"; + return false; + } + + auto win32_unit = std::dynamic_pointer_cast(control_unit_); + if (!win32_unit) { + LogError << "Background managed keys is only supported for Win32 controllers."; + return false; + } + + size_t count = val_size / sizeof(int32_t); + auto keycodes = reinterpret_cast(value); + + return win32_unit->set_background_managed_keys_option(keycodes, count); +} + MAA_CTRL_NS_END diff --git a/source/MaaFramework/Controller/ControllerAgent.h b/source/MaaFramework/Controller/ControllerAgent.h index 53f0edbde3..b69d68eb8c 100644 --- a/source/MaaFramework/Controller/ControllerAgent.h +++ b/source/MaaFramework/Controller/ControllerAgent.h @@ -290,6 +290,7 @@ class ControllerAgent : public MaaController bool set_image_use_raw_size(MaaOptionValue value, MaaOptionValueSize val_size); bool set_mouse_lock_follow_option(MaaOptionValue value, MaaOptionValueSize val_size); bool set_screenshot_resize_method(MaaOptionValue value, MaaOptionValueSize val_size); + bool set_background_managed_keys_option(MaaOptionValue value, MaaOptionValueSize val_size); private: bool need_to_stop_ = false; diff --git a/source/MaaWin32ControlUnit/Input/BackgroundManagedKeyInput.cpp b/source/MaaWin32ControlUnit/Input/BackgroundManagedKeyInput.cpp new file mode 100644 index 0000000000..0a4c506b0b --- /dev/null +++ b/source/MaaWin32ControlUnit/Input/BackgroundManagedKeyInput.cpp @@ -0,0 +1,409 @@ +#include "BackgroundManagedKeyInput.h" + +#include +#include +#include + +#include "MaaUtils/Logger.h" + +#include "InputUtils.h" + +MAA_CTRL_UNIT_NS_BEGIN + +namespace +{ + +constexpr auto guard_interval = std::chrono::milliseconds(5); +constexpr auto apply_timeout = std::chrono::milliseconds(500); +constexpr auto hotkey_wait_timeout = std::chrono::milliseconds(200); +constexpr int managed_hotkey_base = 2000; +constexpr int stop_max_retries = 10; + +} // namespace + +BackgroundManagedKeyInput::BackgroundManagedKeyInput(HWND hwnd) + : hwnd_(hwnd) +{ +} + +BackgroundManagedKeyInput::~BackgroundManagedKeyInput() +{ + { + std::lock_guard lock(mutex_); + for (const int keycode : managed_keys_) { + release_keys_.emplace(keycode); + } + managed_keys_.clear(); + desired_pressed_keys_.clear(); + ++desired_generation_; + stop_thread_ = true; + } + guard_cv_.notify_all(); + if (guard_thread_.joinable()) { + guard_thread_.join(); + } +} + +bool BackgroundManagedKeyInput::set_managed_keys(const std::vector& keycodes) +{ + auto normalized = normalize_keycodes(keycodes); + + uint64_t generation = 0; + bool clear_all = normalized.empty(); + { + std::lock_guard lock(mutex_); + + if (!thread_started_ && !clear_all) { + guard_thread_ = std::thread(&BackgroundManagedKeyInput::guard_loop, this); + thread_started_ = true; + } + + if (clear_all) { + for (const int keycode : managed_keys_) { + release_keys_.emplace(keycode); + desired_pressed_keys_.erase(keycode); + } + managed_keys_.clear(); + } + else { + for (const int keycode : managed_keys_) { + if (!normalized.contains(keycode)) { + release_keys_.emplace(keycode); + desired_pressed_keys_.erase(keycode); + } + } + managed_keys_ = std::move(normalized); + } + generation = ++desired_generation_; + } + + guard_cv_.notify_all(); + return wait_until_applied(generation); +} + +bool BackgroundManagedKeyInput::is_managed_key(int keycode) const +{ + std::lock_guard lock(mutex_); + return managed_keys_.contains(keycode); +} + +bool BackgroundManagedKeyInput::is_key_pressed(int keycode) const +{ + std::lock_guard lock(mutex_); + return desired_pressed_keys_.contains(keycode); +} + +bool BackgroundManagedKeyInput::key_down(int keycode) +{ + if (!is_valid_keycode(keycode)) { + LogError << "Invalid managed keycode" << VAR(keycode); + return false; + } + + bool inserted = false; + uint64_t generation = 0; + { + std::lock_guard lock(mutex_); + if (!managed_keys_.contains(keycode)) { + LogError << "Key is not in background managed domain" << VAR(keycode); + return false; + } + + auto [_, was_inserted] = desired_pressed_keys_.emplace(keycode); + inserted = was_inserted; + if (!inserted) { + return true; + } + generation = ++desired_generation_; + } + + guard_cv_.notify_all(); + return wait_until_applied(generation); +} + +bool BackgroundManagedKeyInput::key_up(int keycode) +{ + if (!is_valid_keycode(keycode)) { + LogError << "Invalid managed keycode" << VAR(keycode); + return false; + } + + uint64_t generation = 0; + { + std::lock_guard lock(mutex_); + if (!managed_keys_.contains(keycode)) { + LogError << "Key is not in background managed domain" << VAR(keycode); + return false; + } + + if (!desired_pressed_keys_.erase(keycode)) { + return true; + } + generation = ++desired_generation_; + } + + guard_cv_.notify_all(); + return wait_until_applied(generation); +} + +bool BackgroundManagedKeyInput::inactive() +{ + uint64_t generation = 0; + bool has_work = false; + { + std::lock_guard lock(mutex_); + has_work = !managed_keys_.empty() || !desired_pressed_keys_.empty() || !release_keys_.empty(); + if (!has_work) { + return true; + } + + for (const int keycode : managed_keys_) { + release_keys_.emplace(keycode); + } + managed_keys_.clear(); + desired_pressed_keys_.clear(); + generation = ++desired_generation_; + } + + guard_cv_.notify_all(); + return wait_until_applied(generation); +} + +std::unordered_set BackgroundManagedKeyInput::normalize_keycodes(const std::vector& keycodes) +{ + std::unordered_set normalized; + for (const int keycode : keycodes) { + if (!is_valid_keycode(keycode)) { + LogError << "Invalid background managed keycode" << VAR(keycode); + return {}; + } + normalized.emplace(keycode); + } + return normalized; +} + +bool BackgroundManagedKeyInput::is_valid_keycode(int keycode) +{ + return keycode > 0 && keycode <= 0xFF; +} + +int BackgroundManagedKeyInput::hotkey_id(int keycode) +{ + return managed_hotkey_base + keycode; +} + +bool BackgroundManagedKeyInput::is_pressed_now(int keycode) +{ + return (GetAsyncKeyState(keycode) & 0x8000) != 0; +} + +void BackgroundManagedKeyInput::send_key_event(int keycode, bool key_up) +{ + INPUT input {}; + input.type = INPUT_KEYBOARD; + input.ki.wVk = static_cast(keycode); + input.ki.dwFlags = key_up ? KEYEVENTF_KEYUP : 0; + SendInput(1, &input, sizeof(INPUT)); +} + +void BackgroundManagedKeyInput::pump_messages() +{ + MSG msg; + while (PeekMessageW(&msg, nullptr, 0, 0, PM_REMOVE)) { + TranslateMessage(&msg); + DispatchMessageW(&msg); + } +} + +void BackgroundManagedKeyInput::send_activation_hint() const +{ + send_activate_message(hwnd_, true); +} + +BackgroundManagedKeyInput::Snapshot BackgroundManagedKeyInput::snapshot_locked() const +{ + Snapshot snapshot; + snapshot.generation = desired_generation_; + snapshot.desired_pressed_keys = desired_pressed_keys_; + snapshot.keys.reserve(managed_keys_.size() + release_keys_.size()); + + for (const int keycode : managed_keys_) { + snapshot.keys.emplace_back(keycode); + } + for (const int keycode : release_keys_) { + snapshot.keys.emplace_back(keycode); + snapshot.release_keys.emplace_back(keycode); + } + + std::sort(snapshot.keys.begin(), snapshot.keys.end()); + snapshot.keys.erase(std::unique(snapshot.keys.begin(), snapshot.keys.end()), snapshot.keys.end()); + return snapshot; +} + +void BackgroundManagedKeyInput::guard_loop() +{ + MSG msg; + PeekMessageW(&msg, nullptr, 0, 0, PM_NOREMOVE); + + int stop_retries = 0; + + while (true) { + Snapshot snapshot; + { + std::unique_lock lock(mutex_); + if (managed_keys_.empty() && release_keys_.empty() && !stop_thread_) { + guard_cv_.wait(lock, [this] { return stop_thread_ || !managed_keys_.empty() || !release_keys_.empty(); }); + } + else if (!stop_thread_) { + guard_cv_.wait_for(lock, guard_interval); + } + + if (stop_thread_ && release_keys_.empty()) { + break; + } + + if (stop_thread_) { + ++stop_retries; + if (stop_retries > stop_max_retries) { + LogWarn << "Exceeded max retries releasing keys on stop, forcing exit" << VAR(release_keys_.size()); + release_keys_.clear(); + break; + } + } + + snapshot = snapshot_locked(); + } + + bool snapshot_applied = true; + if (!snapshot.keys.empty()) { + snapshot_applied = correct_snapshot(snapshot); + } + else { + pump_messages(); + } + + if (snapshot_applied) { + { + std::lock_guard lock(mutex_); + applied_generation_ = std::max(applied_generation_, snapshot.generation); + for (const int keycode : snapshot.release_keys) { + release_keys_.erase(keycode); + } + } + applied_cv_.notify_all(); + } + } +} + +bool BackgroundManagedKeyInput::correct_snapshot(const Snapshot& snapshot) +{ + bool all_applied = true; + for (const int keycode : snapshot.keys) { + const bool desired_pressed = snapshot.desired_pressed_keys.contains(keycode); + all_applied &= ensure_key_state(keycode, desired_pressed); + } + pump_messages(); + return all_applied; +} + +bool BackgroundManagedKeyInput::ensure_key_state(int keycode, bool desired_pressed) +{ + return desired_pressed ? ensure_key_pressed(keycode) : ensure_key_released(keycode); +} + +bool BackgroundManagedKeyInput::ensure_key_pressed(int keycode) +{ + if (is_pressed_now(keycode)) { + return true; + } + + send_activation_hint(); + + const int id = hotkey_id(keycode); + const bool registered = RegisterHotKey(nullptr, id, 0, static_cast(keycode)) != 0; + if (!registered) { + LogWarn << "RegisterHotKey failed for managed key, using fallback injection" << VAR(keycode) << VAR(GetLastError()); + send_key_event(keycode, false); + pump_messages(); + const bool pressed = is_pressed_now(keycode); + return pressed; + } + + send_key_event(keycode, false); + + MSG msg; + int extra_count = 0; + const auto deadline = std::chrono::steady_clock::now() + hotkey_wait_timeout; + bool got_ours = false; + while (std::chrono::steady_clock::now() < deadline) { + while (PeekMessageW(&msg, nullptr, 0, 0, PM_REMOVE)) { + if (msg.message == WM_HOTKEY && static_cast(msg.wParam) == id) { + if (!got_ours) { + got_ours = true; + } + else { + ++extra_count; + } + } + else { + TranslateMessage(&msg); + DispatchMessageW(&msg); + } + } + if (got_ours) { + break; + } + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + + UnregisterHotKey(nullptr, id); + + while (PeekMessageW(&msg, nullptr, WM_HOTKEY, WM_HOTKEY, PM_REMOVE)) { + if (static_cast(msg.wParam) == id) { + ++extra_count; + } + } + + for (int i = 0; i < extra_count; ++i) { + std::array replay {}; + replay[0].type = INPUT_KEYBOARD; + replay[0].ki.wVk = static_cast(keycode); + replay[1].type = INPUT_KEYBOARD; + replay[1].ki.wVk = static_cast(keycode); + replay[1].ki.dwFlags = KEYEVENTF_KEYUP; + SendInput(static_cast(replay.size()), replay.data(), sizeof(INPUT)); + pump_messages(); + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + + if (extra_count > 0 && !is_pressed_now(keycode)) { + send_key_event(keycode, false); + pump_messages(); + } + + const bool pressed = is_pressed_now(keycode); + return pressed; +} + +bool BackgroundManagedKeyInput::ensure_key_released(int keycode) +{ + if (!is_pressed_now(keycode)) { + return true; + } + + send_key_event(keycode, true); + pump_messages(); + return !is_pressed_now(keycode); +} + +bool BackgroundManagedKeyInput::wait_until_applied(uint64_t generation) +{ + std::unique_lock lock(mutex_); + const bool applied = applied_cv_.wait_for(lock, apply_timeout, [this, generation] { return applied_generation_ >= generation; }); + if (!applied) { + LogWarn << "Timed out waiting for background managed key state to apply" << VAR(generation) << VAR(applied_generation_); + } + return applied; +} + +MAA_CTRL_UNIT_NS_END diff --git a/source/MaaWin32ControlUnit/Input/BackgroundManagedKeyInput.h b/source/MaaWin32ControlUnit/Input/BackgroundManagedKeyInput.h new file mode 100644 index 0000000000..1daf46eead --- /dev/null +++ b/source/MaaWin32ControlUnit/Input/BackgroundManagedKeyInput.h @@ -0,0 +1,75 @@ +#pragma once + +#include +#include +#include +#include +#include + +#include "MaaUtils/SafeWindows.hpp" + +#include "Common/Conf.h" + +MAA_CTRL_UNIT_NS_BEGIN + +// 后台受管键守护。与 LegacyEventInput / MessageInput / SeizeInput 不同， +// 它不是某种输入后端，而是独立于常规 InputBase 链路、在 Win32ControlUnitMgr +// 里按键码路由短路到的守护器，因此刻意不继承 InputBase。 +class BackgroundManagedKeyInput +{ +public: + explicit BackgroundManagedKeyInput(HWND hwnd); + ~BackgroundManagedKeyInput(); + +public: + bool set_managed_keys(const std::vector& keycodes); + bool is_managed_key(int keycode) const; + bool is_key_pressed(int keycode) const; + bool key_down(int keycode); + bool key_up(int keycode); + bool inactive(); + +private: + struct Snapshot + { + std::vector keys; + std::unordered_set desired_pressed_keys; + std::vector release_keys; + uint64_t generation = 0; + }; + +private: + static std::unordered_set normalize_keycodes(const std::vector& keycodes); + static bool is_valid_keycode(int keycode); + static int hotkey_id(int keycode); + static bool is_pressed_now(int keycode); + static void send_key_event(int keycode, bool key_up); + static void pump_messages(); + + void send_activation_hint() const; + // 后缀 _locked 约定：调用方必须已持有 mutex_。 + Snapshot snapshot_locked() const; + void guard_loop(); + bool correct_snapshot(const Snapshot& snapshot); + bool ensure_key_state(int keycode, bool desired_pressed); + bool ensure_key_pressed(int keycode); + bool ensure_key_released(int keycode); + bool wait_until_applied(uint64_t generation); + +private: + HWND hwnd_ = nullptr; + + mutable std::mutex mutex_; + std::condition_variable guard_cv_; + std::condition_variable applied_cv_; + std::unordered_set managed_keys_; + std::unordered_set desired_pressed_keys_; + std::unordered_set release_keys_; + uint64_t desired_generation_ = 0; + uint64_t applied_generation_ = 0; + bool stop_thread_ = false; + bool thread_started_ = false; + std::thread guard_thread_; +}; + +MAA_CTRL_UNIT_NS_END diff --git a/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.cpp b/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.cpp index 87d651b5f2..0ffd53a429 100644 --- a/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.cpp +++ b/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.cpp @@ -6,6 +6,7 @@ #include "MaaUtils/Logger.h" #include "MaaUtils/Time.hpp" +#include "Input/BackgroundManagedKeyInput.h" #include "Input/LegacyEventInput.h" #include "Input/MessageInput.h" #include "Input/SeizeInput.h" @@ -30,6 +31,7 @@ Win32ControlUnitMgr::Win32ControlUnitMgr( , screencap_method_(screencap_method) , mouse_method_(mouse_method) , keyboard_method_(keyboard_method) + , background_keyboard_(std::make_shared(hWnd)) { } @@ -345,6 +347,19 @@ bool Win32ControlUnitMgr::relative_move(int dx, int dy) bool Win32ControlUnitMgr::click_key(int key) { + if (managed_keys_.contains(key)) { + if (!background_keyboard_) { + LogError << "background_keyboard_ is null"; + return false; + } + const bool key_down_ok = background_keyboard_->key_down(key); + const bool key_up_ok = background_keyboard_->key_up(key); + if (key_down_ok && !key_up_ok) { + LogError << "Managed key" << VAR(key) << "key_down succeeded but key_up failed; key may remain logically pressed"; + } + return key_down_ok && key_up_ok; + } + if (!keyboard_) { LogError << "keyboard_ is null"; return false; @@ -365,6 +380,14 @@ bool Win32ControlUnitMgr::input_text(const std::string& text) bool Win32ControlUnitMgr::key_down(int key) { + if (managed_keys_.contains(key)) { + if (!background_keyboard_) { + LogError << "background_keyboard_ is null"; + return false; + } + return background_keyboard_->key_down(key); + } + if (!keyboard_) { LogError << "keyboard_ is null"; return false; @@ -375,6 +398,14 @@ bool Win32ControlUnitMgr::key_down(int key) bool Win32ControlUnitMgr::key_up(int key) { + if (managed_keys_.contains(key)) { + if (!background_keyboard_) { + LogError << "background_keyboard_ is null"; + return false; + } + return background_keyboard_->key_up(key); + } + if (!keyboard_) { LogError << "keyboard_ is null"; return false; @@ -409,10 +440,32 @@ bool Win32ControlUnitMgr::set_mouse_lock_follow(bool enabled) return message_input->set_mouse_lock_follow(enabled); } +bool Win32ControlUnitMgr::set_background_managed_keys_option(const int32_t* keycodes, size_t count) +{ + LogFunc << VAR(count); + + if (!background_keyboard_) { + LogError << "background_keyboard_ is null"; + return false; + } + + std::vector keys(keycodes, keycodes + count); + if (!background_keyboard_->set_managed_keys(keys)) { + LogError << "set_managed_keys failed"; + return false; + } + + managed_keys_ = std::unordered_set(keys.begin(), keys.end()); + return true; +} + bool Win32ControlUnitMgr::inactive() { LogFunc; + if (background_keyboard_) { + background_keyboard_->inactive(); + } if (screencap_) { screencap_->inactive(); } diff --git a/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.h b/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.h index acae16c507..036c365c87 100644 --- a/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.h +++ b/source/MaaWin32ControlUnit/Manager/Win32ControlUnitMgr.h @@ -3,8 +3,10 @@ #include #include #include +#include #include "Base/UnitBase.h" +#include "Input/BackgroundManagedKeyInput.h" #include "MaaControlUnit/ControlUnitAPI.h" #include "MaaFramework/MaaDef.h" #include "MaaUtils/SafeWindows.hpp" @@ -53,6 +55,7 @@ class Win32ControlUnitMgr : public Win32ControlUnitAPI virtual bool scroll(int dx, int dy) override; virtual bool set_mouse_lock_follow(bool enabled) override; + virtual bool set_background_managed_keys_option(const int32_t* keycodes, size_t count) override; virtual bool inactive() override; @@ -86,6 +89,9 @@ class Win32ControlUnitMgr : public Win32ControlUnitAPI std::shared_ptr mouse_ = nullptr; std::shared_ptr keyboard_ = nullptr; std::shared_ptr screencap_ = nullptr; + + std::shared_ptr background_keyboard_ = nullptr; + std::unordered_set managed_keys_; }; MAA_CTRL_UNIT_NS_END diff --git a/source/binding/NodeJS/src/apis/controller.cpp b/source/binding/NodeJS/src/apis/controller.cpp index c4452be962..febd589113 100644 --- a/source/binding/NodeJS/src/apis/controller.cpp +++ b/source/binding/NodeJS/src/apis/controller.cpp @@ -229,6 +229,15 @@ void ControllerImpl::set_mouse_lock_follow(bool enabled) } } +void ControllerImpl::set_background_managed_keys(std::vector keycodes) +{ + if (!MaaControllerSetOption(controller, MaaCtrlOption_BackgroundManagedKeys, + keycodes.empty() ? nullptr : keycodes.data(), + static_cast(sizeof(int32_t) * keycodes.size()))) { + throw maajs::MaaError { "Controller set background_managed_keys failed" }; + } +} + maajs::ValueType ControllerImpl::post_key_down(maajs::ValueType self, maajs::EnvType, int32_t keycode) { auto id = MaaControllerPostKeyDown(controller, keycode); @@ -376,6 +385,7 @@ void ControllerImpl::init_proto(maajs::ObjectType proto, maajs::FunctionType) MAA_BIND_FUNC(proto, "post_touch_up", ControllerImpl::post_touch_up); MAA_BIND_FUNC(proto, "post_relative_move", ControllerImpl::post_relative_move); MAA_BIND_SETTER(proto, "mouse_lock_follow", ControllerImpl::set_mouse_lock_follow); + MAA_BIND_SETTER(proto, "background_managed_keys", ControllerImpl::set_background_managed_keys); MAA_BIND_FUNC(proto, "post_key_down", ControllerImpl::post_key_down); MAA_BIND_FUNC(proto, "post_key_up", ControllerImpl::post_key_up); MAA_BIND_FUNC(proto, "post_scroll", ControllerImpl::post_scroll); diff --git a/source/binding/NodeJS/src/apis/controller.d.ts b/source/binding/NodeJS/src/apis/controller.d.ts index 65623ab60f..b9e302fa84 100644 --- a/source/binding/NodeJS/src/apis/controller.d.ts +++ b/source/binding/NodeJS/src/apis/controller.d.ts @@ -196,6 +196,7 @@ declare global { * @returns true if successful, false otherwise */ set mouse_lock_follow(enabled: boolean) + set background_managed_keys(keycodes: number[]) post_key_down(keycode: number): Job post_key_up(keycode: number): Job /** diff --git a/source/binding/NodeJS/src/apis/controller.h b/source/binding/NodeJS/src/apis/controller.h index 5fa2f0f499..17a9fe04d8 100644 --- a/source/binding/NodeJS/src/apis/controller.h +++ b/source/binding/NodeJS/src/apis/controller.h @@ -64,6 +64,7 @@ struct ControllerImpl : public maajs::NativeClassBase maajs::ValueType post_touch_up(maajs::ValueType self, maajs::EnvType env, int32_t contact); maajs::ValueType post_relative_move(maajs::ValueType self, maajs::EnvType env, int32_t dx, int32_t dy); void set_mouse_lock_follow(bool enabled); + void set_background_managed_keys(std::vector keycodes); maajs::ValueType post_key_down(maajs::ValueType self, maajs::EnvType env, int32_t keycode); maajs::ValueType post_key_up(maajs::ValueType self, maajs::EnvType env, int32_t keycode); maajs::ValueType post_scroll(maajs::ValueType self, maajs::EnvType env, int32_t dx, int32_t dy); diff --git a/source/binding/Python/maa/controller.py b/source/binding/Python/maa/controller.py index f0c681849d..a563984d00 100644 --- a/source/binding/Python/maa/controller.py +++ b/source/binding/Python/maa/controller.py @@ -4,7 +4,7 @@ from abc import abstractmethod from ctypes import c_int32 from pathlib import Path -from typing import Any, Dict, Optional, Tuple, Union +from typing import Any, Dict, Optional, Sequence, Tuple, Union from .buffer import ImageBuffer, StringBuffer from .event_sink import EventSink, NotificationType @@ -274,6 +274,15 @@ def set_mouse_lock_follow(self, enabled: bool) -> bool: self._handle, MaaCtrlOptionEnum.MouseLockFollow, ctypes.byref(c_enabled), ctypes.sizeof(c_enabled) ) + def set_background_managed_keys(self, keys: Sequence[int]) -> bool: + key_array = (ctypes.c_int32 * len(keys))(*keys) + return Library.framework().MaaControllerSetOption( + self._handle, + MaaCtrlOptionEnum.BackgroundManagedKeys, + key_array, + ctypes.sizeof(key_array), + ) + def post_scroll(self, dx: int, dy: int) -> Job: """滚动 / Scroll diff --git a/source/binding/Python/maa/define.py b/source/binding/Python/maa/define.py index 0c156624fb..ae8c540763 100644 --- a/source/binding/Python/maa/define.py +++ b/source/binding/Python/maa/define.py @@ -145,6 +145,13 @@ class MaaCtrlOptionEnum(IntEnum): # default is 3 (INTER_AREA) ScreenshotResizeMethod = 6 + # Configure background managed key domain for Win32 controllers. + # Must be set before connection. After setting, matching ClickKey / LongPressKey / KeyDown / KeyUp + # operations automatically route through the background guardian path. + # Only supported by Win32 controllers; other controllers will fail. + # value: int32_t array of virtual key codes; val_size: sizeof(int32_t) * count + BackgroundManagedKeys = 7 + class MaaInferenceDeviceEnum(IntEnum): CPU = -2 diff --git a/test/agent/agent_child_test.py b/test/agent/agent_child_test.py index 1bed5849ce..c64341988f 100644 --- a/test/agent/agent_child_test.py +++ b/test/agent/agent_child_test.py @@ -330,6 +330,13 @@ def run( # 恢复默认值 controller.set_screenshot_resize_method(3) + # 测试 set_background_managed_keys (non-Win32, should fail) + result = controller.set_background_managed_keys([0x57, 0x41]) + print(f" set_background_managed_keys([0x57, 0x41]): {result}") + assert ( + not result + ), "set_background_managed_keys should fail for non-Win32 controller" + # ============================================================ # Tasker API 补充测试 (详情获取) # ============================================================ diff --git a/test/python/binding_test.py b/test/python/binding_test.py index 532ac5a0c4..2d7e4a67fa 100644 --- a/test/python/binding_test.py +++ b/test/python/binding_test.py @@ -743,6 +743,64 @@ def test_toolkit(): print(" PASS: toolkit") +def test_background_managed_keys_api(): + print("\n=== test_background_managed_keys_api ===") + + # Test with DbgController (non-Win32, should fail) + dbg_controller = DbgController( + install_dir / "test" / "PipelineSmoking" / "Screenshot", + ) + dbg_ret = dbg_controller.set_background_managed_keys([0x57, 0x41]) + print(f" dbg_controller set_background_managed_keys: {dbg_ret}") + assert not dbg_ret, "DbgController should not support BackgroundManagedKeys" + + # Test with Win32 controller if available + desktop_windows = Toolkit.find_desktop_windows() + if desktop_windows: + win32_controller = None + for window in desktop_windows: + try: + win32_controller = Win32Controller(window.hwnd) + break + except RuntimeError: + continue + + if win32_controller is not None: + # Set option before connection + ret = win32_controller.set_background_managed_keys([0x57, 0x41]) + print( + f" win32_controller set_background_managed_keys (before connection): {ret}" + ) + assert ( + ret + ), "Win32Controller should support BackgroundManagedKeys before connection" + + # After connection, setting non-empty array should succeed + win32_controller.post_connection().wait() + ret_post = win32_controller.set_background_managed_keys([0x57, 0x41]) + print( + f" win32_controller set_background_managed_keys (after connection): {ret_post}" + ) + assert ( + ret_post + ), "Win32Controller should support BackgroundManagedKeys after connection" + + # Empty array should clear managed keys + ret_clear = win32_controller.set_background_managed_keys([]) + print( + f" win32_controller set_background_managed_keys (clear with empty): {ret_clear}" + ) + assert ( + ret_clear + ), "Win32Controller should support clearing BackgroundManagedKeys with empty array" + else: + print(" SKIP: failed to create Win32 controller") + else: + print(" SKIP: no desktop windows found for Win32 test") + + print(" PASS: background managed keys API") + + def test_win32_relative_move(): print("\n=== test_win32_relative_move ===") @@ -803,6 +861,9 @@ def test_win32_relative_move(): # 测试 Toolkit test_toolkit() + # 测试 BackgroundManagedKeys 选项 + test_background_managed_keys_api() + # 测试 Win32 relative_move 正路径 test_win32_relative_move()