Skip to content

Commit a714789

Browse files
committed
Wire USB passthrough over WebRTC channel, add resume tokens and hotplug refresh
Make USB passthrough work cross-machine and survive reconnects. - WebRTC usb DataChannel: UsbChannelHost / UsbChannelClient adapters bridge the protocol over an aiortc RTCDataChannel (mirroring the file channel). webrtc_host creates the "usb" channel gated on viewer auth + the feature flag; webrtc_viewer exposes viewer.usb_client(). Adapters are transport-decoupled and unit-tested with a fake channel. - Claim resume: OPENED carries a resume_token; a reconnecting viewer sends RESUME to re-bind a claim the host session still holds, keeping device state and claim_id intact (new 0x0A RESUME opcode). - USB Sharing panel auto-refreshes its local device table from the hotplug watcher when enabled. Docs updated; i18n across four languages; import je_auto_control stays Qt-free and aiortc-free.
1 parent 06c5e0b commit a714789

20 files changed

Lines changed: 666 additions & 16 deletions

docs/source/Eng/doc/operations_layer/usb_passthrough_design.rst

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,17 @@ Channel
7777

7878
A dedicated WebRTC ``DataChannel`` named ``usb`` per session, with
7979
``ordered=True`` and ``maxRetransmits=None`` (full reliability).
80+
81+
**Wired up:** the host (``webrtc_host``) creates the ``usb`` channel and
82+
feeds it to a session through ``UsbChannelHost`` (gated on viewer
83+
authentication + the feature flag); the viewer (``webrtc_viewer``)
84+
wraps the incoming channel in ``UsbChannelClient``, exposed via
85+
``viewer.usb_client()`` for ``list_devices`` / ``open`` / ``resume``.
86+
The adapters are transport-decoupled and unit-tested with a fake
87+
channel (see ``test_usb_webrtc_channel``). **Reconnect:** OPENED carries
88+
a ``resume_token``; if the host session outlives the viewer's transport
89+
drop, the viewer sends ``RESUME{resume_token}`` to re-bind the existing
90+
claim — keeping device state and ``claim_id`` — instead of re-OPENing.
8091
Bulk and interrupt USB transfers tolerate the latency far better
8192
than they tolerate loss; the existing video/audio channels already
8293
demonstrate that the underlying SCTP transport handles ordered
@@ -130,6 +141,7 @@ Op (hex) Direction Purpose
130141
``0x07 CREDIT`` viewer ↔ host Backpressure window update
131142
``0x08 CLOSE`` viewer → host Release the claim
132143
``0x09 CLOSED`` host → viewer Acknowledgement (or unsolicited on host-side disconnect)
144+
``0x0A RESUME`` viewer → host Re-bind an existing claim after reconnect via resume_token
133145
``0xFF ERROR`` either Protocol error / unsupported op
134146
================ ========================================= ==============
135147

docs/source/Eng/doc/operations_layer/usb_passthrough_operator_guide.rst

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -279,11 +279,13 @@ What is *not* shipped yet
279279
one (a descriptor read proves the full stack). The *USB Browser* tab's
280280
*Open* button now also works against a **localhost** target via the
281281
same loopback path.
282-
- Cross-machine open still needs the WebRTC ``usb`` DataChannel, which
283-
the viewer GUI does not yet auto-wire — against a remote host the
284-
*Open* button shows a "not yet wired" message. You can drive the
285-
protocol from Python today (including
286-
``UsbPassthroughClient.list_devices()`` over the channel).
282+
- Cross-machine transport is now wired: the WebRTC host creates a
283+
``usb`` DataChannel and the viewer exposes ``viewer.usb_client()``
284+
(a ``UsbChannelClient`` with ``list_devices`` / ``open`` / ``resume``).
285+
Drive it from Python today over a live WebRTC session. The simple
286+
*USB Browser* / *USB Sharing* panels still use the local loopback +
287+
REST paths; auto-wiring those panels to a live WebRTC viewer session
288+
is the remaining GUI integration step.
287289
- Windows WinUSB and macOS IOKit transfer paths are written but not yet
288290
validated against real hardware. Do not use in production until the
289291
Phase 2e hardware test matrix passes.

docs/source/Zh/doc/operations_layer/usb_passthrough_design.rst

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,16 @@ Channel
7070

7171
每個 session 一條專用的 WebRTC ``DataChannel``\ ,名稱 ``usb``\
7272
``ordered=True`` 且 ``maxRetransmits=None``\ (完全可靠傳輸)。
73+
74+
**已串接:** host(``webrtc_host``)會建立 ``usb`` channel 並以
75+
``UsbChannelHost`` 餵給 session(綁定 viewer 認證 + feature flag);
76+
viewer(``webrtc_viewer``)收到該 channel 後以 ``UsbChannelClient``
77+
包裝,透過 ``viewer.usb_client()`` 供上層呼叫 ``list_devices`` /
78+
``open`` / ``resume``。adapter 與 transport 解耦,可用 fake channel
79+
單測(見 ``test_usb_webrtc_channel``)。**斷線續租:** OPENED 會帶
80+
``resume_token``;若 host session 撐過 viewer 的 transport 中斷,
81+
viewer 重連後送 ``RESUME{resume_token}`` 即可重綁原 claim、保留裝置
82+
狀態,不需重新 OPEN。
7383
USB 的 bulk 與 interrupt 傳輸對延遲的容忍度遠高於對遺失的容忍度;
7484
既有的 video/audio channel 也已示範底層 SCTP 傳輸足以承擔有序可靠
7585
串流。
@@ -118,6 +128,7 @@ Op (hex) 方向 用途
118128
``0x07 CREDIT`` viewer ↔ host Backpressure 視窗更新
119129
``0x08 CLOSE`` viewer → host 釋放 claim
120130
``0x09 CLOSED`` host → viewer 確認(host 端斷線時也可主動發出)
131+
``0x0A RESUME`` viewer → host 重連後以 resume_token 重新綁定既有 claim
121132
``0xFF ERROR`` 雙向 協定錯誤/不支援 op
122133
================ ===================================== ======================
123134

docs/source/Zh/doc/operations_layer/usb_passthrough_operator_guide.rst

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -258,9 +258,12 @@ OPEN 後 host 鍵盤停止運作 Linux:HID 裝置被 claim
258258
做 ACL 允許/封鎖;右側經 in-process channel 列出分享裝置並 *開啟*
259259
其中一個(讀描述元即證明整條堆疊運作)。*USB Browser* 分頁的 *Open*
260260
按鈕現在對 **localhost** 目標也會走同一條 loopback 路徑。
261-
- 跨機器開啟仍需 WebRTC ``usb`` DataChannel,viewer GUI 尚未自動串接 —
262-
對遠端主機按 *Open* 會顯示「尚未串接」訊息。今天可以從 Python 驅動
263-
協定(含經 channel 的 ``UsbPassthroughClient.list_devices()``\ )。
261+
- 跨機器 transport 已串接:WebRTC host 會建立 ``usb`` DataChannel,
262+
viewer 以 ``viewer.usb_client()`` 暴露 ``UsbChannelClient``\ (含
263+
``list_devices`` / ``open`` / ``resume``\ )。今天即可在 WebRTC session
264+
上從 Python 驅動。簡易的 *USB Browser* / *USB 分享* 面板目前仍走本機
265+
loopback + REST;把面板自動接到 live WebRTC viewer session 是剩餘的
266+
GUI 整合步驟。
264267
- Windows WinUSB 與 macOS IOKit 的 transfer 路徑已寫但尚未對實體硬體
265268
驗證。在 Phase 2e 硬體測試矩陣通過前請勿用於 production。
266269
- Phase 2e 外部安全審查尚未簽核;feature flag 必須維持顯式 opt-in。

je_auto_control/gui/language_wrapper/english.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,7 @@
155155
"usb_share_acl_exported": "ACL exported.",
156156
"usb_share_acl_imported": "Imported {count} rule(s).",
157157
"usb_share_acl_import_failed": "ACL I/O failed: {error}",
158+
"usb_share_auto_refresh": "Auto-refresh (hotplug)",
158159

159160
# Inspector tab
160161
"inspector_metrics_group": "Rolling metrics",

je_auto_control/gui/language_wrapper/japanese.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,7 @@
153153
"usb_share_acl_exported": "ACL をエクスポートしました。",
154154
"usb_share_acl_imported": "{count} 件のルールをインポートしました。",
155155
"usb_share_acl_import_failed": "ACL 入出力に失敗:{error}",
156+
"usb_share_auto_refresh": "自動更新(ホットプラグ)",
156157

157158
# 監視タブ
158159
"inspector_metrics_group": "集約メトリクス",

je_auto_control/gui/language_wrapper/simplified_chinese.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,7 @@
144144
"usb_share_acl_exported": "ACL 已导出。",
145145
"usb_share_acl_imported": "已导入 {count} 条规则。",
146146
"usb_share_acl_import_failed": "ACL 读写失败:{error}",
147+
"usb_share_auto_refresh": "自动刷新(热插拔)",
147148

148149
# 包监测分页
149150
"inspector_metrics_group": "汇总指标",

je_auto_control/gui/language_wrapper/traditional_chinese.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,7 @@
145145
"usb_share_acl_exported": "ACL 已匯出。",
146146
"usb_share_acl_imported": "已匯入 {count} 條規則。",
147147
"usb_share_acl_import_failed": "ACL 讀寫失敗:{error}",
148+
"usb_share_auto_refresh": "自動刷新(熱插拔)",
148149

149150
# 封包監測分頁
150151
"inspector_metrics_group": "彙整指標",

je_auto_control/gui/usb_passthrough_panel.py

Lines changed: 35 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,11 @@
1919

2020
from typing import Any, Callable, List, Optional
2121

22-
from PySide6.QtCore import QObject, QThread, Signal
22+
from PySide6.QtCore import QObject, QThread, QTimer, Signal
2323
from PySide6.QtWidgets import (
24-
QFileDialog, QGroupBox, QHBoxLayout, QHeaderView, QLabel, QLineEdit,
25-
QMessageBox, QPushButton, QTableWidget, QTableWidgetItem, QVBoxLayout,
26-
QWidget,
24+
QCheckBox, QFileDialog, QGroupBox, QHBoxLayout, QHeaderView, QLabel,
25+
QLineEdit, QMessageBox, QPushButton, QTableWidget, QTableWidgetItem,
26+
QVBoxLayout, QWidget,
2727
)
2828

2929
from je_auto_control.gui._i18n_helpers import TranslatableMixin
@@ -37,6 +37,7 @@
3737
export_acl_to_file, import_acl_from_file,
3838
)
3939
from je_auto_control.utils.usb.usb_devices import list_usb_devices
40+
from je_auto_control.utils.usb.usb_watcher import default_usb_watcher
4041

4142

4243
def _t(key: str) -> str:
@@ -84,6 +85,12 @@ def __init__(self, parent: Optional[QWidget] = None, *,
8485
self._thread: Optional[QThread] = None
8586
self._host_badge = _StatusBadge()
8687
self._viewer_status = QLabel("")
88+
self._auto_check = QCheckBox()
89+
self._auto_check.toggled.connect(self._on_auto_toggled)
90+
self._hotplug_timer = QTimer(self)
91+
self._hotplug_timer.setInterval(2000)
92+
self._hotplug_timer.timeout.connect(self._poll_hotplug)
93+
self._last_seen_seq = 0
8794
self._local_table = _make_table(5)
8895
self._shared_table = _make_table(4)
8996
self._remote_url = QLineEdit("http://127.0.0.1:9939")
@@ -133,6 +140,8 @@ def _build_host_section(self) -> QWidget:
133140
acl_row.addWidget(refresh_btn)
134141
acl_row.addWidget(allow_btn)
135142
acl_row.addWidget(block_btn)
143+
self._tr(self._auto_check, "usb_share_auto_refresh")
144+
acl_row.addWidget(self._auto_check)
136145
layout.addLayout(acl_row)
137146
io_row = QHBoxLayout()
138147
export_btn = self._tr(QPushButton(), "usb_share_export_acl")
@@ -251,6 +260,25 @@ def _refresh_local_devices(self) -> None:
251260
for col, text in enumerate(cells):
252261
self._local_table.setItem(row, col, QTableWidgetItem(text))
253262

263+
def _on_auto_toggled(self, on: bool) -> None:
264+
watcher = default_usb_watcher()
265+
if on:
266+
watcher.start()
267+
self._hotplug_timer.start()
268+
else:
269+
self._hotplug_timer.stop()
270+
watcher.stop()
271+
272+
def _poll_hotplug(self) -> None:
273+
watcher = default_usb_watcher()
274+
if not watcher.is_running:
275+
return
276+
events = watcher.recent_events(since=self._last_seen_seq, limit=20)
277+
if not events:
278+
return
279+
self._last_seen_seq = events[-1]["seq"]
280+
self._refresh_local_devices()
281+
254282
def _set_policy(self, allow: bool) -> None:
255283
row = _selected_row(self._local_table)
256284
if row is None:
@@ -408,6 +436,9 @@ def _cell(table: QTableWidget, row: int, col: int) -> str:
408436
return "" if text == "-" else text
409437

410438
def closeEvent(self, event) -> None: # noqa: N802 # Qt override name
439+
self._hotplug_timer.stop()
440+
if self._auto_check.isChecked():
441+
default_usb_watcher().stop()
411442
self._disable_sharing()
412443
super().closeEvent(event)
413444

je_auto_control/utils/remote_desktop/webrtc_host.py

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,8 @@ def __init__(self, *, token: str, # NOSONAR python:S107 # public constructor;
9999
self._mic_receiver = None # Optional[MicUplinkReceiver]
100100
self._files_channel = None
101101
self._files_receiver = None # Optional[FileTransferReceiver]
102+
self._usb_channel = None
103+
self._usb_host = None # Optional[UsbChannelHost]
102104
self._on_file_received: Optional[Callable] = None
103105
self._on_viewer_video_frame: Optional[Callable] = None
104106
self._viewer_video_task = None
@@ -187,6 +189,8 @@ async def _async_create_offer(self) -> str:
187189
self._wire_mic_channel(self._mic_channel)
188190
self._files_channel = self._pc.createDataChannel("files")
189191
self._wire_files_channel(self._files_channel)
192+
self._usb_channel = self._pc.createDataChannel("usb")
193+
self._wire_usb_channel(self._usb_channel)
190194
self._wire_state_handlers(self._pc)
191195
self._wire_viewer_video_handler(self._pc)
192196
offer = await self._pc.createOffer()
@@ -378,6 +382,36 @@ def _on_message(message) -> None:
378382
on_done=self._on_file_done,
379383
)
380384

385+
def _wire_usb_channel(self, channel) -> None:
386+
"""Carry USB passthrough over the ``usb`` DataChannel.
387+
388+
Gated on viewer authentication *and* the global passthrough flag
389+
(default off); per-device access is governed by the host's
390+
``UsbAcl`` (default deny). The session is built lazily on the
391+
first frame so a host without the USB backend installed pays
392+
nothing until a viewer actually uses the channel.
393+
"""
394+
from je_auto_control.utils.usb.passthrough import UsbChannelHost
395+
396+
def _factory():
397+
from je_auto_control.utils.usb.passthrough import (
398+
UsbAcl, UsbPassthroughSession, default_passthrough_backend,
399+
)
400+
return UsbPassthroughSession(
401+
default_passthrough_backend(), acl=UsbAcl(),
402+
viewer_id=getattr(self, "_viewer_id", None),
403+
)
404+
405+
def _enabled() -> bool:
406+
from je_auto_control.utils.usb.passthrough import (
407+
is_usb_passthrough_enabled,
408+
)
409+
return bool(self._authenticated) and is_usb_passthrough_enabled()
410+
411+
self._usb_host = UsbChannelHost(
412+
channel, session_factory=_factory, enabled_check=_enabled,
413+
)
414+
381415
def set_file_received_callback(self, callback) -> None:
382416
"""Register a sync callback ``cb(path: Path)`` for completed transfers."""
383417
self._on_file_received = callback

0 commit comments

Comments
 (0)