[TEMP] feat: add interactive video stream widgets#13587
Draft
bigcat88 wants to merge 2 commits into
Draft
Conversation
🌐 Website E2ETip All tests passed.
|
🎨 Storybook: ✅ Built — View Storybook🎭 Playwright: ✅ 1690 passed, 0 failed · 4 flaky📊 Browser Reports
📦 Bundle: 7.83 MB gzip 🔴 +5.08 kBDetailsSummary
Category Glance App Entry Points — 53.2 kB (baseline 53.2 kB) • ⚪ 0 BMain entry bundles and manifests
Status: 1 added / 1 removed Graph Workspace — 1.27 MB (baseline 1.27 MB) • 🔴 +1 BGraph editor runtime, canvas, workflow orchestration
Status: 1 added / 1 removed Views & Navigation — 110 kB (baseline 110 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Status: 11 added / 11 removed / 3 unchanged Panels & Settings — 550 kB (baseline 550 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
Status: 11 added / 11 removed / 16 unchanged User & Accounts — 29.1 kB (baseline 29.1 kB) • ⚪ 0 BAuthentication, profile, and account management bundles
Status: 7 added / 7 removed / 3 unchanged Editors & Dialogs — 117 kB (baseline 117 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors
Status: 4 added / 4 removed / 1 unchanged UI Components — 57.5 kB (baseline 57.5 kB) • ⚪ 0 BReusable component library chunks
Status: 5 added / 5 removed / 8 unchanged Data & Services — 276 kB (baseline 269 kB) • 🔴 +6.34 kBStores, services, APIs, and repositories
Status: 13 added / 13 removed / 3 unchanged Utilities & Hooks — 3.41 MB (baseline 3.4 MB) • 🔴 +15.5 kBHelpers, composables, and utility bundles
Status: 16 added / 16 removed / 19 unchanged Vendor & Third-Party — 15.3 MB (baseline 15.3 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 16 unchanged Other — 11.8 MB (baseline 11.8 MB) • ⚪ 0 BBundles that do not match a named category
Status: 67 added / 67 removed / 98 unchanged ⚡ Performance Report
Show regressions
All metrics
Historical variance (last 15 runs)
Trend (last 15 commits on main)
Raw data{
"timestamp": "2026-07-11T12:38:51.533Z",
"gitSha": "81350a69d9abe5b87917473c602c63120e9f0bf0",
"branch": "feat/interactive-webcam-stream",
"measurements": [
{
"name": "canvas-idle",
"durationMs": 2111.7079999999933,
"styleRecalcs": 8,
"styleRecalcDurationMs": 8.083999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 467.62500000000006,
"heapDeltaBytes": -3045612,
"heapUsedBytes": 49662432,
"domNodes": -290,
"jsHeapTotalBytes": 17166336,
"scriptDurationMs": 18.46,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-idle",
"durationMs": 2055.534000000023,
"styleRecalcs": 9,
"styleRecalcDurationMs": 8.805999999999997,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 413.33299999999997,
"heapDeltaBytes": -7499444,
"heapUsedBytes": 62541368,
"domNodes": -284,
"jsHeapTotalBytes": 19546112,
"scriptDurationMs": 16.075000000000003,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1847.1989999999892,
"styleRecalcs": 72,
"styleRecalcDurationMs": 43.989,
"layouts": 12,
"layoutDurationMs": 3.603,
"taskDurationMs": 855.515,
"heapDeltaBytes": -12736288,
"heapUsedBytes": 57304748,
"domNodes": -242,
"jsHeapTotalBytes": 19021824,
"scriptDurationMs": 127.04900000000002,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1882.0279999999912,
"styleRecalcs": 76,
"styleRecalcDurationMs": 43.693000000000005,
"layouts": 12,
"layoutDurationMs": 4.067,
"taskDurationMs": 823.0189999999999,
"heapDeltaBytes": -4703312,
"heapUsedBytes": 47289548,
"domNodes": -272,
"jsHeapTotalBytes": 17952768,
"scriptDurationMs": 124.01500000000001,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1757.6649999999745,
"styleRecalcs": 30,
"styleRecalcDurationMs": 18.374000000000002,
"layouts": 6,
"layoutDurationMs": 0.738,
"taskDurationMs": 363.73900000000003,
"heapDeltaBytes": -3646572,
"heapUsedBytes": 66433776,
"domNodes": -226,
"jsHeapTotalBytes": 12730368,
"scriptDurationMs": 19.046999999999997,
"eventListeners": -184,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1753.1970000000001,
"styleRecalcs": 33,
"styleRecalcDurationMs": 19.386999999999997,
"layouts": 6,
"layoutDurationMs": 0.593,
"taskDurationMs": 388.562,
"heapDeltaBytes": 982060,
"heapUsedBytes": 53175136,
"domNodes": -240,
"jsHeapTotalBytes": 6942720,
"scriptDurationMs": 26.110999999999997,
"eventListeners": -186,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "dom-widget-clipping",
"durationMs": 612.3890000000074,
"styleRecalcs": 10,
"styleRecalcDurationMs": 7.616000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 394.04699999999997,
"heapDeltaBytes": 9085656,
"heapUsedBytes": 61205092,
"domNodes": -297,
"jsHeapTotalBytes": 7729152,
"scriptDurationMs": 59.471000000000004,
"eventListeners": -203,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "dom-widget-clipping",
"durationMs": 600.3679999999463,
"styleRecalcs": 11,
"styleRecalcDurationMs": 9.822,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 362.279,
"heapDeltaBytes": 10026892,
"heapUsedBytes": 58823860,
"domNodes": 18,
"jsHeapTotalBytes": 18874368,
"scriptDurationMs": 61.373,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.663333333333338,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-idle",
"durationMs": 2005.6140000000369,
"styleRecalcs": 10,
"styleRecalcDurationMs": 10.547,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 491.706,
"heapDeltaBytes": -10698420,
"heapUsedBytes": 65242368,
"domNodes": 20,
"jsHeapTotalBytes": 7745536,
"scriptDurationMs": 87.601,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-idle",
"durationMs": 2046.671999999944,
"styleRecalcs": 8,
"styleRecalcDurationMs": 7.758999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 554.112,
"heapDeltaBytes": -8522548,
"heapUsedBytes": 59649288,
"domNodes": -271,
"jsHeapTotalBytes": -1847296,
"scriptDurationMs": 101.625,
"eventListeners": -197,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-pan",
"durationMs": 2130.1179999999817,
"styleRecalcs": 69,
"styleRecalcDurationMs": 18.451999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1229.268,
"heapDeltaBytes": 14362724,
"heapUsedBytes": 83899812,
"domNodes": -273,
"jsHeapTotalBytes": -856064,
"scriptDurationMs": 464.38500000000005,
"eventListeners": -197,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2156.358999999952,
"styleRecalcs": 68,
"styleRecalcDurationMs": 20.147000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1180.2510000000002,
"heapDeltaBytes": 18878696,
"heapUsedBytes": 79877152,
"domNodes": -265,
"jsHeapTotalBytes": 5169152,
"scriptDurationMs": 404.506,
"eventListeners": -128,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "large-graph-zoom",
"durationMs": 3258.7369999999964,
"styleRecalcs": 62,
"styleRecalcDurationMs": 16.906000000000006,
"layouts": 60,
"layoutDurationMs": 8.238000000000001,
"taskDurationMs": 1493.318,
"heapDeltaBytes": 17020872,
"heapUsedBytes": 71810800,
"domNodes": 4,
"jsHeapTotalBytes": 1310720,
"scriptDurationMs": 580.0419999999999,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-zoom",
"durationMs": 3164.4659999999476,
"styleRecalcs": 65,
"styleRecalcDurationMs": 20.409999999999997,
"layouts": 60,
"layoutDurationMs": 8.02,
"taskDurationMs": 1414.7409999999998,
"heapDeltaBytes": 15992016,
"heapUsedBytes": 74283992,
"domNodes": 12,
"jsHeapTotalBytes": 7864320,
"scriptDurationMs": 513.894,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.670000000000012,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2036.5700000000402,
"styleRecalcs": 9,
"styleRecalcDurationMs": 8.5,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 495.041,
"heapDeltaBytes": -10394400,
"heapUsedBytes": 64741784,
"domNodes": 18,
"jsHeapTotalBytes": 10366976,
"scriptDurationMs": 88.41600000000001,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "minimap-idle",
"durationMs": 2032.2909999999865,
"styleRecalcs": 6,
"styleRecalcDurationMs": 6.328,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 626.8500000000001,
"heapDeltaBytes": -16983468,
"heapUsedBytes": 59591588,
"domNodes": -276,
"jsHeapTotalBytes": 3743744,
"scriptDurationMs": 113.446,
"eventListeners": -197,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 606.1699999999632,
"styleRecalcs": 45,
"styleRecalcDurationMs": 10.713000000000003,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 407.567,
"heapDeltaBytes": 9261032,
"heapUsedBytes": 61750880,
"domNodes": -293,
"jsHeapTotalBytes": 9302016,
"scriptDurationMs": 122.63700000000001,
"eventListeners": -195,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 605.3070000000389,
"styleRecalcs": 46,
"styleRecalcDurationMs": 11.551999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 419.859,
"heapDeltaBytes": 12247460,
"heapUsedBytes": 64675556,
"domNodes": -293,
"jsHeapTotalBytes": 4583424,
"scriptDurationMs": 126.616,
"eventListeners": -197,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-idle",
"durationMs": 2013.8539999999807,
"styleRecalcs": 9,
"styleRecalcDurationMs": 8.756,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 379.194,
"heapDeltaBytes": -6225612,
"heapUsedBytes": 46275080,
"domNodes": -291,
"jsHeapTotalBytes": 16904192,
"scriptDurationMs": 13.781,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-idle",
"durationMs": 2015.2410000000032,
"styleRecalcs": 9,
"styleRecalcDurationMs": 9.034,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 400.61600000000004,
"heapDeltaBytes": -3790396,
"heapUsedBytes": 49106768,
"domNodes": -291,
"jsHeapTotalBytes": 16642048,
"scriptDurationMs": 16.220000000000006,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1723.2500000000073,
"styleRecalcs": 75,
"styleRecalcDurationMs": 42.314,
"layouts": 16,
"layoutDurationMs": 4.893,
"taskDurationMs": 755.5840000000001,
"heapDeltaBytes": -16744392,
"heapUsedBytes": 53343584,
"domNodes": -236,
"jsHeapTotalBytes": 20070400,
"scriptDurationMs": 97.97,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1738.6129999999866,
"styleRecalcs": 77,
"styleRecalcDurationMs": 46.446,
"layouts": 16,
"layoutDurationMs": 7.855999999999998,
"taskDurationMs": 765.2110000000001,
"heapDeltaBytes": -16941016,
"heapUsedBytes": 53165204,
"domNodes": -235,
"jsHeapTotalBytes": 18497536,
"scriptDurationMs": 96.69800000000001,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "subgraph-transition-enter",
"durationMs": 1098.9260000000058,
"styleRecalcs": 17,
"styleRecalcDurationMs": 33.51,
"layouts": 13,
"layoutDurationMs": 15.94,
"taskDurationMs": 879.1190000000003,
"heapDeltaBytes": 891084,
"heapUsedBytes": 76196524,
"domNodes": 12230,
"jsHeapTotalBytes": 12800000,
"scriptDurationMs": 32.471,
"eventListeners": 842,
"totalBlockingTimeMs": 167,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "viewport-pan-sweep",
"durationMs": 8165.61200000001,
"styleRecalcs": 250,
"styleRecalcDurationMs": 61.206,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4166.642,
"heapDeltaBytes": -1072916,
"heapUsedBytes": 71594224,
"domNodes": 18,
"jsHeapTotalBytes": 18145280,
"scriptDurationMs": 1335.194,
"eventListeners": 20,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "viewport-pan-sweep",
"durationMs": 8281.557999999904,
"styleRecalcs": 250,
"styleRecalcDurationMs": 57.608999999999995,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4232.648,
"heapDeltaBytes": 20046108,
"heapUsedBytes": 80096356,
"domNodes": -263,
"jsHeapTotalBytes": 4907008,
"scriptDurationMs": 1387.574,
"eventListeners": -124,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-idle",
"durationMs": 12393.40199999998,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 12363.984999999999,
"heapDeltaBytes": -45173696,
"heapUsedBytes": 167059816,
"domNodes": -3300,
"jsHeapTotalBytes": -8155136,
"scriptDurationMs": 555.8019999999999,
"eventListeners": -16380,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.219999999999953,
"p95FrameDurationMs": 16.80000000000291
},
{
"name": "vue-large-graph-idle",
"durationMs": 12905.342000000019,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 12887.622000000001,
"heapDeltaBytes": -72159632,
"heapUsedBytes": 156205988,
"domNodes": -3300,
"jsHeapTotalBytes": 7573504,
"scriptDurationMs": 580.912,
"eventListeners": -16376,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.219999999999953,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-pan",
"durationMs": 15831.788000000017,
"styleRecalcs": 82,
"styleRecalcDurationMs": 20.380999999999982,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 15804.822000000002,
"heapDeltaBytes": -60760312,
"heapUsedBytes": 168436788,
"domNodes": -8313,
"jsHeapTotalBytes": 15462400,
"scriptDurationMs": 842.549,
"eventListeners": -16368,
"totalBlockingTimeMs": 55,
"frameDurationMs": 17.77333333333336,
"p95FrameDurationMs": 16.80000000000291
},
{
"name": "vue-large-graph-pan",
"durationMs": 16141.188000000056,
"styleRecalcs": 87,
"styleRecalcDurationMs": 21.592999999999975,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 16112.490999999998,
"heapDeltaBytes": -12682700,
"heapUsedBytes": 178681372,
"domNodes": -3300,
"jsHeapTotalBytes": 14856192,
"scriptDurationMs": 853.2590000000001,
"eventListeners": -16364,
"totalBlockingTimeMs": 1,
"frameDurationMs": 17.22666666666664,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "workflow-execution",
"durationMs": 454.02899999999136,
"styleRecalcs": 12,
"styleRecalcDurationMs": 19.060000000000002,
"layouts": 3,
"layoutDurationMs": 0.6950000000000001,
"taskDurationMs": 104.428,
"heapDeltaBytes": 4886680,
"heapUsedBytes": 57115480,
"domNodes": 128,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 9.943999999999999,
"eventListeners": 65,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "workflow-execution",
"durationMs": 493.9679999999953,
"styleRecalcs": 12,
"styleRecalcDurationMs": 29.14,
"layouts": 3,
"layoutDurationMs": 0.7100000000000001,
"taskDurationMs": 200.499,
"heapDeltaBytes": -22893200,
"heapUsedBytes": 46232040,
"domNodes": -208,
"jsHeapTotalBytes": 6438912,
"scriptDurationMs": 11.924000000000001,
"eventListeners": -132,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
}
]
} |
Contributor
📝 WalkthroughWalkthroughThe changes add interactive UI schemas and migration support, a validated websocket interaction protocol, webcam-based interactive sessions, connected-client prompt coordination, feature-flag wiring, and subgraph node-removal lifecycle cleanup. ChangesInteractive streaming
Subgraph cleanup
Estimated code review effort: 5 (Critical) | ~90 minutes Sequence Diagram(s)sequenceDiagram
participant InteractiveViewService
participant ComfyApi
participant BackendWebSocket
participant InteractionProtocol
InteractiveViewService->>ComfyApi: Send interaction control
ComfyApi->>BackendWebSocket: Send JSON interaction message
BackendWebSocket->>ComfyApi: Receive interaction control
ComfyApi->>InteractionProtocol: Parse control
ComfyApi->>InteractiveViewService: Dispatch interaction event
InteractiveViewService->>ComfyApi: Send captured media
ComfyApi->>InteractionProtocol: Encode media
ComfyApi->>BackendWebSocket: Send binary media
BackendWebSocket->>ComfyApi: Receive interaction media
ComfyApi->>InteractionProtocol: Decode media
ComfyApi->>InteractiveViewService: Dispatch interaction_media event
Suggested reviewers: Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
bigcat88
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/services/interactionProtocol.ts`:
- Around line 100-126: Resolve the empty-media contract mismatch between
encodeInteractionMedia and decodeInteractionMedia. Either reject zero-length
media in encodeInteractionMedia, or adjust decodeInteractionMedia’s
metadata-length validation so buffers containing valid metadata with no media
payload are accepted; keep truncation and size checks consistent.
In `@src/services/interactiveViewService.test.ts`:
- Around line 115-262: Add tests covering the feature’s untested error and edge
branches: mock getUserMedia rejection and assert the camera-access message, open
an interaction without an active stream and assert a cancel control, and emit an
oversized interaction_media blob after opening a session with max_frame_bytes
and assert cancellation. Reuse the existing createNode, startWebcam, emit, and
mocks.controls helpers, and clean up each session.
In `@src/services/interactiveViewService.ts`:
- Around line 474-485: Replace the hardcoded webcam labels and status messages
in the interactive view construction with keys added to
src/locales/en/main.json, and retrieve them through the Vue I18n composition API
within the relevant service flow. Update the Start/Stop button text and both
status branches to use the localized translations while preserving the existing
feature-support logic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d6a038f9-02e0-4fa7-b3b3-158f55bfe578
📒 Files selected for processing (14)
packages/object-info-parser/src/__tests__/nodeDefSchema.test.tspackages/object-info-parser/src/schemas/nodeDefSchema.tssrc/config/clientFeatureFlags.jsonsrc/lib/litegraph/src/LGraph.test.tssrc/lib/litegraph/src/LGraph.tssrc/schemas/nodeDef/migration.test.tssrc/schemas/nodeDef/migration.tssrc/schemas/nodeDef/nodeDefSchemaV2.tssrc/scripts/api.tssrc/services/interactionProtocol.test.tssrc/services/interactionProtocol.tssrc/services/interactiveViewService.test.tssrc/services/interactiveViewService.tssrc/services/litegraphService.ts
Comment on lines
+100
to
+126
| if (media.byteLength > MAX_INTERACTION_MEDIA_BYTES) | ||
| throw new Error('Interaction media is too large') | ||
| const result = new Uint8Array(8 + encoded.length + media.byteLength) | ||
| new DataView(result.buffer).setUint32(0, INTERACTION_MEDIA) | ||
| new DataView(result.buffer).setUint32(4, encoded.length) | ||
| result.set(encoded, 8) | ||
| result.set(new Uint8Array(media), 8 + encoded.length) | ||
| return result.buffer | ||
| } | ||
|
|
||
| export function decodeInteractionMedia(data: ArrayBuffer): { | ||
| metadata: InteractionMediaMetadata | ||
| media: ArrayBuffer | ||
| } { | ||
| if (data.byteLength < 9) throw new Error('Truncated interaction media') | ||
| const view = new DataView(data) | ||
| if (view.getUint32(0) !== INTERACTION_MEDIA) | ||
| throw new Error('Invalid interaction media type') | ||
| const length = view.getUint32(4) | ||
| if ( | ||
| length === 0 || | ||
| length > MAX_INTERACTION_METADATA_BYTES || | ||
| 8 + length >= data.byteLength | ||
| ) | ||
| throw new Error('Invalid interaction metadata length') | ||
| if (data.byteLength - 8 - length > MAX_INTERACTION_MEDIA_BYTES) | ||
| throw new Error('Interaction media is too large') |
Contributor
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Encode/decode disagree on empty media.
encodeInteractionMedia places no lower bound on media.byteLength, so encoding a 0-byte frame yields a buffer of size 8 + metadataLength. decodeInteractionMedia then rejects it, because Line 122's 8 + length >= data.byteLength is true when there is no media payload. The two ends of the contract are inconsistent for the empty-media case. In practice JPEG frames are non-empty, so impact is limited, but the asymmetry is latent. Either forbid empty media in encode or relax the decode bound.
Proposed decode relaxation
- if (
- length === 0 ||
- length > MAX_INTERACTION_METADATA_BYTES ||
- 8 + length >= data.byteLength
- )
+ if (
+ length === 0 ||
+ length > MAX_INTERACTION_METADATA_BYTES ||
+ 8 + length > data.byteLength
+ )
throw new Error('Invalid interaction metadata length')📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if (media.byteLength > MAX_INTERACTION_MEDIA_BYTES) | |
| throw new Error('Interaction media is too large') | |
| const result = new Uint8Array(8 + encoded.length + media.byteLength) | |
| new DataView(result.buffer).setUint32(0, INTERACTION_MEDIA) | |
| new DataView(result.buffer).setUint32(4, encoded.length) | |
| result.set(encoded, 8) | |
| result.set(new Uint8Array(media), 8 + encoded.length) | |
| return result.buffer | |
| } | |
| export function decodeInteractionMedia(data: ArrayBuffer): { | |
| metadata: InteractionMediaMetadata | |
| media: ArrayBuffer | |
| } { | |
| if (data.byteLength < 9) throw new Error('Truncated interaction media') | |
| const view = new DataView(data) | |
| if (view.getUint32(0) !== INTERACTION_MEDIA) | |
| throw new Error('Invalid interaction media type') | |
| const length = view.getUint32(4) | |
| if ( | |
| length === 0 || | |
| length > MAX_INTERACTION_METADATA_BYTES || | |
| 8 + length >= data.byteLength | |
| ) | |
| throw new Error('Invalid interaction metadata length') | |
| if (data.byteLength - 8 - length > MAX_INTERACTION_MEDIA_BYTES) | |
| throw new Error('Interaction media is too large') | |
| if ( | |
| length === 0 || | |
| length > MAX_INTERACTION_METADATA_BYTES || | |
| 8 + length > data.byteLength | |
| ) | |
| throw new Error('Invalid interaction metadata length') |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/interactionProtocol.ts` around lines 100 - 126, Resolve the
empty-media contract mismatch between encodeInteractionMedia and
decodeInteractionMedia. Either reject zero-length media in
encodeInteractionMedia, or adjust decodeInteractionMedia’s metadata-length
validation so buffers containing valid metadata with no media payload are
accepted; keep truncation and size checks consistent.
Comment on lines
+115
to
+262
| describe('interactive view sessions', () => { | ||
| it('does not request camera access when interactions are unavailable', () => { | ||
| mocks.serverSupportsFeature.mockReturnValue(false) | ||
| const { node, element } = createNode() | ||
|
|
||
| const start = element.querySelector('button')! | ||
| expect(start.hasAttribute('disabled')).toBe(true) | ||
| start.click() | ||
| expect(navigator.mediaDevices.getUserMedia).not.toHaveBeenCalled() | ||
|
|
||
| node.onRemoved?.() | ||
| }) | ||
|
|
||
| it('keeps prompt routing stable and resends cancellation after reconnect', async () => { | ||
| const { node, element } = createNode() | ||
| await startWebcam(element) | ||
| const createBitmap = vi.fn().mockResolvedValue({ | ||
| width: 1, | ||
| height: 1, | ||
| close: vi.fn() | ||
| }) | ||
| vi.stubGlobal('createImageBitmap', createBitmap) | ||
| vi.spyOn(HTMLCanvasElement.prototype, 'getContext').mockReturnValue({ | ||
| drawImage: vi.fn() | ||
| } as unknown as GPUCanvasContext) | ||
|
|
||
| emit('interaction', { | ||
| v: 1, | ||
| op: 'open', | ||
| interaction_id: 'interaction-a', | ||
| prompt_id: 'prompt-a', | ||
| display_node_id: '12', | ||
| group_id: 'video', | ||
| limits: { max_frame_bytes: 10 } | ||
| }) | ||
| expect(mocks.controls).toContainEqual( | ||
| expect.objectContaining({ op: 'ready', prompt_id: 'prompt-a' }) | ||
| ) | ||
|
|
||
| emit('interaction_media', { | ||
| metadata: { | ||
| v: 1, | ||
| interaction_id: 'interaction-a', | ||
| channel: 'result', | ||
| seq: 0, | ||
| capture_ts_ms: 1, | ||
| mime: 'image/jpeg' | ||
| }, | ||
| blob: new Blob([new Uint8Array(1)], { type: 'image/jpeg' }) | ||
| }) | ||
| await vi.waitFor(() => expect(createBitmap).toHaveBeenCalledOnce()) | ||
|
|
||
| const controlCount = mocks.controls.length | ||
| emit('interaction', { | ||
| v: 1, | ||
| op: 'resume', | ||
| interaction_id: 'interaction-a', | ||
| prompt_id: 'prompt-b' | ||
| }) | ||
| expect(mocks.controls).toHaveLength(controlCount) | ||
|
|
||
| emit('interaction', { | ||
| v: 1, | ||
| op: 'resume', | ||
| interaction_id: 'interaction-a', | ||
| prompt_id: 'prompt-a' | ||
| }) | ||
| emit('interaction_media', { | ||
| metadata: { | ||
| v: 1, | ||
| interaction_id: 'interaction-a', | ||
| channel: 'result', | ||
| seq: 1, | ||
| capture_ts_ms: 2, | ||
| mime: 'image/jpeg' | ||
| }, | ||
| blob: new Blob([new Uint8Array(11)], { type: 'image/jpeg' }) | ||
| }) | ||
| emit('interaction_media', { | ||
| metadata: { | ||
| v: 1, | ||
| interaction_id: 'interaction-a', | ||
| channel: 'result', | ||
| seq: 0, | ||
| capture_ts_ms: 1, | ||
| mime: 'image/jpeg' | ||
| }, | ||
| blob: new Blob([new Uint8Array(1)], { type: 'image/jpeg' }) | ||
| }) | ||
| await Promise.resolve() | ||
| expect(createBitmap).toHaveBeenCalledOnce() | ||
|
|
||
| vi.useFakeTimers() | ||
| emit('reconnecting') | ||
| await vi.advanceTimersByTimeAsync(6500) | ||
| expect(mocks.controls.at(-1)).toMatchObject({ | ||
| op: 'cancel', | ||
| prompt_id: 'prompt-a' | ||
| }) | ||
|
|
||
| const cancellationCount = mocks.controls.length | ||
| emit('reconnected') | ||
| expect(mocks.controls).toHaveLength(cancellationCount) | ||
| emit('feature_flags') | ||
| expect(mocks.controls.at(-1)).toMatchObject({ | ||
| op: 'cancel', | ||
| prompt_id: 'prompt-a' | ||
| }) | ||
|
|
||
| node.onRemoved?.() | ||
| await vi.runAllTimersAsync() | ||
| }) | ||
|
|
||
| it('cancels an active stream when reconnect negotiation drops support', async () => { | ||
| const { node, element } = createNode() | ||
| await startWebcam(element) | ||
| emit('interaction', { | ||
| v: 1, | ||
| op: 'open', | ||
| interaction_id: 'interaction-b', | ||
| prompt_id: 'prompt-b', | ||
| display_node_id: '12', | ||
| group_id: 'video' | ||
| }) | ||
|
|
||
| emit('reconnecting') | ||
| emit('reconnected') | ||
| mocks.serverSupportsFeature.mockReturnValue(false) | ||
| emit('feature_flags') | ||
|
|
||
| expect(mocks.controls.at(-1)).toMatchObject({ | ||
| op: 'cancel', | ||
| interaction_id: 'interaction-b', | ||
| prompt_id: 'prompt-b' | ||
| }) | ||
| expect(element.textContent).toContain( | ||
| 'Interactive streaming is unavailable.' | ||
| ) | ||
|
|
||
| emit('interaction', { | ||
| v: 1, | ||
| op: 'closed', | ||
| interaction_id: 'interaction-b', | ||
| prompt_id: 'prompt-b' | ||
| }) | ||
| node.onRemoved?.() | ||
| }) | ||
| }) |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Consider adding error/edge-path coverage.
The suite covers availability gating and reconnect negotiation well, but doesn't exercise the failure paths introduced by the feature: oversized captured frames (blob.size > maxFrameBytes → cancel), camera-denied (getUserMedia rejection → "Allow camera access to start."), or a session with no active stream receiving open (→ cancel). Adding these would guard the cancellation/error branches.
As per path instructions, cover "important edge cases introduced by the feature (e.g., invalid/malformed inputs, error paths, capability/negotiation failures)."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/interactiveViewService.test.ts` around lines 115 - 262, Add
tests covering the feature’s untested error and edge branches: mock getUserMedia
rejection and assert the camera-access message, open an interaction without an
active stream and assert a cancel control, and emit an oversized
interaction_media blob after opening a session with max_frame_bytes and assert
cancellation. Reuse the existing createNode, startWebcam, emit, and
mocks.controls helpers, and clean up each session.
Source: Path instructions
Comment on lines
+474
to
+485
| const startButton = document.createElement('button') | ||
| startButton.className = 'rounded bg-primary px-3 py-1 text-primary-foreground' | ||
| startButton.textContent = 'Start Webcam' | ||
| const stopButton = document.createElement('button') | ||
| stopButton.className = | ||
| 'rounded bg-secondary px-3 py-1 text-secondary-foreground' | ||
| stopButton.textContent = 'Stop Stream' | ||
| const status = document.createElement('span') | ||
| status.className = 'text-muted-foreground' | ||
| status.textContent = api.serverSupportsFeature('supports_interactions_v1') | ||
| ? 'Webcam stopped.' | ||
| : 'Interactive streaming is unavailable.' |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## interactiveViewService.ts excerpt\n'
sed -n '430,540p' src/services/interactiveViewService.ts | cat -n
printf '\n## locale search\n'
rg -n "Start Webcam|Stop Stream|Webcam stopped|Interactive streaming is unavailable|interactive|webcam|stream" src/locales/en/main.json src/services/interactiveViewService.ts || trueRepository: Comfy-Org/ComfyUI_frontend
Length of output: 8581
Route interactive view labels and status text through vue-i18n. The webcam start/stop labels and status messages in src/services/interactiveViewService.ts are still hardcoded; move them to src/locales/en/main.json and read them from the composition API.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/interactiveViewService.ts` around lines 474 - 485, Replace the
hardcoded webcam labels and status messages in the interactive view construction
with keys added to src/locales/en/main.json, and retrieve them through the Vue
I18n composition API within the relevant service flow. Update the Start/Stop
button text and both status branches to use the localized translations while
preserving the existing feature-support logic.
Source: Coding guidelines
bigcat88
added
the
preview-gpu
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/scripts/api.ts (1)
953-968: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMagic backpressure threshold.
2 * 1024 * 1024bufferedAmount threshold is inlined. Extracting it to a named constant (e.g.INTERACTION_MEDIA_BACKPRESSURE_BYTES) would make the backpressure policy self-documenting and easier to tune.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/scripts/api.ts` around lines 953 - 968, Extract the inlined 2 * 1024 * 1024 bufferedAmount limit in sendInteractionMedia into a clearly named constant such as INTERACTION_MEDIA_BACKPRESSURE_BYTES, and use that constant in the backpressure condition without changing the existing behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/scripts/api.ts`:
- Around line 1109-1123: The waitForSocketSession method uses timer-based
polling for confirmation; replace this loop with an event- or promise-based wait
that resolves when the message handler’s status branch sets confirmedSocket and
confirmedClientId. Preserve the existing timeout behavior and
WebSocket/open-session validation, and reuse the confirmation state symbols
rather than introducing polling.
In `@src/scripts/app.ts`:
- Around line 1670-1678: In the requiresConnectedClient calculation, hoist
useNodeDefStore() into a local nodeDefStore constant before
Object.values(p.output).some, then access nodeDefsByName through that variable
inside the callback while preserving the existing interactive_ui check.
---
Outside diff comments:
In `@src/scripts/api.ts`:
- Around line 953-968: Extract the inlined 2 * 1024 * 1024 bufferedAmount limit
in sendInteractionMedia into a clearly named constant such as
INTERACTION_MEDIA_BACKPRESSURE_BYTES, and use that constant in the backpressure
condition without changing the existing behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 474ee3af-cae6-4d67-a2a7-1c5a291a6afc
📒 Files selected for processing (4)
src/scripts/api.featureFlags.test.tssrc/scripts/api.tssrc/scripts/app.tssrc/stores/nodeDefStore.ts
Comment on lines
+1109
to
+1123
| private async waitForSocketSession( | ||
| timeout = 5000 | ||
| ): Promise<string | undefined> { | ||
| const deadline = Date.now() + timeout | ||
| while (Date.now() < deadline) { | ||
| if ( | ||
| this.socket && | ||
| this.socket === this.confirmedSocket && | ||
| this.socket.readyState === WebSocket.OPEN | ||
| ) { | ||
| return this.confirmedClientId | ||
| } | ||
| await new Promise((resolve) => window.setTimeout(resolve, 50)) | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win
Busy-polling for socket confirmation.
waitForSocketSession polls this.confirmedSocket/readyState every 50ms for up to 5s. Since confirmation is set synchronously inside the message handler's status branch, an event/promise-based wait (e.g. resolving a pending promise from that branch, or awaiting a one-shot custom event) would avoid up to 50ms of wasted latency per call and avoid a timer-based loop.
#!/bin/bash
rg -n "confirmedSocket|confirmedClientId" src/scripts/api.ts🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/scripts/api.ts` around lines 1109 - 1123, The waitForSocketSession method
uses timer-based polling for confirmation; replace this loop with an event- or
promise-based wait that resolves when the message handler’s status branch sets
confirmedSocket and confirmedClientId. Preserve the existing timeout behavior
and WebSocket/open-session validation, and reuse the confirmation state symbols
rather than introducing polling.
Comment on lines
+1670
to
+1678
| const requiresConnectedClient = Object.values(p.output).some( | ||
| (node) => | ||
| !!useNodeDefStore().nodeDefsByName[node.class_type] | ||
| ?.interactive_ui?.length | ||
| ) | ||
| const res = await api.queuePrompt(number, p, { | ||
| partialExecutionTargets: queueNodeIds, | ||
| previewMethod | ||
| previewMethod, | ||
| requireConnectedClient: requiresConnectedClient |
Contributor
There was a problem hiding this comment.
🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value
Hoist useNodeDefStore() out of the .some() callback.
useNodeDefStore() is called once per node in p.output. Hoisting it to a local const nodeDefStore = useNodeDefStore() before .some() avoids repeated store-instance lookups per node.
♻️ Proposed fix
- const requiresConnectedClient = Object.values(p.output).some(
- (node) =>
- !!useNodeDefStore().nodeDefsByName[node.class_type]
- ?.interactive_ui?.length
- )
+ const nodeDefStore = useNodeDefStore()
+ const requiresConnectedClient = Object.values(p.output).some(
+ (node) =>
+ !!nodeDefStore.nodeDefsByName[node.class_type]
+ ?.interactive_ui?.length
+ )📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| const requiresConnectedClient = Object.values(p.output).some( | |
| (node) => | |
| !!useNodeDefStore().nodeDefsByName[node.class_type] | |
| ?.interactive_ui?.length | |
| ) | |
| const res = await api.queuePrompt(number, p, { | |
| partialExecutionTargets: queueNodeIds, | |
| previewMethod | |
| previewMethod, | |
| requireConnectedClient: requiresConnectedClient | |
| const nodeDefStore = useNodeDefStore() | |
| const requiresConnectedClient = Object.values(p.output).some( | |
| (node) => | |
| !!nodeDefStore.nodeDefsByName[node.class_type] | |
| ?.interactive_ui?.length | |
| ) | |
| const res = await api.queuePrompt(number, p, { | |
| partialExecutionTargets: queueNodeIds, | |
| previewMethod, | |
| requireConnectedClient: requiresConnectedClient |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/scripts/app.ts` around lines 1670 - 1678, In the requiresConnectedClient
calculation, hoist useNodeDefStore() into a local nodeDefStore constant before
Object.values(p.output).some, then access nodeDefsByName through that variable
inside the callback while preserving the existing interactive_ui check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a first-class interactive node surface for bidirectional video streaming over ComfyUI's existing WebSocket connection.
The demo renderer shows a local webcam preview and a backend-processed result side by side. Python node definitions declare the surface through
interactive_ui, so the same frontend implementation can be used by local ComfyUI and Comfy Cloud.Protocol
prompt_idrouting for cloud job ownershipCloud can support this without a separate frontend branch by relaying the same interaction messages behind its existing
/wsendpoint.Demo behavior
VIDEOoutput.Requires the corresponding draft ComfyUI backend interaction broker and demo node changes.
Validation
pnpm typecheckpnpm knip --cache