Skip to content

[experimental][draft] in-browser agent for playing with ComfyUI#11547

Draft
snomiao wants to merge 10 commits into
sno-frontend-previewfrom
sno-agent
Draft

[experimental][draft] in-browser agent for playing with ComfyUI#11547
snomiao wants to merge 10 commits into
sno-frontend-previewfrom
sno-agent

Conversation

@snomiao

@snomiao snomiao commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

⚠️ Experimental, draft, fast-iterating. Expect breaking changes,
rebases, and dead-ends. The shape will keep shifting until it settles.
Reviews welcome but please don't block on individual lines.

What this is

A frontend-only, in-browser LLM agent (ComfyAI) that lets users drive
ComfyUI through natural language. Lives entirely in src/agent/
no backend changes required for the core experience. Cloud API
nodes (Tripo / Tencent / Meshy / Gemini / etc.) reuse the existing
Comfy Cloud auth flow.

┌─────────────────────────────────────────────────────────────┐
│  AgentRoot.vue ── FAB + xterm-driven chat panel (`c` keybind)│
│       │                                                       │
│       ├── XtermPanel.vue ── readline + history replay         │
│       └── useAgentSession ── Vercel AI SDK ── streamText      │
│                  │                                             │
│                  └── run_shell tool ── shell runtime           │
│                         │                                       │
│                         ├── coreutils                           │
│                         ├── Comfy.* command dispatch            │
│                         ├── graph / queue / state introspection │
│                         ├── workflow + node ops                 │
│                         ├── templates + image pipes             │
│                         ├── cloud install + Gemini validate     │
│                         └── run-js fallback                     │
└─────────────────────────────────────────────────────────────┘

What you can ask it today

  • "build a basic SD1.5 text-to-image workflow" — adds nodes, wires
    sockets by name, sets prompts, runs.
  • "give me a 3D model of a tiny gamer room with a tree outside the
    window"
    — chains local SD1.5 → Tripo image-to-3D → Preview3D, with
    Gemini visual validation between phases.
  • "list all orphan nodes" — composes a graph traversal in run-js,
    cross-checks the result with an independent verifier.
  • "install hunyuan_3d_v2.1.safetensors" — looks up the URL in
    ComfyUI-Manager's curated DB, queues + monitors the download.
  • "see" — captures the canvas, sends it to gemini-3-1-pro for
    visual analysis (catch disconnected nodes, missing previews, etc).

Architecture highlights

  • Single-tool LLM. The model only ever calls run_shell. All
    capability lives behind shell-style commands — keeps the tool
    surface small and predictable.
  • Composable shell. Pipes, &&/||, redirection, run-js for the
    long tail. The agent composes deterministic commands instead of
    inventing tool args.
  • Programmatic guardrails (not prompt rules). Promissory-stop
    detection, silent-fail detection, fragile-shell-idiom blocklist,
    and definitive-claim verifier all live in code so they survive
    prompt drift and model variance.
  • IndexedDB-backed history. Chat persists across reloads, replays
    on reopen with a clear divider between past and current sessions.
  • Cloud where cheap, local where free. Hybrid pipelines (local
    SD1.5 → cloud Tripo → cloud Gemini validation) demonstrate the
    intended cost/quality tradeoff.

Known rough edges

  • xterm focus traps don't always survive global Comfy hotkeys; a few
    keystroke bleeds remain in edge cases (sidebars, after reload).
  • Cloud nodes need the backend launched with
    --comfy-api-base https://api.comfy.org (prod) or staging URL
    matching the frontend's Firebase config.
  • Layer 3 of the reasoning guardrails (structured JSON answers with
    provenance) deferred — needs SDK plumbing.
  • Verifier + idiom-blocklist registries are small; they'll grow as
    more failure modes surface in real use.
  • Some Comfy.* commands open modal dialogs the agent can't fill —
    most have shell-command bypasses now (save-as, rename-workflow,
    set-subgraph-desc, etc.), but coverage is incomplete.

Why open it now if it's still moving

  • The shape of the user-facing surface is stable (xterm panel, NL prompt,
    c keybind to toggle). What's iterating is the agent's internal
    reasoning quality and the breadth of shell commands.
  • Reviewers can pull and play with it locally; feedback on the shape
    matters more than line-level review at this stage.
  • Forcing the work onto a branch + PR also forces it to typecheck and
    lint clean against sno-frontend-preview, which we keep doing.

Try it online (no setup needed)

👉 https://sno-agent.comfy-ui.pages.dev/ — the Cloudflare Pages preview
tracks this branch. Press c (or click the floating ComfyAI button) to
open the agent. The first time you open it without an API key configured
you'll see a settings prompt — paste an OpenAI or OpenRouter key, choose
a model, done.

Try it locally

pnpm install
USE_PROD_CONFIG=true pnpm dev
# In the SPA, press `c` (or click the floating ComfyAI button)
# Type any natural-language ask. `help` lists all built-ins.

Out of scope (for now)

  • Mobile / responsive layout for the agent panel
  • E2E tests for the LLM-driven pipelines (deterministic shell tests
    exist; LLM behavior tests need replay infrastructure)
  • Manager-modal automation for asset/custom-node installs
  • Linear-app-mode integration

@coderabbitai

coderabbitai Bot commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9607ade9-abae-49e4-9cf5-ba65af3e6582

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sno-agent

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

🌐 Frontend Preview: ✅ Built — 🌐 Open Preview

Details

⏰ Completed at: 05/02/2026, 08:46:49 AM UTC

Links

@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 05/02/2026, 08:45:48 AM UTC

Links

@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

🎭 Playwright: ❌ 1296 passed, 17 failed · 1 flaky

❌ Failed Tests

📊 Browser Reports
  • chromium: View Report (✅ 1281 / ❌ 17 / ⚠️ 1 / ⏭️ 4)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 12 / ❌ 0 / ⚠️ 0 / ⏭️ 0)

@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

Website Preview: https://comfy-website-preview-pr-11547.vercel.app

This commit: https://website-frontend-eqk2i2i5f-comfyui.vercel.app

Last updated: 2026-04-23T11:36:51Z for a45e4b6

@codecov

codecov Bot commented Apr 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 24.81151% with 2194 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/agent/shell/commands/nodeOps.ts 0.00% 372 Missing ⚠️
src/agent/ui/FoldablePanel.vue 0.00% 263 Missing and 14 partials ⚠️
src/agent/llm/session.ts 0.00% 165 Missing ⚠️
src/agent/shell/commands/layout.ts 0.00% 150 Missing ⚠️
src/agent/composables/useAgentSession.ts 0.00% 113 Missing ⚠️
src/agent/composables/useLocalBridge.ts 0.00% 103 Missing ⚠️
src/agent/shell/commands/workflow.ts 0.00% 103 Missing ⚠️
src/agent/shell/commands/install.ts 0.00% 94 Missing ⚠️
src/agent/shell/commands/comfyNs.ts 0.00% 89 Missing ⚠️
src/agent/shell/commands/browser.ts 0.00% 80 Missing ⚠️
... and 24 more
@@                   Coverage Diff                    @@
##           sno-frontend-preview   #11547      +/-   ##
========================================================
- Coverage                 54.62%   47.80%   -6.83%     
========================================================
  Files                      1379     1408      +29     
  Lines                     70661    73254    +2593     
  Branches                  18781    20453    +1672     
========================================================
- Hits                      38601    35019    -3582     
- Misses                    31510    37617    +6107     
- Partials                    550      618      +68     
Flag Coverage Δ
unit 47.80% <24.81%> (-6.83%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/App.vue 0.00% <ø> (ø)
src/agent/shell/types.ts 100.00% <100.00%> (ø)
src/platform/connectionPanel/backendReachable.ts 100.00% <100.00%> (ø)
src/agent/shell/vfs/memory.ts 96.42% <96.42%> (ø)
src/agent/stores/agentStore.ts 96.00% <96.00%> (ø)
src/agent/shell/commands/comfy.ts 86.66% <86.66%> (ø)
src/agent/shell/commands/execution.ts 93.65% <93.65%> (ø)
src/agent/shell/parser.ts 93.58% <93.58%> (ø)
src/agent/shell/vfs/mount.ts 90.41% <90.41%> (ø)
src/agent/shell/runtime.ts 88.63% <88.63%> (ø)
... and 27 more

... and 175 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

📦 Bundle Size

⏳ Size data collection in progress…

⚡ Performance Report

canvas-idle: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 64.2 MB heap
canvas-mouse-sweep: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 60.0 MB heap
canvas-zoom-sweep: · 60.0 avg FPS · 59.9 P5 FPS ✅ (target: ≥52) · 0ms TBT · 67.8 MB heap
dom-widget-clipping: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 50.7 MB heap
large-graph-idle: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 87.9 MB heap
large-graph-pan: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 74.0 MB heap
large-graph-zoom: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 91.7 MB heap
minimap-idle: · 60.0 avg FPS · 59.9 P5 FPS ✅ (target: ≥52) · 0ms TBT · 59.5 MB heap
subgraph-dom-widget-clipping: · 60.0 avg FPS · 59.9 P5 FPS ✅ (target: ≥52) · 0ms TBT · 50.7 MB heap
subgraph-idle: · 60.0 avg FPS · 59.9 P5 FPS ✅ (target: ≥52) · 0ms TBT · 53.8 MB heap
subgraph-mouse-sweep: · 60.0 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 50.4 MB heap
viewport-pan-sweep: · 60.0 avg FPS · 59.9 P5 FPS ✅ (target: ≥52) · 0ms TBT · 77.5 MB heap
vue-large-graph-idle: · 58.1 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 0ms TBT · 163.4 MB heap
vue-large-graph-pan: · 58.1 avg FPS · 59.5 P5 FPS ✅ (target: ≥52) · 5ms TBT · 277.3 MB heap
workflow-execution: · 60.0 avg FPS · 59.9 P5 FPS ✅ (target: ≥52) · 0ms TBT · 49.8 MB heap

No baseline found — showing absolute values.

Metric Value
canvas-idle: avg frame time 17ms
canvas-idle: p95 frame time 17ms
canvas-idle: layout duration 0ms
canvas-idle: style recalc duration 9ms
canvas-idle: layout count 0
canvas-idle: style recalc count 8
canvas-idle: task duration 438ms
canvas-idle: script duration 23ms
canvas-idle: TBT 0ms
canvas-idle: heap used 64.2 MB
canvas-idle: DOM nodes 16
canvas-idle: event listeners 10
canvas-mouse-sweep: avg frame time 17ms
canvas-mouse-sweep: p95 frame time 17ms
canvas-mouse-sweep: layout duration 3ms
canvas-mouse-sweep: style recalc duration 43ms
canvas-mouse-sweep: layout count 12
canvas-mouse-sweep: style recalc count 79
canvas-mouse-sweep: task duration 912ms
canvas-mouse-sweep: script duration 140ms
canvas-mouse-sweep: TBT 0ms
canvas-mouse-sweep: heap used 60.0 MB
canvas-mouse-sweep: DOM nodes 62
canvas-mouse-sweep: event listeners 10
canvas-zoom-sweep: avg frame time 17ms
canvas-zoom-sweep: p95 frame time 17ms
canvas-zoom-sweep: layout duration 1ms
canvas-zoom-sweep: style recalc duration 18ms
canvas-zoom-sweep: layout count 6
canvas-zoom-sweep: style recalc count 31
canvas-zoom-sweep: task duration 315ms
canvas-zoom-sweep: script duration 24ms
canvas-zoom-sweep: TBT 0ms
canvas-zoom-sweep: heap used 67.8 MB
canvas-zoom-sweep: DOM nodes 78
canvas-zoom-sweep: event listeners 23
dom-widget-clipping: avg frame time 17ms
dom-widget-clipping: p95 frame time 17ms
dom-widget-clipping: layout duration 0ms
dom-widget-clipping: style recalc duration 8ms
dom-widget-clipping: layout count 0
dom-widget-clipping: style recalc count 12
dom-widget-clipping: task duration 328ms
dom-widget-clipping: script duration 61ms
dom-widget-clipping: TBT 0ms
dom-widget-clipping: heap used 50.7 MB
dom-widget-clipping: DOM nodes 20
dom-widget-clipping: event listeners 2
large-graph-idle: avg frame time 17ms
large-graph-idle: p95 frame time 17ms
large-graph-idle: layout duration 0ms
large-graph-idle: style recalc duration 9ms
large-graph-idle: layout count 0
large-graph-idle: style recalc count 10
large-graph-idle: task duration 594ms
large-graph-idle: script duration 102ms
large-graph-idle: TBT 0ms
large-graph-idle: heap used 87.9 MB
large-graph-idle: DOM nodes -260
large-graph-idle: event listeners -123
large-graph-pan: avg frame time 17ms
large-graph-pan: p95 frame time 17ms
large-graph-pan: layout duration 0ms
large-graph-pan: style recalc duration 19ms
large-graph-pan: layout count 0
large-graph-pan: style recalc count 68
large-graph-pan: task duration 1163ms
large-graph-pan: script duration 410ms
large-graph-pan: TBT 0ms
large-graph-pan: heap used 74.0 MB
large-graph-pan: DOM nodes -264
large-graph-pan: event listeners -123
large-graph-zoom: avg frame time 17ms
large-graph-zoom: p95 frame time 17ms
large-graph-zoom: layout duration 8ms
large-graph-zoom: style recalc duration 18ms
large-graph-zoom: layout count 60
large-graph-zoom: style recalc count 65
large-graph-zoom: task duration 1385ms
large-graph-zoom: script duration 504ms
large-graph-zoom: TBT 0ms
large-graph-zoom: heap used 91.7 MB
large-graph-zoom: DOM nodes -268
large-graph-zoom: event listeners -121
minimap-idle: avg frame time 17ms
minimap-idle: p95 frame time 17ms
minimap-idle: layout duration 0ms
minimap-idle: style recalc duration 8ms
minimap-idle: layout count 0
minimap-idle: style recalc count 8
minimap-idle: task duration 518ms
minimap-idle: script duration 90ms
minimap-idle: TBT 0ms
minimap-idle: heap used 59.5 MB
minimap-idle: DOM nodes -265
minimap-idle: event listeners -125
subgraph-dom-widget-clipping: avg frame time 17ms
subgraph-dom-widget-clipping: p95 frame time 17ms
subgraph-dom-widget-clipping: layout duration 0ms
subgraph-dom-widget-clipping: style recalc duration 11ms
subgraph-dom-widget-clipping: layout count 0
subgraph-dom-widget-clipping: style recalc count 47
subgraph-dom-widget-clipping: task duration 349ms
subgraph-dom-widget-clipping: script duration 121ms
subgraph-dom-widget-clipping: TBT 0ms
subgraph-dom-widget-clipping: heap used 50.7 MB
subgraph-dom-widget-clipping: DOM nodes 20
subgraph-dom-widget-clipping: event listeners 8
subgraph-idle: avg frame time 17ms
subgraph-idle: p95 frame time 17ms
subgraph-idle: layout duration 0ms
subgraph-idle: style recalc duration 9ms
subgraph-idle: layout count 0
subgraph-idle: style recalc count 10
subgraph-idle: task duration 415ms
subgraph-idle: script duration 18ms
subgraph-idle: TBT 0ms
subgraph-idle: heap used 53.8 MB
subgraph-idle: DOM nodes -260
subgraph-idle: event listeners -127
subgraph-mouse-sweep: avg frame time 17ms
subgraph-mouse-sweep: p95 frame time 17ms
subgraph-mouse-sweep: layout duration 4ms
subgraph-mouse-sweep: style recalc duration 46ms
subgraph-mouse-sweep: layout count 16
subgraph-mouse-sweep: style recalc count 84
subgraph-mouse-sweep: task duration 937ms
subgraph-mouse-sweep: script duration 99ms
subgraph-mouse-sweep: TBT 0ms
subgraph-mouse-sweep: heap used 50.4 MB
subgraph-mouse-sweep: DOM nodes -259
subgraph-mouse-sweep: event listeners -121
viewport-pan-sweep: avg frame time 17ms
viewport-pan-sweep: p95 frame time 17ms
viewport-pan-sweep: layout duration 0ms
viewport-pan-sweep: style recalc duration 51ms
viewport-pan-sweep: layout count 0
viewport-pan-sweep: style recalc count 250
viewport-pan-sweep: task duration 4026ms
viewport-pan-sweep: script duration 1311ms
viewport-pan-sweep: TBT 0ms
viewport-pan-sweep: heap used 77.5 MB
viewport-pan-sweep: DOM nodes -262
viewport-pan-sweep: event listeners -101
vue-large-graph-idle: avg frame time 17ms
vue-large-graph-idle: p95 frame time 17ms
vue-large-graph-idle: layout duration 0ms
vue-large-graph-idle: style recalc duration 0ms
vue-large-graph-idle: layout count 0
vue-large-graph-idle: style recalc count 0
vue-large-graph-idle: task duration 12268ms
vue-large-graph-idle: script duration 625ms
vue-large-graph-idle: TBT 0ms
vue-large-graph-idle: heap used 163.4 MB
vue-large-graph-idle: DOM nodes -9850
vue-large-graph-idle: event listeners -23953
vue-large-graph-pan: avg frame time 17ms
vue-large-graph-pan: p95 frame time 17ms
vue-large-graph-pan: layout duration 0ms
vue-large-graph-pan: style recalc duration 18ms
vue-large-graph-pan: layout count 0
vue-large-graph-pan: style recalc count 104
vue-large-graph-pan: task duration 16936ms
vue-large-graph-pan: script duration 1009ms
vue-large-graph-pan: TBT 5ms
vue-large-graph-pan: heap used 277.3 MB
vue-large-graph-pan: DOM nodes -9850
vue-large-graph-pan: event listeners -23981
workflow-execution: avg frame time 17ms
workflow-execution: p95 frame time 17ms
workflow-execution: layout duration 2ms
workflow-execution: style recalc duration 23ms
workflow-execution: layout count 5
workflow-execution: style recalc count 18
workflow-execution: task duration 126ms
workflow-execution: script duration 30ms
workflow-execution: TBT 0ms
workflow-execution: heap used 49.8 MB
workflow-execution: DOM nodes 157
workflow-execution: event listeners 71
Raw data
{
  "timestamp": "2026-05-02T08:58:15.438Z",
  "gitSha": "fa792aa3abb4374597449cd6e505d7bbffee3db6",
  "branch": "sno-agent",
  "measurements": [
    {
      "name": "canvas-idle",
      "durationMs": 2045.2660000000833,
      "styleRecalcs": 8,
      "styleRecalcDurationMs": 8.911,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 438.078,
      "heapDeltaBytes": 21480880,
      "heapUsedBytes": 67889464,
      "domNodes": 16,
      "jsHeapTotalBytes": 22544384,
      "scriptDurationMs": 23.408000000000005,
      "eventListeners": 10,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "canvas-idle",
      "durationMs": 2033.933000000161,
      "styleRecalcs": 8,
      "styleRecalcDurationMs": 7.1480000000000015,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 414.76,
      "heapDeltaBytes": 8250464,
      "heapUsedBytes": 54692544,
      "domNodes": -267,
      "jsHeapTotalBytes": 26288128,
      "scriptDurationMs": 16.104999999999997,
      "eventListeners": -127,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "canvas-idle",
      "durationMs": 2004.8860000001696,
      "styleRecalcs": 10,
      "styleRecalcDurationMs": 10.474999999999998,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 440.08,
      "heapDeltaBytes": 21284044,
      "heapUsedBytes": 67292580,
      "domNodes": 19,
      "jsHeapTotalBytes": 23068672,
      "scriptDurationMs": 24.911999999999995,
      "eventListeners": 10,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "canvas-mouse-sweep",
      "durationMs": 1864.5109999999931,
      "styleRecalcs": 75,
      "styleRecalcDurationMs": 43.419999999999995,
      "layouts": 12,
      "layoutDurationMs": 4.528,
      "taskDurationMs": 862.027,
      "heapDeltaBytes": 16811620,
      "heapUsedBytes": 62973728,
      "domNodes": 57,
      "jsHeapTotalBytes": 23592960,
      "scriptDurationMs": 139.67499999999998,
      "eventListeners": 8,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "canvas-mouse-sweep",
      "durationMs": 1965.7439999998587,
      "styleRecalcs": 79,
      "styleRecalcDurationMs": 38.794999999999995,
      "layouts": 12,
      "layoutDurationMs": 3.4079999999999995,
      "taskDurationMs": 911.501,
      "heapDeltaBytes": 16536300,
      "heapUsedBytes": 62914320,
      "domNodes": 62,
      "jsHeapTotalBytes": 22806528,
      "scriptDurationMs": 123.025,
      "eventListeners": 10,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "canvas-mouse-sweep",
      "durationMs": 2025.0550000000658,
      "styleRecalcs": 84,
      "styleRecalcDurationMs": 48.379000000000005,
      "layouts": 12,
      "layoutDurationMs": 3.331,
      "taskDurationMs": 1053.805,
      "heapDeltaBytes": 16905728,
      "heapUsedBytes": 62600092,
      "domNodes": 65,
      "jsHeapTotalBytes": 23330816,
      "scriptDurationMs": 145.831,
      "eventListeners": 10,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "canvas-zoom-sweep",
      "durationMs": 1741.329000000178,
      "styleRecalcs": 31,
      "styleRecalcDurationMs": 15.506,
      "layouts": 6,
      "layoutDurationMs": 0.5139999999999999,
      "taskDurationMs": 290.962,
      "heapDeltaBytes": 25395984,
      "heapUsedBytes": 71161160,
      "domNodes": 78,
      "jsHeapTotalBytes": 21495808,
      "scriptDurationMs": 24.255,
      "eventListeners": 23,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "canvas-zoom-sweep",
      "durationMs": 1753.8889999998446,
      "styleRecalcs": 31,
      "styleRecalcDurationMs": 19.818,
      "layouts": 6,
      "layoutDurationMs": 0.642,
      "taskDurationMs": 314.542,
      "heapDeltaBytes": 25228488,
      "heapUsedBytes": 71052832,
      "domNodes": 79,
      "jsHeapTotalBytes": 22020096,
      "scriptDurationMs": 25.106,
      "eventListeners": 23,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333335,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "canvas-zoom-sweep",
      "durationMs": 1738.0439999999453,
      "styleRecalcs": 32,
      "styleRecalcDurationMs": 17.801000000000005,
      "layouts": 6,
      "layoutDurationMs": 0.6619999999999998,
      "taskDurationMs": 343.619,
      "heapDeltaBytes": 22988404,
      "heapUsedBytes": 69412104,
      "domNodes": 78,
      "jsHeapTotalBytes": 34340864,
      "scriptDurationMs": 22.201,
      "eventListeners": 25,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "dom-widget-clipping",
      "durationMs": 542.6089999998567,
      "styleRecalcs": 12,
      "styleRecalcDurationMs": 7.880999999999999,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 318.88899999999995,
      "heapDeltaBytes": 7169008,
      "heapUsedBytes": 53179128,
      "domNodes": 20,
      "jsHeapTotalBytes": 12845056,
      "scriptDurationMs": 56.763000000000005,
      "eventListeners": 2,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66666666666665,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "dom-widget-clipping",
      "durationMs": 529.3249999999716,
      "styleRecalcs": 12,
      "styleRecalcDurationMs": 9.149,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 328.332,
      "heapDeltaBytes": 7062612,
      "heapUsedBytes": 53141548,
      "domNodes": 20,
      "jsHeapTotalBytes": 13107200,
      "scriptDurationMs": 62.80899999999999,
      "eventListeners": 2,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000273
    },
    {
      "name": "dom-widget-clipping",
      "durationMs": 601.1120000000574,
      "styleRecalcs": 11,
      "styleRecalcDurationMs": 7.929000000000002,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 361.42999999999995,
      "heapDeltaBytes": 7087092,
      "heapUsedBytes": 53134108,
      "domNodes": 18,
      "jsHeapTotalBytes": 12845056,
      "scriptDurationMs": 60.573,
      "eventListeners": 2,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.799999999999727
    },
    {
      "name": "large-graph-idle",
      "durationMs": 2026.0490000000573,
      "styleRecalcs": 10,
      "styleRecalcDurationMs": 9.213000000000003,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 582.504,
      "heapDeltaBytes": 32417188,
      "heapUsedBytes": 88350500,
      "domNodes": -260,
      "jsHeapTotalBytes": 38694912,
      "scriptDurationMs": 98.364,
      "eventListeners": -123,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.670000000000012,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "large-graph-idle",
      "durationMs": 2037.7140000000509,
      "styleRecalcs": 10,
      "styleRecalcDurationMs": 8.577000000000002,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 594.187,
      "heapDeltaBytes": 36627864,
      "heapUsedBytes": 92216504,
      "domNodes": -260,
      "jsHeapTotalBytes": 38170624,
      "scriptDurationMs": 101.75899999999999,
      "eventListeners": -123,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.670000000000012,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "large-graph-idle",
      "durationMs": 2047.461000000112,
      "styleRecalcs": 10,
      "styleRecalcDurationMs": 11.387,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 744.6689999999999,
      "heapDeltaBytes": 41541252,
      "heapUsedBytes": 97764576,
      "domNodes": -260,
      "jsHeapTotalBytes": 38432768,
      "scriptDurationMs": 117.27499999999999,
      "eventListeners": -123,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "large-graph-pan",
      "durationMs": 2169.016000000056,
      "styleRecalcs": 68,
      "styleRecalcDurationMs": 18.607000000000003,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 1081.92,
      "heapDeltaBytes": -8532804,
      "heapUsedBytes": 49220704,
      "domNodes": -264,
      "jsHeapTotalBytes": 22704128,
      "scriptDurationMs": 392.741,
      "eventListeners": -123,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "large-graph-pan",
      "durationMs": 2195.803999999953,
      "styleRecalcs": 69,
      "styleRecalcDurationMs": 18.347,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 1162.674,
      "heapDeltaBytes": 48825092,
      "heapUsedBytes": 107296804,
      "domNodes": -264,
      "jsHeapTotalBytes": 47288320,
      "scriptDurationMs": 416.86899999999997,
      "eventListeners": -123,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "large-graph-pan",
      "durationMs": 2143.004999999903,
      "styleRecalcs": 68,
      "styleRecalcDurationMs": 18.854000000000003,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 1194.679,
      "heapDeltaBytes": 8379788,
      "heapUsedBytes": 77635248,
      "domNodes": -264,
      "jsHeapTotalBytes": 13905920,
      "scriptDurationMs": 409.862,
      "eventListeners": -125,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.699999999999818
    },
    {
      "name": "large-graph-zoom",
      "durationMs": 3147.2650000000613,
      "styleRecalcs": 65,
      "styleRecalcDurationMs": 18.305000000000003,
      "layouts": 60,
      "layoutDurationMs": 7.548000000000001,
      "taskDurationMs": 1368.4879999999998,
      "heapDeltaBytes": 1433144,
      "heapUsedBytes": 60469532,
      "domNodes": -268,
      "jsHeapTotalBytes": 19034112,
      "scriptDurationMs": 503.66200000000003,
      "eventListeners": -121,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "large-graph-zoom",
      "durationMs": 3198.4449999999924,
      "styleRecalcs": 64,
      "styleRecalcDurationMs": 16.494,
      "layouts": 60,
      "layoutDurationMs": 7.066,
      "taskDurationMs": 1384.5770000000002,
      "heapDeltaBytes": 36169176,
      "heapUsedBytes": 96123452,
      "domNodes": -268,
      "jsHeapTotalBytes": 48394240,
      "scriptDurationMs": 500.169,
      "eventListeners": -121,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "large-graph-zoom",
      "durationMs": 3317.596999999978,
      "styleRecalcs": 65,
      "styleRecalcDurationMs": 25.769000000000002,
      "layouts": 60,
      "layoutDurationMs": 7.806,
      "taskDurationMs": 1546.16,
      "heapDeltaBytes": 49586968,
      "heapUsedBytes": 109247040,
      "domNodes": -268,
      "jsHeapTotalBytes": 50491392,
      "scriptDurationMs": 532.6329999999999,
      "eventListeners": -119,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.699999999999818
    },
    {
      "name": "minimap-idle",
      "durationMs": 2030.281000000059,
      "styleRecalcs": 9,
      "styleRecalcDurationMs": 8.001999999999999,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 494.597,
      "heapDeltaBytes": -4907200,
      "heapUsedBytes": 54041576,
      "domNodes": -242,
      "jsHeapTotalBytes": 15659008,
      "scriptDurationMs": 86.501,
      "eventListeners": -119,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "minimap-idle",
      "durationMs": 2024.0600000001905,
      "styleRecalcs": 8,
      "styleRecalcDurationMs": 7.289000000000001,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 517.9119999999999,
      "heapDeltaBytes": 3618516,
      "heapUsedBytes": 62415872,
      "domNodes": -265,
      "jsHeapTotalBytes": 16183296,
      "scriptDurationMs": 89.87,
      "eventListeners": -125,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "minimap-idle",
      "durationMs": 2021.5109999999186,
      "styleRecalcs": 8,
      "styleRecalcDurationMs": 8.452000000000002,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 561.898,
      "heapDeltaBytes": 3772132,
      "heapUsedBytes": 62743140,
      "domNodes": -266,
      "jsHeapTotalBytes": 16445440,
      "scriptDurationMs": 99.798,
      "eventListeners": -125,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "subgraph-dom-widget-clipping",
      "durationMs": 564.2460000001392,
      "styleRecalcs": 47,
      "styleRecalcDurationMs": 11.152,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 348.99800000000005,
      "heapDeltaBytes": 7437004,
      "heapUsedBytes": 53395100,
      "domNodes": 20,
      "jsHeapTotalBytes": 12582912,
      "scriptDurationMs": 123.756,
      "eventListeners": 8,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000273
    },
    {
      "name": "subgraph-dom-widget-clipping",
      "durationMs": 540.8890000001065,
      "styleRecalcs": 47,
      "styleRecalcDurationMs": 11.263,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 341.866,
      "heapDeltaBytes": 7089284,
      "heapUsedBytes": 53065076,
      "domNodes": 20,
      "jsHeapTotalBytes": 12845056,
      "scriptDurationMs": 120.72699999999999,
      "eventListeners": 8,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.669999999999998,
      "p95FrameDurationMs": 16.700000000000273
    },
    {
      "name": "subgraph-dom-widget-clipping",
      "durationMs": 544.2299999999705,
      "styleRecalcs": 46,
      "styleRecalcDurationMs": 11.23,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 355.421,
      "heapDeltaBytes": 7039156,
      "heapUsedBytes": 53112956,
      "domNodes": 18,
      "jsHeapTotalBytes": 13369344,
      "scriptDurationMs": 119.649,
      "eventListeners": 8,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "subgraph-idle",
      "durationMs": 1991.6740000001028,
      "styleRecalcs": 10,
      "styleRecalcDurationMs": 9.059000000000001,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 414.721,
      "heapDeltaBytes": -535452,
      "heapUsedBytes": 46037264,
      "domNodes": -260,
      "jsHeapTotalBytes": 27525120,
      "scriptDurationMs": 17.895999999999994,
      "eventListeners": -131,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.670000000000012,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "subgraph-idle",
      "durationMs": 2006.2809999999445,
      "styleRecalcs": 9,
      "styleRecalcDurationMs": 8.206,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 403.598,
      "heapDeltaBytes": 9938084,
      "heapUsedBytes": 56415152,
      "domNodes": -261,
      "jsHeapTotalBytes": 27525120,
      "scriptDurationMs": 14.415000000000001,
      "eventListeners": -127,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333335,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "subgraph-idle",
      "durationMs": 1996.6400000000704,
      "styleRecalcs": 10,
      "styleRecalcDurationMs": 10.152000000000001,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 424.01800000000003,
      "heapDeltaBytes": 20787696,
      "heapUsedBytes": 66951168,
      "domNodes": 20,
      "jsHeapTotalBytes": 23068672,
      "scriptDurationMs": 25.529000000000003,
      "eventListeners": 10,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "subgraph-mouse-sweep",
      "durationMs": 1992.8930000000946,
      "styleRecalcs": 84,
      "styleRecalcDurationMs": 45.553000000000004,
      "layouts": 16,
      "layoutDurationMs": 4.318,
      "taskDurationMs": 936.9159999999999,
      "heapDeltaBytes": 6814740,
      "heapUsedBytes": 52857128,
      "domNodes": -259,
      "jsHeapTotalBytes": 12320768,
      "scriptDurationMs": 98.558,
      "eventListeners": -121,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66999999999998,
      "p95FrameDurationMs": 16.800000000000182
    },
    {
      "name": "subgraph-mouse-sweep",
      "durationMs": 1758.9170000001104,
      "styleRecalcs": 76,
      "styleRecalcDurationMs": 36.624,
      "layouts": 16,
      "layoutDurationMs": 4.165,
      "taskDurationMs": 680.676,
      "heapDeltaBytes": 17605620,
      "heapUsedBytes": 63649416,
      "domNodes": -247,
      "jsHeapTotalBytes": 20185088,
      "scriptDurationMs": 90.66300000000001,
      "eventListeners": -121,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "subgraph-mouse-sweep",
      "durationMs": 2001.258000000007,
      "styleRecalcs": 86,
      "styleRecalcDurationMs": 51.26800000000001,
      "layouts": 16,
      "layoutDurationMs": 4.742,
      "taskDurationMs": 1011.1619999999999,
      "heapDeltaBytes": 1628804,
      "heapUsedBytes": 47651844,
      "domNodes": -262,
      "jsHeapTotalBytes": 13631488,
      "scriptDurationMs": 109.987,
      "eventListeners": -123,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "viewport-pan-sweep",
      "durationMs": 8138.157999999976,
      "styleRecalcs": 249,
      "styleRecalcDurationMs": 50.86500000000001,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 3747.9109999999996,
      "heapDeltaBytes": 22801164,
      "heapUsedBytes": 79656068,
      "domNodes": -263,
      "jsHeapTotalBytes": 21860352,
      "scriptDurationMs": 1311.1139999999998,
      "eventListeners": -99,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "viewport-pan-sweep",
      "durationMs": 8410.55799999981,
      "styleRecalcs": 250,
      "styleRecalcDurationMs": 51.05,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 4025.624,
      "heapDeltaBytes": 81909864,
      "heapUsedBytes": 139099432,
      "domNodes": -261,
      "jsHeapTotalBytes": 87134208,
      "scriptDurationMs": 1295.983,
      "eventListeners": -119,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "viewport-pan-sweep",
      "durationMs": 8161.632000000054,
      "styleRecalcs": 250,
      "styleRecalcDurationMs": 55.205,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 4209.33,
      "heapDeltaBytes": 24954500,
      "heapUsedBytes": 81251208,
      "domNodes": -262,
      "jsHeapTotalBytes": 21860352,
      "scriptDurationMs": 1438.2659999999998,
      "eventListeners": -101,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.66333333333332,
      "p95FrameDurationMs": 16.700000000000728
    },
    {
      "name": "vue-large-graph-idle",
      "durationMs": 14855.67900000001,
      "styleRecalcs": 0,
      "styleRecalcDurationMs": 0,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 14817.485,
      "heapDeltaBytes": 20062228,
      "heapUsedBytes": 273973436,
      "domNodes": -9848,
      "jsHeapTotalBytes": 29945856,
      "scriptDurationMs": 676.8969999999999,
      "eventListeners": -23983,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.670000000000073,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "vue-large-graph-idle",
      "durationMs": 12141.567000000123,
      "styleRecalcs": 0,
      "styleRecalcDurationMs": 0,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 12120.541000000003,
      "heapDeltaBytes": -57637860,
      "heapUsedBytes": 163513556,
      "domNodes": -9850,
      "jsHeapTotalBytes": 17625088,
      "scriptDurationMs": 589.846,
      "eventListeners": -23953,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 17.223333333333358,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "vue-large-graph-idle",
      "durationMs": 12278.479000000061,
      "styleRecalcs": 0,
      "styleRecalcDurationMs": 0,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 12267.765000000001,
      "heapDeltaBytes": -53076408,
      "heapUsedBytes": 171302436,
      "domNodes": -9850,
      "jsHeapTotalBytes": 28110848,
      "scriptDurationMs": 625.38,
      "eventListeners": -23951,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 17.223333333333358,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "vue-large-graph-pan",
      "durationMs": 14219.277000000147,
      "styleRecalcs": 68,
      "styleRecalcDurationMs": 15.654999999999974,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 14197.188,
      "heapDeltaBytes": -63342960,
      "heapUsedBytes": 160045132,
      "domNodes": -9850,
      "jsHeapTotalBytes": -15405056,
      "scriptDurationMs": 904.836,
      "eventListeners": -23983,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 17.223333333333358,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "vue-large-graph-pan",
      "durationMs": 16963.7630000002,
      "styleRecalcs": 105,
      "styleRecalcDurationMs": 18.531999999999993,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 16935.922,
      "heapDeltaBytes": 12561360,
      "heapUsedBytes": 290785948,
      "domNodes": -9850,
      "jsHeapTotalBytes": 15003648,
      "scriptDurationMs": 1018.036,
      "eventListeners": -23981,
      "totalBlockingTimeMs": 12,
      "frameDurationMs": 17.220000000000073,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "vue-large-graph-pan",
      "durationMs": 17033.804000000146,
      "styleRecalcs": 104,
      "styleRecalcDurationMs": 18.271000000000036,
      "layouts": 0,
      "layoutDurationMs": 0,
      "taskDurationMs": 17010.682000000004,
      "heapDeltaBytes": 45137024,
      "heapUsedBytes": 323715204,
      "domNodes": -9850,
      "jsHeapTotalBytes": 31956992,
      "scriptDurationMs": 1009.285,
      "eventListeners": -23981,
      "totalBlockingTimeMs": 5,
      "frameDurationMs": 17.779999999999927,
      "p95FrameDurationMs": 16.799999999999272
    },
    {
      "name": "workflow-execution",
      "durationMs": 130.66800000001422,
      "styleRecalcs": 12,
      "styleRecalcDurationMs": 21.320999999999998,
      "layouts": 4,
      "layoutDurationMs": 1.6340000000000001,
      "taskDurationMs": 101.651,
      "heapDeltaBytes": 3577268,
      "heapUsedBytes": 50916156,
      "domNodes": 150,
      "jsHeapTotalBytes": 0,
      "scriptDurationMs": 24.165000000000003,
      "eventListeners": 37,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000273
    },
    {
      "name": "workflow-execution",
      "durationMs": 462.8949999998895,
      "styleRecalcs": 21,
      "styleRecalcDurationMs": 30.875999999999998,
      "layouts": 6,
      "layoutDurationMs": 1.7519999999999998,
      "taskDurationMs": 144.93800000000002,
      "heapDeltaBytes": 5197816,
      "heapUsedBytes": 52210220,
      "domNodes": 158,
      "jsHeapTotalBytes": 262144,
      "scriptDurationMs": 31.991000000000007,
      "eventListeners": 71,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.666666666666668,
      "p95FrameDurationMs": 16.700000000000273
    },
    {
      "name": "workflow-execution",
      "durationMs": 463.2080000001224,
      "styleRecalcs": 18,
      "styleRecalcDurationMs": 23.366,
      "layouts": 5,
      "layoutDurationMs": 1.3079999999999998,
      "taskDurationMs": 126.23100000000001,
      "heapDeltaBytes": 5159652,
      "heapUsedBytes": 52929924,
      "domNodes": 157,
      "jsHeapTotalBytes": 0,
      "scriptDurationMs": 30.166,
      "eventListeners": 71,
      "totalBlockingTimeMs": 0,
      "frameDurationMs": 16.663333333333338,
      "p95FrameDurationMs": 16.700000000000273
    }
  ]
}

@github-actions

github-actions Bot commented Apr 23, 2026

Copy link
Copy Markdown

🔗 Website Preview

Website Preview: https://comfy-website-preview-pr-11547.vercel.app

This commit: https://website-frontend-ipzbeb3ls-comfyui.vercel.app

Last updated: 2026-04-26T00:21:42Z for aec9f93

🌐 Website E2E

Caution

Some tests failed.

Status ❌ Failed
Report View Report
Screenshot diffs 6
Other failures 0
  • Update website screenshots (6 screenshot diffs)

@socket-security

socket-security Bot commented Apr 24, 2026

Copy link
Copy Markdown

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report

@snomiao snomiao changed the title docs: client-side agent design notes [experimental][draft] in-browser agent for playing with ComfyUI Apr 25, 2026
snomiao added a commit that referenced this pull request Apr 26, 2026
Stress-test surfaced three new wrong-answer patterns the existing
verifiers didn't catch:

(a) Queue-state contradictions
    Agent claims 'image generation in progress' / 'queue started'
    after 'comfy queueprompt' returns ok, but the actual queue is
    empty because workflow validation rejected the prompt. Verifier
    runs queue-status + workflow-errors, fires contradiction if
    queue is empty or errors are present.

(b) Pre-refusal without trying
    Agent says 'X doesn\'t seem to be available' / 'requires Y
    which isn\'t here' as the first response, no tool call. The
    system prompt forbids this (ZERO-TOOL REFUSAL) but the model
    still does it when refusal feels polite. Verifier always fires
    a contradiction — forces an actual attempt before letting any
    'unavailable' claim through.

(c) Deferral / punt-to-user
    Agent ends with 'you can try queuing again' / 'please review
    the error overlay' / 'if everything seems connected'. Looks
    helpful but is just handing the work back. Verifier fires
    contradiction unless the response ends with a question mark
    (which signals genuine clarification request, not punt).

All three live in the same code-side ClaimVerifier registry as the
orphan/missing-models verifiers. Detected during the stress-test
run reported in PR #11547.
snomiao added a commit that referenced this pull request Apr 26, 2026
… frontend-only banner

When the user opens the agent for the first time without an API key
configured, they used to see an empty terminal and a vague 'no key'
error after typing. Now:

- Settings tab opens by default when apiKey is empty (showSettings
  initialized from session.apiKey.value).
- Settings panel leads with an azure 'Frontend-only agent' callout
  reassuring the user where their key goes (browser localStorage,
  sent only to the configured API endpoint).
- A yellow ⚠️ row appears in the callout when no key is set.
- New compact 3-field layout: API base URL / API key / Model in
  order, font-mono inputs, monospace placeholders.
- baseURL is a new field (Comfy.Agent.BaseURL localStorage key) so
  users can target OpenRouter, a local LLM proxy, or any other
  OpenAI-compatible gateway. Empty = OpenAI default.
- Reasoning effort + system-prompt append moved into a collapsed
  'Advanced' section to reduce visual noise.

The fall-back error message in send() also updated: when no key is
set, it points at the gear icon and reiterates the frontend-only
guarantee instead of an inscrutable VITE_OPENAI_API_KEY hint.

Default model remains gpt-5.4. Future: integration with Comfy Cloud
sign-in to auto-fill credentials when the user is already
authenticated for cloud API nodes (#11547 / out of scope this PR).
snomiao added a commit that referenced this pull request Apr 26, 2026
Capture the architectural decision that PR #11547's agent stays
frontend-only for now, and document the preferred shape for a
future local-agent bridge (opt-in publish bus, SPA stays
source-of-truth + execution sandbox, local agents propose actions
they don't execute).

Records the alternatives considered + rejected:
- Option A: ComfyUI backend as session broker — rejected for scope
  creep + privacy regression.
- Option B: browser extension or local sidecar daemon — rejected
  as default path for cross-platform packaging burden.
- Option C: opt-in publish bus, accepted as future direction.

Also records the related but distinct work item of reusing Comfy
Cloud credentials when the user is signed in for cloud nodes,
which interacts with the local-agent identity story.
snomiao added a commit that referenced this pull request Apr 26, 2026
Squashes 60 incremental agent commits into one cohesive change after
prior history accumulated 119 unrelated upstream commits during a
bad rebase. Working tree restored from sno-agent@aec9f9380; new deps
added freshly (no stale lockfile entries); changeTracker API call
sites updated for current sno-frontend-preview (captureCanvasState →
checkState).

What's in this commit:

UI: src/agent/ui/{AgentRoot,AgentFab,XtermPanel,AgentSettings,
useXtermReadline}.vue
- Floating ComfyAI button + draggable xterm panel toggled via 'c'
  keybind; auto-focuses the terminal helper textarea on open.
- xterm-driven readline with Tab completion, Shift+Enter newline,
  Ctrl-A/E/U/K/L/C, history navigation, multiline buffers.
- Liquid-glass theme using Comfy design tokens.
- Settings panel auto-opens when no API key set; compact 3-field
  layout (API base / API key / model) + collapsed advanced section.

Session loop: src/agent/llm/session.ts, composables/useAgentSession.ts
- Vercel AI SDK streamText with run_shell as the only tool.
- IndexedDB-persisted message history (300-message cap), replays
  on reopen with a 'previous session' divider.
- Programmatic guardrails: PROMISSORY_PATTERN auto-continue,
  silent-fail auto-continue, fragile-shell-idiom blocklist
  (Layer 1), definitive-claim verifier registry (Layer 2 — orphans,
  missing-models, queue-state, pre-refusal, punt-to-user).
- Configurable baseURL for OpenRouter / local LLM proxies; default
  gpt-5.4 via OpenAI.

Shell runtime: src/agent/shell/
- POSIX-ish parser (shell-quote based), AsyncIterable pipes,
  redirection, &&/||/; sequencing.
- VFS: in-memory /tmp + UserdataVFS-backed /workflows.
- Coreutils + Comfy.* command dispatch + run-js fallback.

Commands: src/agent/shell/commands/
- comfy/comfyNs: registered command discovery + namespace dispatch.
- workflow: save-as / new-workflow / rename-workflow / clear-workflow
  --force / set-subgraph-{desc,aliases} (modal-bypass equivalents).
- nodeOps: node-search, add-node (smart placement), connect (auto-
  layout on link), disconnect, remove-node, layout, align-nodes,
  distribute-nodes, select, get-widget, toggle-panel.
- graph/state/execution/sweep: introspection + queue + sweep helpers.
- templates / images / install / validate / see: template loading,
  output→input copy, Manager model install, Gemini canvas vision.

Tests: browser_tests/tests/agentTerminal.spec.ts, plus
src/agent/**/*.test.ts unit tests.

Docs: docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md
records the architectural choice + rejected alternatives.

i18n: minimal en additions in src/locales/en/main.json for
agent.{fab,input,panel,settings} namespaces.

Deps added: @ai-sdk/openai, ai, zod, shell-quote, idb-keyval,
@xterm/xterm, @xterm/addon-fit, es-toolkit, @types/shell-quote.

PR #11547 — experimental, draft, expect breaking changes.
Preview: https://pr-11547.comfy-ui.pages.dev/
@snomiao snomiao force-pushed the sno-agent branch 2 times, most recently from 90d3cd2 to 3dff19f Compare April 27, 2026 00:06
snomiao added a commit that referenced this pull request Apr 27, 2026
Squashes 60 incremental agent commits into one cohesive change after
prior history accumulated 119 unrelated upstream commits during a
bad rebase. Working tree restored from sno-agent@aec9f9380; new deps
added freshly (no stale lockfile entries); changeTracker API call
sites updated for current sno-frontend-preview (captureCanvasState →
checkState).

What's in this commit:

UI: src/agent/ui/{AgentRoot,AgentFab,XtermPanel,AgentSettings,
useXtermReadline}.vue
- Floating ComfyAI button + draggable xterm panel toggled via 'c'
  keybind; auto-focuses the terminal helper textarea on open.
- xterm-driven readline with Tab completion, Shift+Enter newline,
  Ctrl-A/E/U/K/L/C, history navigation, multiline buffers.
- Liquid-glass theme using Comfy design tokens.
- Settings panel auto-opens when no API key set; compact 3-field
  layout (API base / API key / model) + collapsed advanced section.

Session loop: src/agent/llm/session.ts, composables/useAgentSession.ts
- Vercel AI SDK streamText with run_shell as the only tool.
- IndexedDB-persisted message history (300-message cap), replays
  on reopen with a 'previous session' divider.
- Programmatic guardrails: PROMISSORY_PATTERN auto-continue,
  silent-fail auto-continue, fragile-shell-idiom blocklist
  (Layer 1), definitive-claim verifier registry (Layer 2 — orphans,
  missing-models, queue-state, pre-refusal, punt-to-user).
- Configurable baseURL for OpenRouter / local LLM proxies; default
  gpt-5.4 via OpenAI.

Shell runtime: src/agent/shell/
- POSIX-ish parser (shell-quote based), AsyncIterable pipes,
  redirection, &&/||/; sequencing.
- VFS: in-memory /tmp + UserdataVFS-backed /workflows.
- Coreutils + Comfy.* command dispatch + run-js fallback.

Commands: src/agent/shell/commands/
- comfy/comfyNs: registered command discovery + namespace dispatch.
- workflow: save-as / new-workflow / rename-workflow / clear-workflow
  --force / set-subgraph-{desc,aliases} (modal-bypass equivalents).
- nodeOps: node-search, add-node (smart placement), connect (auto-
  layout on link), disconnect, remove-node, layout, align-nodes,
  distribute-nodes, select, get-widget, toggle-panel.
- graph/state/execution/sweep: introspection + queue + sweep helpers.
- templates / images / install / validate / see: template loading,
  output→input copy, Manager model install, Gemini canvas vision.

Tests: browser_tests/tests/agentTerminal.spec.ts, plus
src/agent/**/*.test.ts unit tests.

Docs: docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md
records the architectural choice + rejected alternatives.

i18n: minimal en additions in src/locales/en/main.json for
agent.{fab,input,panel,settings} namespaces.

Deps added: @ai-sdk/openai, ai, zod, shell-quote, idb-keyval,
@xterm/xterm, @xterm/addon-fit, es-toolkit, @types/shell-quote.

PR #11547 — experimental, draft, expect breaking changes.
Preview: https://pr-11547.comfy-ui.pages.dev/
@snomiao snomiao force-pushed the sno-frontend-preview branch from e2492a4 to d840020 Compare April 27, 2026 00:07
@snomiao snomiao requested a review from Copilot April 27, 2026 00:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an experimental, frontend-only in-browser agent (“ComfyAI”) for driving ComfyUI via natural language, plus supporting infrastructure for PR previews and connecting the SPA to a configurable backend.

Changes:

  • Adds a new src/agent/ subsystem: UI (FAB + foldable xterm panel), session management, shell runtime + commands, VFS, and logging.
  • Adds a /connect route and backend reachability probing, plus support for overriding backend origin via localStorage.
  • Enhances PR preview builds with CI metadata globals, PR preview badges, and Cloudflare Pages preview deployment workflow/script.

Reviewed changes

Copilot reviewed 75 out of 79 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
vite.config.mts Adds dev plugin + CI globals
src/vite-env.d.ts Declares new CI globals
src/views/templates/BaseViewTemplate.vue Layout tweak for centering
src/views/ConnectionPanelView.test.ts Adds connection panel tests
src/scripts/defaultGraph.ts Adds model metadata to default graph
src/scripts/api.ts Supports remote backend origin override
src/schemas/apiSchema.ts Adds comfy_api_base to schema
src/router.ts Adds /connect + reachability redirect + base path fix
src/platform/connectionPanel/resolveBackendCloudBase.ts Resolves cloud base from system info
src/platform/connectionPanel/backendReachable.ts Backend probing helper
src/platform/connectionPanel/backendReachable.test.ts Tests backend probing
src/extensions/core/prPreviewBadges.ts PR preview UI badges
src/extensions/core/index.ts Conditionally loads PR preview badges
src/config/comfyApi.ts Maps API base → platform base
src/components/connection/CopyCodeBlock.vue Copyable code snippet component
src/agent/ui/AgentSettings.vue Agent settings UI
src/agent/ui/AgentRoot.vue Teleported root + keybinding/command registration
src/agent/ui/AgentFab.vue Draggable FAB + drop ingest
src/agent/stores/agentStore.ts Agent state + persistence
src/agent/stores/agentStore.test.ts Agent store tests
src/agent/shell/vfs/userdata.ts Userdata-backed VFS
src/agent/shell/vfs/userdata.test.ts Userdata VFS tests
src/agent/shell/vfs/mount.ts Mountable VFS router
src/agent/shell/vfs/mount.test.ts Mounted VFS tests
src/agent/shell/vfs/memory.ts In-memory VFS
src/agent/shell/vfs/memory.test.ts Memory VFS tests
src/agent/shell/types.ts Shell AST + interfaces
src/agent/shell/runtime.ts Shell executor + redirect/pipe/logic
src/agent/shell/runtime.test.ts Shell runtime tests
src/agent/shell/parser.ts Shell parser (subset)
src/agent/shell/parser.test.ts Shell parser tests
src/agent/shell/integration.test.ts End-to-end shell integration tests
src/agent/shell/commands/workflow.ts Workflow manipulation commands
src/agent/shell/commands/validate.ts Gemini validation command
src/agent/shell/commands/templates.ts Template list/load commands
src/agent/shell/commands/sweep.ts Widget sweep runner
src/agent/shell/commands/state.ts UI state inspection commands
src/agent/shell/commands/state.test.ts State command tests
src/agent/shell/commands/see.ts Canvas capture → Gemini analysis
src/agent/shell/commands/registrySearch.ts Registry pack/node search commands
src/agent/shell/commands/images.ts Image copy + latest output name
src/agent/shell/commands/graph.ts Graph introspection + set-widget
src/agent/shell/commands/graph.test.ts Graph command tests
src/agent/shell/commands/execution.ts Queue/history/wait helpers
src/agent/shell/commands/execution.test.ts Execution command tests
src/agent/shell/commands/coreutils.ts Minimal coreutils-style commands
src/agent/shell/commands/coreutils.test.ts Coreutils tests
src/agent/shell/commands/comfyNs.ts comfy namespace dispatcher
src/agent/shell/commands/comfy.ts cmd + cmd-list commands
src/agent/shell/commands/codesearch.ts Remote code search command
src/agent/shell/commands/browser.ts run-js + describe commands
src/agent/services/logger.ts Browser-side JSONL logger
src/agent/composables/useImageNodeDrop.ts Drop image → LoadImage node
src/agent/composables/useAssetIngest.ts Clipboard/drop asset ingest + upload
src/agent/composables/useAssetIngest.test.ts Asset ingest tests
src/agent/composables/useAgentSession.ts Session + streaming + tool plumbing
src/App.vue Mounts AgentRoot globally
scripts/cicd/pr-preview-deploy-and-comment.sh Deploy preview + PR comment automation
pnpm-workspace.yaml Adds agent-related deps to catalog
package.json Adds agent-related dependencies
global.d.ts Declares new CI globals
eslint.config.ts Registers new globals for lint
docs/backend-cloud-api-base-feature-flag.md Documents backend flag detection approach
docs/adr/README.md Adds ADR index entry
docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md New ADR for agent architecture
build/plugins/index.ts Exports new Vite plugin
build/plugins/agentLog.ts Dev-only /__agent-log endpoint
browser_tests/tests/agentTerminal.spec.ts Playwright E2E coverage for agent terminal
browser_tests/assets/default.json Updates default workflow fixture
.gitignore Ignores additional local artifacts
.github/workflows/ci-deploy-preview.yaml Adds Cloudflare Pages PR preview workflow
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Comment on lines +10 to +22
function resolveProbeBase(): string {
const stored = localStorage.getItem(BACKEND_URL_KEY)
if (stored) {
try {
// Only treat the stored value as a backend override when it's a
// well-formed absolute URL — otherwise fall through to same-origin.
const url = new URL(stored)
return url.origin + url.pathname.replace(/\/+$/, '')
} catch {
// Ignore malformed entries; same-origin probe is safer than a
// relative URL that misses the router's subpath base.
}
}

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveProbeBase() reads localStorage before checking typeof window, which will throw in non-browser contexts (SSR/unit tests without localStorage). Consider guarding at the top of the function (or in isBackendReachable) with if (typeof window === 'undefined' || !('localStorage' in window)) return '' before accessing localStorage.

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d4e5317: moved the typeof-window guard to the top of resolveProbeBase(), before the localStorage.getItem() call, and removed the now-redundant check further down.

snomiao added a commit that referenced this pull request Apr 28, 2026
Squashes 60 incremental agent commits into one cohesive change after
prior history accumulated 119 unrelated upstream commits during a
bad rebase. Working tree restored from sno-agent@aec9f9380; new deps
added freshly (no stale lockfile entries); changeTracker API call
sites updated for current sno-frontend-preview (captureCanvasState →
checkState).

What's in this commit:

UI: src/agent/ui/{AgentRoot,AgentFab,XtermPanel,AgentSettings,
useXtermReadline}.vue
- Floating ComfyAI button + draggable xterm panel toggled via 'c'
  keybind; auto-focuses the terminal helper textarea on open.
- xterm-driven readline with Tab completion, Shift+Enter newline,
  Ctrl-A/E/U/K/L/C, history navigation, multiline buffers.
- Liquid-glass theme using Comfy design tokens.
- Settings panel auto-opens when no API key set; compact 3-field
  layout (API base / API key / model) + collapsed advanced section.

Session loop: src/agent/llm/session.ts, composables/useAgentSession.ts
- Vercel AI SDK streamText with run_shell as the only tool.
- IndexedDB-persisted message history (300-message cap), replays
  on reopen with a 'previous session' divider.
- Programmatic guardrails: PROMISSORY_PATTERN auto-continue,
  silent-fail auto-continue, fragile-shell-idiom blocklist
  (Layer 1), definitive-claim verifier registry (Layer 2 — orphans,
  missing-models, queue-state, pre-refusal, punt-to-user).
- Configurable baseURL for OpenRouter / local LLM proxies; default
  gpt-5.4 via OpenAI.

Shell runtime: src/agent/shell/
- POSIX-ish parser (shell-quote based), AsyncIterable pipes,
  redirection, &&/||/; sequencing.
- VFS: in-memory /tmp + UserdataVFS-backed /workflows.
- Coreutils + Comfy.* command dispatch + run-js fallback.

Commands: src/agent/shell/commands/
- comfy/comfyNs: registered command discovery + namespace dispatch.
- workflow: save-as / new-workflow / rename-workflow / clear-workflow
  --force / set-subgraph-{desc,aliases} (modal-bypass equivalents).
- nodeOps: node-search, add-node (smart placement), connect (auto-
  layout on link), disconnect, remove-node, layout, align-nodes,
  distribute-nodes, select, get-widget, toggle-panel.
- graph/state/execution/sweep: introspection + queue + sweep helpers.
- templates / images / install / validate / see: template loading,
  output→input copy, Manager model install, Gemini canvas vision.

Tests: browser_tests/tests/agentTerminal.spec.ts, plus
src/agent/**/*.test.ts unit tests.

Docs: docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md
records the architectural choice + rejected alternatives.

i18n: minimal en additions in src/locales/en/main.json for
agent.{fab,input,panel,settings} namespaces.

Deps added: @ai-sdk/openai, ai, zod, shell-quote, idb-keyval,
@xterm/xterm, @xterm/addon-fit, es-toolkit, @types/shell-quote.

PR #11547 — experimental, draft, expect breaking changes.
Preview: https://pr-11547.comfy-ui.pages.dev/
@socket-security

socket-security Bot commented Apr 28, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​ai-sdk/​openai@​3.0.53731008898100
Updated@​astrojs/​check@​0.9.8 ⏵ 0.9.91001008093100
Added@​types/​shell-quote@​1.7.51001008780100
Addedai@​6.0.1689210010099100

View full report

snomiao added a commit that referenced this pull request Apr 28, 2026
Squashes 60 incremental agent commits into one cohesive change after
prior history accumulated 119 unrelated upstream commits during a
bad rebase. Working tree restored from sno-agent@aec9f9380; new deps
added freshly (no stale lockfile entries); changeTracker API call
sites updated for current sno-frontend-preview (captureCanvasState →
checkState).

What's in this commit:

UI: src/agent/ui/{AgentRoot,AgentFab,XtermPanel,AgentSettings,
useXtermReadline}.vue
- Floating ComfyAI button + draggable xterm panel toggled via 'c'
  keybind; auto-focuses the terminal helper textarea on open.
- xterm-driven readline with Tab completion, Shift+Enter newline,
  Ctrl-A/E/U/K/L/C, history navigation, multiline buffers.
- Liquid-glass theme using Comfy design tokens.
- Settings panel auto-opens when no API key set; compact 3-field
  layout (API base / API key / model) + collapsed advanced section.

Session loop: src/agent/llm/session.ts, composables/useAgentSession.ts
- Vercel AI SDK streamText with run_shell as the only tool.
- IndexedDB-persisted message history (300-message cap), replays
  on reopen with a 'previous session' divider.
- Programmatic guardrails: PROMISSORY_PATTERN auto-continue,
  silent-fail auto-continue, fragile-shell-idiom blocklist
  (Layer 1), definitive-claim verifier registry (Layer 2 — orphans,
  missing-models, queue-state, pre-refusal, punt-to-user).
- Configurable baseURL for OpenRouter / local LLM proxies; default
  gpt-5.4 via OpenAI.

Shell runtime: src/agent/shell/
- POSIX-ish parser (shell-quote based), AsyncIterable pipes,
  redirection, &&/||/; sequencing.
- VFS: in-memory /tmp + UserdataVFS-backed /workflows.
- Coreutils + Comfy.* command dispatch + run-js fallback.

Commands: src/agent/shell/commands/
- comfy/comfyNs: registered command discovery + namespace dispatch.
- workflow: save-as / new-workflow / rename-workflow / clear-workflow
  --force / set-subgraph-{desc,aliases} (modal-bypass equivalents).
- nodeOps: node-search, add-node (smart placement), connect (auto-
  layout on link), disconnect, remove-node, layout, align-nodes,
  distribute-nodes, select, get-widget, toggle-panel.
- graph/state/execution/sweep: introspection + queue + sweep helpers.
- templates / images / install / validate / see: template loading,
  output→input copy, Manager model install, Gemini canvas vision.

Tests: browser_tests/tests/agentTerminal.spec.ts, plus
src/agent/**/*.test.ts unit tests.

Docs: docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md
records the architectural choice + rejected alternatives.

i18n: minimal en additions in src/locales/en/main.json for
agent.{fab,input,panel,settings} namespaces.

Deps added: @ai-sdk/openai, ai, zod, shell-quote, idb-keyval,
@xterm/xterm, @xterm/addon-fit, es-toolkit, @types/shell-quote.

PR #11547 — experimental, draft, expect breaking changes.
Preview: https://pr-11547.comfy-ui.pages.dev/
snomiao added 2 commits May 2, 2026 17:39
Squashes 60 incremental agent commits into one cohesive change after
prior history accumulated 119 unrelated upstream commits during a
bad rebase. Working tree restored from sno-agent@aec9f9380; new deps
added freshly (no stale lockfile entries); changeTracker API call
sites updated for current sno-frontend-preview (captureCanvasState →
checkState).

What's in this commit:

UI: src/agent/ui/{AgentRoot,AgentFab,XtermPanel,AgentSettings,
useXtermReadline}.vue
- Floating ComfyAI button + draggable xterm panel toggled via 'c'
  keybind; auto-focuses the terminal helper textarea on open.
- xterm-driven readline with Tab completion, Shift+Enter newline,
  Ctrl-A/E/U/K/L/C, history navigation, multiline buffers.
- Liquid-glass theme using Comfy design tokens.
- Settings panel auto-opens when no API key set; compact 3-field
  layout (API base / API key / model) + collapsed advanced section.

Session loop: src/agent/llm/session.ts, composables/useAgentSession.ts
- Vercel AI SDK streamText with run_shell as the only tool.
- IndexedDB-persisted message history (300-message cap), replays
  on reopen with a 'previous session' divider.
- Programmatic guardrails: PROMISSORY_PATTERN auto-continue,
  silent-fail auto-continue, fragile-shell-idiom blocklist
  (Layer 1), definitive-claim verifier registry (Layer 2 — orphans,
  missing-models, queue-state, pre-refusal, punt-to-user).
- Configurable baseURL for OpenRouter / local LLM proxies; default
  gpt-5.4 via OpenAI.

Shell runtime: src/agent/shell/
- POSIX-ish parser (shell-quote based), AsyncIterable pipes,
  redirection, &&/||/; sequencing.
- VFS: in-memory /tmp + UserdataVFS-backed /workflows.
- Coreutils + Comfy.* command dispatch + run-js fallback.

Commands: src/agent/shell/commands/
- comfy/comfyNs: registered command discovery + namespace dispatch.
- workflow: save-as / new-workflow / rename-workflow / clear-workflow
  --force / set-subgraph-{desc,aliases} (modal-bypass equivalents).
- nodeOps: node-search, add-node (smart placement), connect (auto-
  layout on link), disconnect, remove-node, layout, align-nodes,
  distribute-nodes, select, get-widget, toggle-panel.
- graph/state/execution/sweep: introspection + queue + sweep helpers.
- templates / images / install / validate / see: template loading,
  output→input copy, Manager model install, Gemini canvas vision.

Tests: browser_tests/tests/agentTerminal.spec.ts, plus
src/agent/**/*.test.ts unit tests.

Docs: docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md
records the architectural choice + rejected alternatives.

i18n: minimal en additions in src/locales/en/main.json for
agent.{fab,input,panel,settings} namespaces.

Deps added: @ai-sdk/openai, ai, zod, shell-quote, idb-keyval,
@xterm/xterm, @xterm/addon-fit, es-toolkit, @types/shell-quote.

PR #11547 — experimental, draft, expect breaking changes.
Preview: https://pr-11547.comfy-ui.pages.dev/
Vite dev plugin /__agent-log now writes one file per browser session
(tmp/agent-logs/<date>-<sessionId>.jsonl) instead of mixing all sessions
into a single daily log. The browser logger already attaches sessionId
to every entry; this just routes by it on the server side.

Restores build/plugins/agentLog.ts that was lost in an earlier squash.
snomiao and others added 8 commits May 2, 2026 17:39
Layer 1.5 guardrail: any script touching mutating commands (add-node,
connect, set-widget, clear-workflow, save-as, …) is rejected by the
shell runtime until at least one canvas-observing read (graph, node-
list, active-workflow, …) has run in the same turn. Stops the agent
from blindly overwriting an existing workflow it never inspected.

Auto-continue tweaks:
- PROMISSORY_PATTERN extended to catch stop-early phrases ("here's a
  plan", "next, I'll", "we can also add", "to build/finish/complete")
  that surfaced as premature exits in real sessions.
- MAX_AUTOCONTINUE 3 → 6 so multi-step builds (load → encode → sample →
  decode → save) finish without prematurely punting back to the user.
New shell commands so the agent can find custom nodes the user has not
yet installed, instead of giving up when local node-search is empty:

- node-search-registry <q>  fuzzy node-class search across all
                            published packs (Comfy Registry)
- pack-search <q>           pack-name/description search
- pack-info <pack_id>       list every node-class a pack provides
- comfy-codesearch <q>      full-text source search across the WHOLE
                            public ComfyUI community via cs.comfy.org
                            (catches unregistered repos)

Discovery escalation chain in the system prompt is now:
  node-search → node-search-registry → pack-search/info → comfy-codesearch
The agent must walk down before claiming a node is missing.
Replaces the xterm.js scrollback with a Vue-native terminal that can
fold tool-call output. Tool calls collapse to a one-line summary by
default — "$ <cmd>   N lines, exit M" with green/red exit-code
colour — and Ctrl+O (or ⌘+O) toggles fold/unfold across every block at
once. Click the block header to toggle a single tool call.

Why this trade-off:
- xterm.js is a character grid; folding requires a parallel logical
  model and a full re-render on every toggle. Vue components track
  semantic blocks natively, so fold state is just reactive data.
- Side benefits: per-block text selection/copy, native theming via
  Tailwind tokens, no ANSI cursor math, no @xterm/* deps in the agent
  bundle, easier to extend (per-tool icons, inline images, links).

Implementation notes:
- store.AgentMessage gains an optional 'tool' meta field carrying
  script/stdout/stderr/exitCode so the renderer doesn't have to parse
  the synthesized text summary back into structure.
- Global keydown listener with capture:true preempts the browser's
  default Open-File dialog before Ctrl+O reaches it.
- Drops src/agent/ui/XtermPanel.vue, useXtermReadline.ts, and
  shell/useCompletion.ts. The xterm package is still used elsewhere
  in the codebase (logs/CommandTerminal), so no dep removal.
Previously the textarea was a separate footer with its own border/bg,
breaking the terminal illusion. Move the prompt+input INTO the
scrollable container so it visually flows as the next line of
scrollback — same font, same color scheme as the user-message blocks,
no border, no background.

- Pending-asset chips now render inline just above the prompt within
  the same scroll container (drops the heavy footer chrome).
- Click any blank space in the scrollback focuses the input — like a
  real terminal where typing always lands at the prompt regardless of
  scroll position. Clicks on actual messages are left alone so text
  selection still works.
- The prompt scrolls with content; auto-scroll-to-bottom keeps it in
  view when new messages arrive.
- agentStore: useIDBKeyval is async; the previous code seeded `messages`
  with an empty snapshot then enabled the write-back watcher immediately,
  so any send before the IDB read settled would clobber stored history
  with []. Gate the watcher on `persisted.isFinished` and merge the
  hydrated entries when they arrive.
- FoldablePanel: dropping `/` from the shell-operator regex. The
  attachment flow prefills the composer with paths like /input/foo.png,
  and the previous regex routed those to direct exec instead of the LLM.
- coreutils grep: drain stdin eagerly so the no-match case can return
  exit code 1. The previous generator-based version always returned 0,
  breaking `grep ... && ...` and `grep ... || ...` decision flows the
  agent relies on.
- registrySearch: stop advertising a non-existent `install-pack`
  command. Direct the user to ComfyUI-Manager UI instead until a real
  install-pack shell command exists.
- agentTerminal.spec: rewrite for the textarea-based panel; the old
  `.xterm-rows` / `.xterm-helper-textarea` locators no longer resolve.
  Drops the Tab-completion test (useCompletion was deleted) and adds a
  Ctrl+O fold/unfold smoke test.
Add links to OpenAI and OpenRouter key pages below the API key input so
users know where to obtain a key and how to configure the base URL for
non-OpenAI providers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- useLocalBridge: WS singleton connects to ws://127.0.0.1:7437/spa on
  AgentRoot mount, relays send/eval/abort/ping from CLI to SPA runtime
- useBridgeStatus: read-only bridge state (connected, activePairCode)
  readable from any component without lifecycle side-effects
- AgentSettings: 'Local agent bridge' section shows connection dot +
  status text; 'Generate pair code' button sends pair-request to daemon
  and shows the comfy-ai pair ... command to run in terminal; clears
  automatically when CLI claims the code

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ding

- Inject CSS to hide agent UI in all E2E screenshot tests; tag agent
  terminal tests with @agent to opt out of the hide (fixes 170+ CI
  screenshot failures caused by the FAB overlapping #graph-canvas)
- Guard localStorage access in resolveProbeBase() behind a typeof-window
  check so it doesn't throw in SSR/unit-test contexts (Copilot comment)
- Rebase onto sno-frontend-preview (3 upstream commits: API key test
  button, connection panel refactor, pr-badge i18n fix)
- Implement comfy-ai tail: useLocalBridge now watches agentStore.messages
  and agentStore.isStreaming, emitting SpaEvent frames to the daemon so
  `comfy-ai tail` streams live session activity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants