Merge pull request #48 from CopilotKit/feat/deep-agents

Upgrade to LangChain deep agents

Author: GeneralJerel

README.md

@@ -1,6 +1,6 @@
 # Open Generative UI
 
-An open-source showcase for building rich, interactive AI-generated UI with [CopilotKit](https://copilotkit.ai) and [LangGraph](https://langchain-ai.github.io/langgraph/). Ask the agent to visualize algorithms, create 3D animations, render charts, or generate interactive diagrams — all rendered as live HTML/SVG inside a sandboxed iframe.
+An open-source showcase for building rich, interactive AI-generated UI with [CopilotKit](https://copilotkit.ai) and [LangChain Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview). Ask the agent to visualize algorithms, create 3D animations, render charts, or generate interactive diagrams — all rendered as live HTML/SVG inside a sandboxed iframe.
 
 https://github.com/user-attachments/assets/ed28c734-e54e-4412-873f-4801da544a7f
 
@@ -104,14 +104,27 @@ Turborepo monorepo with three packages:
 ```
 apps/
 ├── app/       Next.js 16 frontend (CopilotKit v2, React 19, Tailwind 4)
-├── agent/     LangGraph Python agent (GPT-5.4, CopilotKit middleware)
+├── agent/     Deep Agent (deepagents + CopilotKit middleware, skills-based)
 └── mcp/       Standalone MCP server (design system + skills + document assembler)
 ```
 
+### Deep Agent + Skills
+
+The agent backend uses [LangChain Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview) (`create_deep_agent`) with a skills-based architecture. Instead of injecting all visualization instructions into the system prompt, skills are defined as `SKILL.md` files in `apps/agent/skills/` and loaded on-demand via progressive disclosure:
+
+```
+apps/agent/skills/
+├── advanced-visualization/SKILL.md   # UI mockups, dashboards, Chart.js, generative art
+├── master-playbook/SKILL.md          # Response philosophy, decision trees, narration patterns
+└── svg-diagrams/SKILL.md             # SVG generation rules, component patterns, diagram types
+```
+
+Deep agents also provide built-in planning (`write_todos`), filesystem tools, and sub-agent support.
+
 ### How It Works
 
 1. **User sends a prompt** via the CopilotKit chat UI
-2. **Agent decides** whether to respond with text, call a tool, or render a visual component
+2. **Deep agent decides** whether to respond with text, call a tool, or render a visual component — consulting relevant skills as needed
 3. **`widgetRenderer`** — a frontend `useComponent` hook — receives the agent's HTML and renders it in a sandboxed iframe
 4. **Skeleton loading** shows while the iframe loads, then content fades in smoothly
 5. **ResizeObserver** inside the iframe reports content height back to the parent for seamless auto-sizing
@@ -154,7 +167,7 @@ apps/
 
 ## Tech Stack
 
-Next.js 16, React 19, Tailwind CSS 4, LangGraph, CopilotKit v2, Turborepo, Recharts
+Next.js 16, React 19, Tailwind CSS 4, LangChain Deep Agents, LangGraph, CopilotKit v2, Turborepo, Recharts
 
 ## License
 

apps/agent/main.py

@@ -4,26 +4,24 @@
 """
 
 import os
+from pathlib import Path
 
 from copilotkit import CopilotKitMiddleware
-from langchain.agents import create_agent
+from deepagents import create_deep_agent
 from langchain_openai import ChatOpenAI
 
 from src.query import query_data
 from src.todos import AgentState, todo_tools
 from src.form import generate_form
 from src.templates import template_tools
-from skills import load_all_skills
 
-# Load all visualization skills
-_skills_text = load_all_skills()
-
-agent = create_agent(
+agent = create_deep_agent(
     model=ChatOpenAI(model=os.environ.get("LLM_MODEL", "gpt-5.4-2026-03-05")),
     tools=[query_data, *todo_tools, generate_form, *template_tools],
     middleware=[CopilotKitMiddleware()],
-    state_schema=AgentState,
-    system_prompt=f"""
+    context_schema=AgentState,
+    skills=[str(Path(__file__).parent / "skills")],
+    system_prompt="""
         You are a helpful assistant that helps users understand CopilotKit and LangGraph used together.
 
         Be brief in your explanations of CopilotKit and LangGraph, 1 to 2 sentences.
@@ -47,10 +45,6 @@
         - Pre-styled form elements (buttons, inputs, sliders look native automatically)
         - Pre-built SVG CSS classes for color ramps (.c-purple, .c-teal, .c-blue, etc.)
 
-        Follow the skills below for how to produce high-quality visuals:
-
-        {_skills_text}
-
         ## UI Templates
 
         Users can save generated UIs as reusable templates and apply them later.

apps/agent/pyproject.toml

@@ -16,4 +16,5 @@ dependencies = [
     "copilotkit>=0.1.77",
     "langgraph-api>=0.7.16",
     "langchain-mcp-adapters>=0.2.1",
+    "deepagents>=0.1.0",
 ]

apps/agent/skills/__init__.py

@@ -1,3 +1,9 @@
+---
+name: "Advanced Visualization Techniques"
+description: "UI mockups, dashboards, advanced interactivity, generative art, simulations, math visualizations, and design system rules for producing rich HTML widget output."
+allowed-tools: []
+---
+
 # Agent Visualization Skills — Volume 2: Advanced Techniques
 
 Prerequisite: Volume 1 (SVG diagrams, basic interactive widgets, Chart.js, Mermaid).
@@ -427,7 +433,7 @@ hardcoded hex values.
 ```
 - Height goes on the wrapper div ONLY, never on canvas.
 - Always set `responsive: true, maintainAspectRatio: false`.
-- For horizontal bar charts: height = (bars × 40) + 80 pixels.
+- For horizontal bar charts: height = (bars x 40) + 80 pixels.
 
 ### Custom Legend (Always Use This)
 Disable Chart.js default legend and build HTML:
@@ -450,7 +456,7 @@ plugins: { legend: { display: false } }
 ```
 
 ### Dashboard Layout
-Metric cards on top → chart below → sendPrompt for drill-down:
+Metric cards on top -> chart below -> sendPrompt for drill-down:
 ```html
 <!-- Metric cards grid -->
 <div style="display: grid;

apps/agent/skills/agent-skills-vol2.txt …t/skills/advanced-visualization/SKILL.mdapps/agent/skills/agent-skills-vol2.txt renamed to apps/agent/skills/advanced-visualization/SKILL.md apps/agent/skills/agent-skills-vol2.txt renamed to apps/agent/skills/advanced-visualization/SKILL.md

@@ -1,3 +1,9 @@
+---
+name: "Master Agent Playbook"
+description: "Philosophy, decision-making framework, and technical skills for delivering visual, interactive, and educational AI responses."
+allowed-tools: []
+---
+
 # Master Agent Playbook: Making AI Responses Extraordinary
 
 This playbook teaches an AI coding agent how to go beyond plain text and deliver
@@ -22,20 +28,20 @@ The principle: **Show, don't just tell.** Before writing any response, ask:
 
 ```
 User asks a question
-  │
-  ├─ Is it a quick factual answer? → Answer in 1-2 sentences.
-  │
-  ├─ Is it conceptual / "how does X work"?
-  │   ├─ Is it spatial or visual? → SVG illustrative diagram
-  │   ├─ Is it a process/flow? → SVG flowchart or HTML stepper
-  │   ├─ Is it data-driven? → Interactive chart (Chart.js / Recharts)
-  │   └─ Is it abstract but explorable? → Interactive HTML widget with controls
-  │
-  ├─ Is it "build me X"? → Working code artifact, fully functional
-  │
-  ├─ Is it a comparison? → Side-by-side table or comparative visual
-  │
-  └─ Is it emotional/personal? → Warm text response. No visuals needed.
+  |
+  +- Is it a quick factual answer? -> Answer in 1-2 sentences.
+  |
+  +- Is it conceptual / "how does X work"?
+  |   +- Is it spatial or visual? -> SVG illustrative diagram
+  |   +- Is it a process/flow? -> SVG flowchart or HTML stepper
+  |   +- Is it data-driven? -> Interactive chart (Chart.js / Recharts)
+  |   +- Is it abstract but explorable? -> Interactive HTML widget with controls
+  |
+  +- Is it "build me X"? -> Working code artifact, fully functional
+  |
+  +- Is it a comparison? -> Side-by-side table or comparative visual
+  |
+  +- Is it emotional/personal? -> Warm text response. No visuals needed.
 ```
 
 ### The 3-Layer Response Pattern

…s/agent/skills/master-agent-playbook.txt …ps/agent/skills/master-playbook/SKILL.mdapps/agent/skills/master-agent-playbook.txt renamed to apps/agent/skills/master-playbook/SKILL.md apps/agent/skills/master-agent-playbook.txt renamed to apps/agent/skills/master-playbook/SKILL.md

@@ -1,3 +1,9 @@
+---
+name: "SVG Diagram Generation"
+description: "Generating rich inline SVG diagrams to visually explain systems, processes, architectures, and abstract concepts."
+allowed-tools: []
+---
+
 # SVG Diagram Generation Skill
 
 You can generate rich, inline SVG diagrams to visually explain concepts. Use this skill whenever a visual would help the user understand a system, process, architecture, or mechanism better than text alone.
@@ -46,9 +52,9 @@ Always use this template:
 - **Sentence case always**. Never Title Case or ALL CAPS.
 
 ### Text Width Estimation
-At 14px, each character ≈ 8px wide. At 12px, each character ≈ 7px wide.
-- "Load Balancer" (13 chars) at 14px ≈ 104px → needs rect ≈ 140px wide (with padding).
-- Always compute: `rect_width = max(title_chars × 8, subtitle_chars × 7) + 48px padding`.
+At 14px, each character ~ 8px wide. At 12px, each character ~ 7px wide.
+- "Load Balancer" (13 chars) at 14px ~ 104px -> needs rect ~ 140px wide (with padding).
+- Always compute: `rect_width = max(title_chars x 8, subtitle_chars x 7) + 48px padding`.
 
 ### Colors (Light/Dark Mode Safe)
 Use these semantic color sets that work in both modes:
@@ -169,7 +175,7 @@ If you're rendering inside a system that supports CSS variables, prefer:
 
 1. **ViewBox height**: Find your lowest element (max y + height). Set H = that + 40px.
 2. **No content past x=640 or below y=(H-40)**.
-3. **Text fits in boxes**: `(char_count × 8) + 48 < rect_width` for 14px text.
+3. **Text fits in boxes**: `(char_count x 8) + 48 < rect_width` for 14px text.
 4. **No arrows through boxes**: Trace every line's path — if it crosses a rect, reroute.
 5. **All `<path>` connectors have `fill="none"`**.
 6. **All text has appropriate fill color** — never rely on inheritance (SVG defaults to black).
@@ -207,7 +213,7 @@ For complex topics, use multiple smaller SVGs instead of one dense one:
   <text x="340" y="56" text-anchor="middle" dominant-baseline="central"
         font-size="12" fill="#0F6E56">HTTP POST /api/data</text>
 
-  <!-- Arrow 1→2 -->
+  <!-- Arrow 1->2 -->
   <line x1="340" y1="76" x2="340" y2="100" stroke="#534AB7"
         stroke-width="1.5" marker-end="url(#arrow)"/>
 
@@ -219,7 +225,7 @@ For complex topics, use multiple smaller SVGs instead of one dense one:
   <text x="340" y="142" text-anchor="middle" dominant-baseline="central"
         font-size="12" fill="#534AB7">Validate and transform</text>
 
-  <!-- Arrow 2→3 -->
+  <!-- Arrow 2->3 -->
   <line x1="340" y1="162" x2="340" y2="186" stroke="#854F0B"
         stroke-width="1.5" marker-end="url(#arrow)"/>
 

apps/agent/skills/svg-diagram-skill.txt apps/agent/skills/svg-diagrams/SKILL.mdapps/agent/skills/svg-diagram-skill.txt renamed to apps/agent/skills/svg-diagrams/SKILL.md

@@ -369,7 +369,7 @@ window.addEventListener('message', function(e) {
     var scriptCloses = (rawHtml.match(/<\\/script>/gi) || []).length;
     var allScriptsClosed = scriptOpens <= scriptCloses;
     tmp.querySelectorAll('script').forEach(function(s) {
-      incomingScripts.push({ src: s.src, text: s.textContent });
+      incomingScripts.push({ src: s.src, text: s.textContent, type: s.type || '' });
       s.remove();
     });
 
@@ -414,24 +414,36 @@ window.addEventListener('message', function(e) {
       content.innerHTML = tmp.innerHTML;
     }
 
-    // Execute only new scripts — skip entirely while a <script> tag is still streaming
+    // Execute only new scripts — skip entirely while a <script> tag is still streaming.
+    // External scripts (src) are loaded sequentially and we wait for each to finish
+    // before running subsequent scripts, so inline code can use libraries like Three.js.
     if (allScriptsClosed) {
-      incomingScripts.forEach(function(scriptInfo) {
+      (function runScripts(scripts, idx) {
+        if (idx >= scripts.length) return;
+        var scriptInfo = scripts[idx];
         var key = scriptInfo.src || scriptInfo.text;
-        if (!key || !key.trim()) return;
+        if (!key || !key.trim()) { runScripts(scripts, idx + 1); return; }
         var hash = btoa(unescape(encodeURIComponent(key))).slice(0, 16).replace(/[^a-zA-Z0-9]/g, '');
-        if (!hash || content.getAttribute('data-exec-' + hash)) return;
+        if (!hash || content.getAttribute('data-exec-' + hash)) { runScripts(scripts, idx + 1); return; }
         content.setAttribute('data-exec-' + hash, '1');
         try {
           var newScript = document.createElement('script');
+          if (scriptInfo.type) newScript.type = scriptInfo.type;
           if (scriptInfo.src) {
             newScript.src = scriptInfo.src;
+            newScript.onload = function() { runScripts(scripts, idx + 1); };
+            newScript.onerror = function() { runScripts(scripts, idx + 1); };
+            content.appendChild(newScript);
           } else {
             newScript.textContent = scriptInfo.text;
+            content.appendChild(newScript);
+            runScripts(scripts, idx + 1);
           }
-          content.appendChild(newScript);
-        } catch(e) { console.warn('[widget] script exec failed:', e); }
-      });
+        } catch(e) {
+          console.warn('[widget] script exec failed:', e);
+          runScripts(scripts, idx + 1);
+        }
+      })(incomingScripts, 0);
     }
     reportHeight();
   }
@@ -469,7 +481,11 @@ function assembleShell(initialHtml: string = ""): string {
     style-src 'unsafe-inline';
     img-src 'self' data: blob:;
     font-src 'self' data:;
-    connect-src 'self';
+    connect-src 'self'
+      https://cdnjs.cloudflare.com
+      https://esm.sh
+      https://cdn.jsdelivr.net
+      https://unpkg.com;
   ">
   <style>
     ${THEME_CSS}