Skip to content

Latest commit

 

History

History
310 lines (285 loc) · 20 KB

File metadata and controls

310 lines (285 loc) · 20 KB

5 Optimization

Part of: User Journey Documentation


Phase 5: Ongoing Monitoring & Optimization

5.1 Statistics Dashboard

User navigates to Statistics dashboard:
   ↓
Views analytics:
- Ticket statistics: Completion rates, cycle times
- Agent performance: Tasks completed, code quality
- Code change statistics: Lines changed, files modified
- Project health: WIP violations, budget status
- Discovery analytics: Discovery rates by type
- Cost tracking: LLM costs per workflow

💡 For detailed cost management flows, see 11_cost_memory_management.md (Cost Dashboard, Budgets, Forecasting).

5.2 Agents Overview Page

Note: The primary sidebar navigation now links to Sandboxes (/sandboxes) instead of Agents. The /agents page is still accessible via direct URL and provides agent-level health monitoring distinct from sandbox execution tracking. See 03_execution_monitoring.md § 3.2 for the Sandbox List journey.

User navigates to /agents (via direct URL or deep link):
   ↓
Views Agents Overview Page:
   ↓
┌─────────────────────────────────────────────────────────────┐
│  Agents Overview                              [Spawn Agent] │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Agent Metrics                                       │  │
│  │                                                      │  │
│  │  ┌──────┐  ┌──────┐  ┌──────┐  ┌──────┐          │  │
│  │  │  5   │  │  3   │  │  2   │  │  1   │          │  │
│  │  │Total │  │Active│  │Idle  │  │Stuck │          │  │
│  │  └──────┘  └──────┘  └──────┘  └──────┘          │  │
│  │                                                      │  │
│  │  Average Alignment: 78%                            │  │
│  │  Tasks Completed Today: 12                          │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Agent List                                          │  │
│  │                                                      │  │
│  │  ┌──────────────────────────────────────────────┐  │  │
│  │  │ Agent: worker-1                               │  │  │
│  │  │ Status: 🟢 Active                             │  │  │
│  │  │ Phase: PHASE_IMPLEMENTATION                    │  │  │
│  │  │ Tasks: 28 total, 22 done, 2 active          │  │  │
│  │  │ Agents: 2 active                             │  │  │
│  │  │ Discoveries: 3 new branches spawned         │  │  │
│  │  │ Current Task: "Implement JWT"                │  │  │
│  │  │ Alignment: 85%                                │  │  │
│  │  │ Tasks Completed: 8                            │  │  │
│  │  │ Commits: 15                                   │  │  │
│  │  │ Lines Changed: +2,450 -120                    │  │  │
│  │  │ [View Details] [Intervene]                    │  │  │
│  │  └──────────────────────────────────────────────┘  │  │
│  │                                                      │  │
│  │  ┌──────────────────────────────────────────────┐  │  │
│  │  │ Agent: worker-2                               │  │  │
│  │  │ Status: 🟡 Idle                                │  │  │
│  │  │ Phase: PHASE_TESTING                          │  │  │
│  │  │ Tasks: 23 total, 22 done, 0 active          │  │  │
│  │  │ Agents: 0 active                             │  │  │
│  │  │ Current Task: None                            │  │  │
│  │  │ Alignment: N/A                                │  │  │
│  │  │ Tasks Completed: 5                            │  │  │
│  │  │ Commits: 8                                    │  │  │
│  │  │ Lines Changed: +890 -45                       │  │  │
│  │  │ [View Details] [Assign Task]                  │  │  │
│  │  └──────────────────────────────────────────────┘  │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  Filters: [All ▼] [Active] [Idle] [Stuck] [By Phase ▼]     │
│  Search: [________________] [🔍]                            │
└─────────────────────────────────────────────────────────────┘

Agent Metrics:

  • Total agents count
  • Active agents (currently working)
  • Idle agents (waiting for tasks)
  • Stuck agents (needs intervention)
  • Average alignment score across all agents
  • Tasks completed today
  • Total commits made
  • Total lines changed

Agent Card Details:

  • Agent ID and type
  • Current status (Active, Idle, Stuck, Failed)
  • Phase assignment (agents specialized per phase)
  • Phase-specific metrics (cost, latency, error rate per phase)
  • Phase bottlenecks (queue depth, WIP limits per phase)
  • Current task (if active)
  • Alignment score (if active)
  • Performance metrics (tasks completed, commits, lines changed)
  • Quick actions ([View Details] [Intervene] [Assign Task])

5.3 Theme Settings

User navigates to Settings → Appearance:
   ↓
Views Theme Settings:
   ↓
┌─────────────────────────────────────────────────────────────┐
│  Theme Settings                                            │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Theme Mode:                                               │
│  ○ Light (default)                                         │
│  ● Dark                                                    │
│  ○ System (follows OS preference)                          │
│                                                              │
│  Accent Color:                                             │
│  [Select Color ▼]                                          │
│  • Blue (default)                                          │
│  • Green                                                   │
│  • Purple                                                  │
│  • Orange                                                  │
│                                                              │
│  Font Size:                                                │
│  [Small] [Medium] [Large]                                  │
│                                                              │
│  Reduced Motion:                                           │
│  ☐ Enable reduced motion animations                        │
│                                                              │
│  [Save Changes]                                            │
└─────────────────────────────────────────────────────────────┘

Theme Options:

  • Light Mode: Default light theme
  • Dark Mode: Dark theme for low-light environments
  • System: Automatically follows OS theme preference
  • Accent Color: Customize primary color scheme
  • Font Size: Adjustable text size
  • Reduced Motion: Disable animations for accessibility

5.4 Search & Filtering

User uses Command Palette (Cmd+K) or Search bar:
   ↓
Search across:
- Tickets (by title, description, phase)
- Tasks (by description, status)
- Commits (by message, author, date)
- Agents (by status, capabilities)
- Code changes (by file, agent)
   ↓
Advanced filters:
- Date range
- Phase/status
- Agent
- Project
- Discovery type

5.5 Audit Trails

User views audit trail for ticket:
   ↓
Sees complete history:
- When ticket created
- All phase transitions
- All agent assignments
- All code commits
- All discoveries made
- All interventions sent
- All approvals/rejections
   ↓
Can export audit trail:
- PDF report
- CSV export
- JSON export

5.6 Monitoring System Insights

User navigates to System Health → Insights tab:
   ↓
Views monitoring patterns and adaptive learning:
   ↓
┌─────────────────────────────────────────────────────────────┐
│  Monitoring Insights                          [Export] [⚙️] │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Intervention Effectiveness                          │  │
│  │                                                      │  │
│  │  Overall Success Rate: 92% (46/50 interventions)    │  │
│  │  ████████████████████████████████████░░░░ 92%       │  │
│  │                                                      │  │
│  │  By Type:                                           │  │
│  │  • Refocus:    96% success (24/25)                  │  │
│  │  • Prioritize: 90% success (18/20)                  │  │
│  │  • Stop:       80% success (4/5)                    │  │
│  │                                                      │  │
│  │  Average Recovery Time: 2.3 minutes                 │  │
│  │  Fastest Recovery: 0.5 minutes                      │  │
│  │  Slowest Recovery: 8.2 minutes                      │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Pattern Learning Progress                           │  │
│  │                                                      │  │
│  │  Successful Patterns Stored: 12                     │  │
│  │  • OAuth implementation patterns (3)                 │  │
│  │  • Database connection patterns (4)                  │  │
│  │  • Testing workflow patterns (5)                     │  │
│  │                                                      │  │
│  │  Failure Patterns Avoided: 3                        │  │
│  │  • Infinite retry loops                              │  │
│  │  • Scope creep detection                            │  │
│  │  • Constraint violation patterns                     │  │
│  │                                                      │  │
│  │  Adaptive Thresholds:                               │  │
│  │  • Alignment threshold: 70% → 72% (adjusted)        │  │
│  │  • Intervention delay: 60s → 45s (optimized)        │  │
│  │  • Last updated: 2 hours ago                        │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Common Drift Patterns Detected                      │  │
│  │                                                      │  │
│  │  ┌─────────────────────────────────────────────┐   │  │
│  │  │ Scope Creep (34%)                           │   │  │
│  │  │ ████████████████░░░░░░░░░░░░░░░░░░░░░░░░░  │   │  │
│  │  │ Agent adds features not in requirements     │   │  │
│  │  └─────────────────────────────────────────────┘   │  │
│  │                                                      │  │
│  │  ┌─────────────────────────────────────────────┐   │  │
│  │  │ Test Skipping (28%)                         │   │  │
│  │  │ ████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  │   │  │
│  │  │ Agent proceeds without running tests        │   │  │
│  │  └─────────────────────────────────────────────┘   │  │
│  │                                                      │  │
│  │  ┌─────────────────────────────────────────────┐   │  │
│  │  │ Constraint Violations (22%)                 │   │  │
│  │  │ ██████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  │   │  │
│  │  │ Agent ignores constraints from earlier      │   │  │
│  │  └─────────────────────────────────────────────┘   │  │
│  │                                                      │  │
│  │  ┌─────────────────────────────────────────────┐   │  │
│  │  │ Idle After Completion (16%)                 │   │  │
│  │  │ ███████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  │   │  │
│  │  │ Agent finishes but doesn't update status    │   │  │
│  │  └─────────────────────────────────────────────┘   │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Conductor Coherence History                         │  │
│  │                                                      │  │
│  │  System Coherence (Last 24h):                       │  │
│  │  ████████████████████████████████████████ 96%       │  │
│  │                                                      │  │
│  │  Duplicate Work Prevented: 2 instances              │  │
│  │  • worker-1 and worker-3 both starting on JWT       │  │
│  │  • worker-2 attempting completed task               │  │
│  │                                                      │  │
│  │  Agent Conflicts Resolved: 1 instance               │  │
│  │  • Merge conflict between parallel branches         │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Cross-Project Learning (Organization-wide)          │  │
│  │                                                      │  │
│  │  Patterns shared from other projects: 8             │  │
│  │  Patterns contributed to org knowledge: 4           │  │
│  │                                                      │  │
│  │  Top Shared Patterns:                               │  │
│  │  • PostgreSQL connection pooling (used 12x)         │  │
│  │  • OAuth2 token refresh handling (used 8x)          │  │
│  │  • API rate limiting patterns (used 6x)             │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Monitoring Insights Available:

  • Intervention Effectiveness: Success rates by intervention type, recovery times
  • Pattern Learning: Successful patterns stored, failure patterns avoided, adaptive threshold adjustments
  • Drift Pattern Analysis: Common reasons agents drift (scope creep, test skipping, constraint violations, idle detection)
  • Conductor Coherence History: Duplicate work prevention, conflict resolution
  • Cross-Project Learning: Patterns shared across organization projects

How the System Learns:

  1. Successful interventions → Stored as patterns → Reused in similar situations
  2. Failed interventions → Marked as failure patterns → Avoided in future
  3. Agent discoveries → Stored in semantic memory → Referenced by future agents
  4. Threshold adjustments → Based on success/failure rates → Continuously optimized


Next: See README.md for complete documentation index.