Read this before modifying orchestration, sentinel protocol, session reconciliation, or reflection loops.
PolyPilot's multi-agent system lets you create a team of AI sessions that work together. Each session can use a different AI model. An orchestrator coordinates work dispatch, response collection, and quality evaluation.
| File | Purpose |
|---|---|
PolyPilot/Services/CopilotService.Organization.cs |
Orchestration engine (dispatch, reflection loop, reconciliation, group deletion) |
PolyPilot/Models/SessionOrganization.cs |
SessionGroup, SessionMeta, MultiAgentMode, MultiAgentRole |
PolyPilot/Models/ReflectionCycle.cs |
Reflection state, stall detection, sentinel parsing, evaluator prompts |
PolyPilot/Models/ModelCapabilities.cs |
GroupPreset, UserPresets (three-tier merge), built-in presets |
PolyPilot/Models/SquadDiscovery.cs |
Squad directory parser (.squad/ → GroupPreset) |
PolyPilot/Models/SquadWriter.cs |
Squad directory writer (GroupPreset → .squad/) |
PolyPilot/Services/CopilotService.Events.cs |
TCS completion (IsProcessing → TrySetResult ordering) |
PolyPilot/Components/Layout/SessionSidebar.razor |
Preset picker UI (sectioned: From Repo / Built-in / My Presets) |
PolyPilot.Tests/MultiAgentRegressionTests.cs |
37 regression tests covering all known bugs |
PolyPilot.Tests/SessionOrganizationTests.cs |
15 grouping stability tests |
PolyPilot.Tests/SquadDiscoveryTests.cs |
22 Squad discovery tests |
PolyPilot.Tests/SquadWriterTests.cs |
15 Squad write-back tests |
PolyPilot.Tests/Scenarios/multi-agent-scenarios.json |
Executable CDP test scenarios |
Same prompt sent to all sessions simultaneously. No orchestrator. Each session responds independently. Use for: comparing model outputs, getting diverse perspectives.
Prompt sent to sessions one at a time. Each session sees previous responses. Use for: chain-of-thought across models, iterative refinement.
One orchestrator session plans and delegates:
- Plan — Orchestrator receives user prompt + list of available workers with their models
- Dispatch — Orchestrator emits
@worker:name taskassignments, parsed byParseTaskAssignments - Collect — Workers execute in parallel (
Task.WhenAll), each with 10-min timeout - Synthesize — Worker results sent back to orchestrator for final synthesis
No iteration. One pass through the loop.
Same as Orchestrator but loops until the goal is met, quality stalls, or max iterations reached. This is the primary mode for serious multi-agent work.
There are two kinds of text inputs in multi-agent groups:
- Group-level inputs — "Send All" broadcast bars that dispatch a prompt to all sessions at once (or to the orchestrator). These appear in the group header area.
- Per-session inputs — Individual chat inputs on each session card/expanded view. These let users talk to any individual session directly.
Rule: Per-session inputs are ALWAYS visible. Users must be able to chat with individual sessions regardless of orchestration mode. Only group-level inputs change visibility based on the mode.
| Mode | Group Input | Notes |
|---|---|---|
| Broadcast | ✅ Visible | Textarea + "📡 Send All" button. Sends same prompt to all sessions. |
| Sequential | ✅ Visible | Textarea + "📡 Send All" button. Sends prompt to sessions one at a time. |
| Orchestrator | ❌ Hidden | User types in the orchestrator session's own input instead. |
| OrchestratorReflect | ❌ Hidden | Iterations spinner shown instead. User types in orchestrator session's input. |
| # | Location | File | CSS class | Type | Visibility |
|---|---|---|---|---|---|
| 1 | Dashboard expanded toolbar | Dashboard.razor |
ma-expanded-toolbar-input |
Group | Hidden for Orchestrator/Reflect |
| 2 | Dashboard grid header | Dashboard.razor |
ma-broadcast-input |
Group | Hidden for Orchestrator (Reflect shows iterations only) |
| 3 | Sidebar group controls | SessionSidebar.razor |
sidebar-ma-input-bar |
Group | Hidden for Orchestrator/Reflect |
| 4 | Grid card | SessionCard.razor |
card-input |
Per-session | Always visible |
| 5 | Expanded chat view | ExpandedSessionView.razor |
input-area |
Per-session | Always visible |
| Element | Broadcast | Sequential | Orchestrator | Reflect |
|---|---|---|---|---|
| Mode dropdown | ✅ | ✅ | ✅ | ✅ |
| Group "Send All" input | ✅ | ✅ | ❌ | ❌ |
| Max iterations spinner | ❌ | ❌ | ❌ | ✅ |
| Per-session inputs | ✅ | ✅ | ✅ | ✅ |
| Phase indicator | ✅ | ✅ | ✅ | ✅ |
| Reflection progress | ❌ | ❌ | ❌ | ✅ |
When the user types in the orchestrator session's input (per-session input #4 or #5 above), the message is routed through SendViaOrchestratorAsync / SendViaOrchestratorReflectAsync — it doesn't just go to the orchestrator as a plain chat message. This routing is handled in Dashboard.razor's send handler by checking if the session belongs to a multi-agent group in Orchestrator/Reflect mode. See SendPromptToSession → group mode check → SendToMultiAgentGroupAsync.
- 1 Orchestrator — Plans, delegates, synthesizes. Set via
SessionMeta.Role = Orchestrator - N Workers — Execute assigned tasks in parallel. Each can use a different model (
SessionMeta.PreferredModel) and have a system prompt (SessionMeta.SystemPrompt) that defines their specialization - 1 Evaluator (optional) — Independent quality judge on a separate model (
ReflectionCycle.EvaluatorSessionName)
while (IsActive && !IsPaused && CurrentIteration < MaxIterations):
CurrentIteration++
Phase 1: PLAN
├── Iteration 1: BuildOrchestratorPlanningPrompt(userPrompt, workerNames)
└── Iteration 2+: BuildReplanPrompt(lastEvaluation, workerNames, userPrompt)
Orchestrator responds with task assignments:
@worker:worker-1 Implement the auth module
@worker:worker-2 Write tests for the auth module
ParseTaskAssignments extracts these → List<TaskAssignment>
If no assignments AND iteration == 1 → error (retry up to 3 times)
If no assignments AND iteration > 1 → orchestrator decided goal is met → break
Phase 2: DISPATCH
└── Send each assignment to its worker in parallel (Task.WhenAll)
Each worker gets: "You are a worker agent..." + original prompt + assigned task
Phase 3: COLLECT
└── Wait for all workers (SendPromptAndWaitAsync, 10-min timeout per worker)
Returns List<WorkerResult> (response, success, duration)
Phase 4: EVALUATE (two paths)
├── WITH dedicated evaluator:
│ ├── Orchestrator synthesizes worker results
│ ├── Evaluator scores quality (0.0–1.0) with rationale
│ ├── Score ≥ 0.9 or [[GROUP_REFLECT_COMPLETE]] → goal met → break
│ └── RecordEvaluation tracks trend (Improving/Stable/Degrading)
│
└── SELF-evaluation (no evaluator):
├── Orchestrator gets combined synthesis + eval prompt
├── [[GROUP_REFLECT_COMPLETE]] sentinel → goal met → break
└── [[NEEDS_ITERATION]] sentinel → scored as 0.4, continue
Phase 5: STALL DETECTION
├── CheckStall() compares synthesis response to previous
├── Jaccard token similarity > 0.9 → stall detected
├── 1st consecutive stall: warn but continue
└── 2nd consecutive stall: IsStalled = true → break
Phase 6: AUTO-ADJUST
└── AutoAdjustFromFeedback analyzes worker results, may suggest model changes
SaveOrganization() after each iteration
| Condition | How Detected | State |
|---|---|---|
| ✅ Goal met | Evaluator score ≥ 0.9 or [[GROUP_REFLECT_COMPLETE]] sentinel |
GoalMet = true |
| ⏱️ Max iterations | CurrentIteration >= MaxIterations |
IsCancelled = true |
| 2 consecutive responses with >90% Jaccard similarity | IsStalled = true, IsCancelled = true |
|
| 3 consecutive errors within a single iteration | IsStalled = true, IsCancelled = true |
|
| 🛑 Cancelled | CancellationToken triggered or user StopGroupReflection |
IsCancelled = true |
| ⏸️ Paused | User set IsPaused = true |
Loop condition fails |
IsCancelled invariant: Every non-success exit MUST set IsCancelled = true. This allows BuildCompletionSummary() to distinguish successful completion from abnormal termination. GoalMet = true paths must NOT set IsCancelled.
Where: CopilotService.Events.cs → CompleteResponse() and SessionErrorEvent handler
The rule: When completing a response via the TaskCompletionSource (TCS), you MUST set IsProcessing = false BEFORE calling TrySetResult() or TrySetException().
Why: In reflection loops, the TCS continuation runs synchronously. The next SendPromptAsync in the loop checks IsProcessing — if it's still true, it throws "already processing". This killed reflection loops after 1 iteration.
// ✅ CORRECT ORDER
state.IsProcessing = false; // 1. Clear flag first
state.ResponseCompletion?.TrySetResult(response); // 2. Then signal completion
// ❌ WRONG — breaks reflection loops
state.ResponseCompletion?.TrySetResult(response); // Continuation runs NOW
state.IsProcessing = false; // Too late — next SendPromptAsync already threwSame rule applies to error paths (TrySetException).
Where: CopilotService.Organization.cs → ReconcileOrganization()
The rule: Sessions that belong to multi-agent groups must NOT be auto-moved to repo groups during reconciliation. Two protections:
- Active group members: If a session's
GroupIdmatches anyIsMultiAgentgroup, skip it - Orphaned multi-agent sessions (group was deleted): If
Role == OrchestratororPreferredModel != null, don't auto-move to repo groups — these markers indicate the session was part of a multi-agent group
Why: Reconciliation runs twice on startup (once in LoadOrganization, once after RestorePreviousSessionsAsync). Without protection, it redistributes multi-agent sessions across repo-based groups, destroying the team.
Why: The app calls SaveOrganization() from ~30 places, constantly overwriting the file with its in-memory state. Any external edits are lost within seconds. To fix organization state: kill app → edit file → relaunch.
Sentinels:
[[GROUP_REFLECT_COMPLETE]]— Goal achieved, stop iterating[[NEEDS_ITERATION]]— More work needed, continue[[REFLECTION_COMPLETE]]— Single-agent reflection goal met
Detection: StringComparison.OrdinalIgnoreCase for multi-agent; strict regex ^\s*\[\[REFLECTION_COMPLETE\]\]\s*$ (multiline) for single-agent.
Where: ExecuteWorkerAsync (line ~772)
Why: Workers receive only their assigned subtask from the orchestrator. Without the original user request as context, they can't understand the broader goal. The prompt format is:
You are a worker agent. Complete the following task thoroughly.
## Original User Request (context)
{originalPrompt}
## Your Assigned Task
{task}
Where: CopilotService.Events.cs → HandleSessionEvent, isCurrentState gate
The rule: When a session is reconnected, the old session's event handler becomes orphaned. ALL events from orphaned handlers must be blocked (not just terminal events). The isCurrentState check compares the captured state object with _sessions[sessionName] — if they don't match, the handler is orphaned.
Why: Orphaned handlers can produce ghost text deltas, phantom tool executions, and stale history entries that corrupt the current session's state.
Where: CopilotService.cs → reconnect logic
The rule: _sessions[sessionName] = newState MUST execute BEFORE newSession.On(evt => HandleSessionEvent(newState, evt)). If the handler is wired first, early events from the new session see isCurrentState=false (because _sessions still points to old state) and get incorrectly dropped.
Where: CopilotService.cs and CopilotService.Events.cs — all _queuedImagePaths access
The rule: Every mutation of _queuedImagePaths (enqueue, dequeue, remove, clear, rename, close) must be inside lock (_imageQueueLock). The inner lists (List<List<string>>) are not thread-safe.
Where: CopilotService.Events.cs → CompleteResponse, SessionErrorEvent, watchdog timeout
The rule: state.Info.IsResumed = false must be set in every code path that sets IsProcessing = false. Otherwise, subsequent turns inherit the resumed session's 600s tool timeout.
Where: All new TaskCompletionSource() in CopilotService.Events.cs
The rule: Always use new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously). Without this, TCS continuations can run inline on the completing thread, causing reentrancy and stack overflows in reflection loops.
Two mechanisms, both in ReflectionCycle.CheckStall():
- Exact string match — Sliding window of last 5 full response strings. If current response matches any (full string equality, no hash) → stall.
- Jaccard token similarity — Tokenize current and previous response by whitespace. If intersection/union > 0.9 → stall.
Tolerance: 2 consecutive stalls required before stopping. First stall generates a warning. This prevents false positives from models that happen to produce similar phrasing once.
Reset: ResetStallDetection() clears history. Called when resuming from pause.
ReflectionCycle.EvaluationHistory records per-iteration:
Score(0.0–1.0)Rationale(string)EvaluatorModel(which model evaluated)Timestamp
RecordEvaluation() returns a QualityTrend:
- Improving — Latest score > previous + 0.1
- Stable — Within ±0.1
- Degrading — Latest score < previous - 0.1
Degrading trend triggers a PendingAdjustments warning suggesting model changes.
OrganizationState
├── Groups: List<SessionGroup>
│ ├── Id (GUID string)
│ ├── Name
│ ├── IsMultiAgent (bool)
│ ├── OrchestratorMode (Broadcast/Sequential/Orchestrator/OrchestratorReflect)
│ ├── OrchestratorPrompt (optional system prompt for orchestrator)
│ ├── ReflectionState: ReflectionCycle? (active cycle state)
│ ├── SharedContext (from decisions.md — prepended to worker prompts)
│ ├── RoutingContext (from routing.md — injected into orchestrator planning)
│ ├── WorktreeId, RepoId (links to repo/worktree)
│ └── SortOrder
│
└── Sessions: List<SessionMeta>
├── SessionName
├── GroupId (→ SessionGroup.Id)
├── Role (Worker/Orchestrator)
├── PreferredModel (e.g., "claude-opus-4.6")
├── SystemPrompt (worker specialization, e.g., "You are a security auditor...")
├── WorktreeId
└── IsPinned, ManualOrder
- File:
~/.polypilot/organization.json - Save:
SaveOrganization()called from ~30 places (group CRUD, session moves, reflection state updates) - Load:
LoadOrganization()on startup → deserialize →ReconcileOrganization() - Reconciliation: Matches sessions to repo groups by
WorktreeId/RepoId, prunes stale groups, protects multi-agent sessions
CreateGroupFromPresetAsync(GroupPreset) creates a full team:
- Creates
SessionGroupwith mode and metadata - Creates orchestrator session with
Role = Orchestrator,PreferredModelset - Creates N worker sessions with
PreferredModelandSystemPromptset per worker - All sessions get
WorktreeIdif provided
Worker System Prompts: Each worker can have a SystemPrompt defining its specialization. This prompt is:
- Included in
BuildOrchestratorPlanningPromptso the orchestrator knows each worker's expertise and routes tasks accordingly - Prepended to the worker's task in
ExecuteWorkerAsync(replaces the generic "You are a worker agent" prompt) - Set via
SetSessionSystemPrompt(sessionName, prompt)or viaGroupPreset.WorkerSystemPrompts
Critical: Both Role and PreferredModel must be set on all sessions. These are the markers that ReconcileOrganization uses to identify multi-agent sessions. Without them, sessions get scattered on restart.
Deleting a group via DeleteGroup(groupId) behaves differently based on group type:
-
Multi-agent groups (
IsMultiAgent == true): All sessions in the group are removed from the organization and closed asynchronously. Multi-agent sessions are meaningless without their group — they have orchestrator/worker roles, preferred models, and system prompts that only make sense within the team context. Leaving them orphaned in the default group (the old behavior) caused confusion in the sidebar. -
Regular groups (repo groups, etc.): Sessions are moved to the default group. These are standalone sessions that the user may still want to access.
Invariant: After DeleteGroup on a multi-agent group, Organization.Sessions must contain zero entries with the deleted group's ID. The async close fires CloseSessionAsync on each session (disposing the SDK session, cleaning up image queues, and tracking closed session IDs to prevent merge re-addition).
try {
// ... full iteration (plan → dispatch → collect → evaluate)
}
catch (OperationCanceledException) {
IsCancelled = true; // Mark as cancelled for BuildCompletionSummary
throw; // User cancellation propagates
}
catch (Exception ex) {
CurrentIteration--; // Retry same iteration, don't skip ahead
ConsecutiveErrors++; // Separate error counter (ConsecutiveStalls tracks repetition)
if (ConsecutiveErrors >= 3) {
IsStalled = true; // Give up after 3 retries
IsCancelled = true; // Non-success termination
break;
}
await Task.Delay(2000); // Back off before retry
}
This prevents a single transient error (network hiccup, model timeout) from killing the entire reflection cycle. ConsecutiveErrors resets to 0 on successful iterations (alongside ConsecutiveStalls), so errors must be truly consecutive.
The orchestrator's planning prompt tells it to emit assignments in this format:
@worker:worker-name-1 Description of the task for this worker
@worker:worker-name-2 Description of the task for this worker
ParseTaskAssignments uses regex @worker:(\S+)\s*([\s\S]*?)(?:@end|(?=@worker:)|$) to extract these. Workers are matched against the availableWorkers list (case-insensitive, fuzzy-matched).
If no @worker: assignments are found, the orchestrator handled the request directly and the loop exits.
MultiAgentRegressionTests.cs(37 tests) — JSON corruption, reconciliation scattering, preset markers, mode enums, reflection loop logic, TCS ordering, lifecycle scenarios, persona testsSessionOrganizationTests.cs→GroupingStabilityTests(15 tests) — JSON round-trips, delete+cleanup, orphan handling, multi-agent vs regular group deletionSquadDiscoveryTests.cs(22 tests) — Squad directory discovery, team.md parsing, charter→system-prompt, decisions/routing context, three-tier merge, legacy.ai-team/compatScenarioReferenceTests.cs— Validates scenario JSON structure, unique IDs, Squad integration scenario presence
PolyPilot.Tests/Scenarios/multi-agent-scenarios.json— CDP-based scenarios for MauiDevFlow testing against a running app
- Changed orchestration logic? → Run
MultiAgentRegressionTests - Changed reconciliation? → Run
GroupingStabilityTests - Changed TCS/event handling? → Run
ProcessingWatchdogTests+ verify reflection loop completes - Changed sentinel parsing? → Run
ReflectionCycleTests - Changed session persistence? → Run full suite, verify
organization.jsonsurvives restart
PolyPilot can discover and load team definitions from bradygaster/squad format directories (.squad/ or the legacy .ai-team/). Any repository that has been "squadified" automatically gets its teams available as presets in PolyPilot's multi-agent group creation flow.
| Squad File | PolyPilot Concept | How It's Used |
|---|---|---|
.squad/team.md |
SessionGroup + workers |
Roster parsed for agent names and roles |
.squad/agents/{name}/charter.md |
SessionMeta.SystemPrompt |
Charter content becomes worker system prompt |
.squad/routing.md |
Orchestrator planning context | Injected into BuildOrchestratorPlanningPrompt |
.squad/decisions.md |
Shared worker context | Prepended to all worker prompts as shared team knowledge |
| Squad coordinator | MultiAgentMode.OrchestratorReflect |
Squad's iterative coordinator maps to PolyPilot's reflect loop |
- User clicks 🤖 Multi → selects a worktree
SquadDiscovery.Discover(worktreePath)scans for.squad/or.ai-team/- If found, parses
team.md+ agent charters → builds aGroupPreset - Preset appears in the picker under "📂 From Repo (Squad)" section, above built-in presets
- User clicks the Squad preset →
CreateGroupFromPresetAsynccreates the group with all agents and their charters as system prompts
Built-in presets < User presets (~/.polypilot/presets.json) < Repo teams (.squad/)
Repo teams shadow built-in/user presets with the same name when working in that repo's worktree.
When a user saves a multi-agent group as a preset and the group is associated with a worktree, PolyPilot writes the team definition back to .squad/ format in the worktree root:
SaveGroupAsPresetresolves the worktree path from the group'sWorktreeIdSquadWriter.WriteFromGroupconverts the liveSessionGroup+SessionMetainto Squad files:.squad/team.md— Team name + agent roster table (Member | Role).squad/agents/{name}/charter.md— Worker system prompt as charter.squad/decisions.md— Shared context (fromGroupPreset.SharedContext).squad/routing.md— Routing context (fromGroupPreset.RoutingContext)
- The preset is also saved to
presets.jsonas a personal backup
Agent names are sanitized: team-name prefixes are stripped (e.g., "Code Review Team-worker-1" → "worker-1"), names are lowercased and non-alphanumeric characters replaced with hyphens. Roles are derived from the first sentence of the system prompt, stripping "You are a/an" prefix.
This enables round-tripping: discover a Squad team → modify it in PolyPilot → save back → others can use the updated team definition from the repo.
- No
history.mdpersistence — Squad agents accumulate learnings; PolyPilot sessions are stateless across restarts - No Scribe agent — Squad's silent decision-logger is not replicated
- No GitHub Actions integration — Squad's label triage workflows are out of scope
- No casting system — Squad's thematic name universes; PolyPilot uses agent names as-is
- Agent charters (system prompts) are capped at 4,000 characters
- Model slugs are validated against
ModelCapabilities.AllModels; unknown slugs fall back to app default - Repo presets show a 📂 source badge so users know the definition came from the repo
- No file-read directives or code execution from parsed files
public record GroupPreset(...)
{
public bool IsUserDefined { get; init; }
public bool IsRepoLevel { get; init; } // Loaded from .squad/
public string? SourcePath { get; init; } // Path to .squad/ dir
public string?[]? WorkerSystemPrompts { get; init; }
public string? SharedContext { get; init; } // From decisions.md
public string? RoutingContext { get; init; } // From routing.md
}