Skip to content

Commit a7c7dfa

Browse files
committed
feat(orchestration): AgentExecutor seam + parallel combinator (Phase 1)
Introduces a new `orchestration` module: the AgentExecutor trait — the seam between the framework's orchestration grammar and the host's placement/transport/scheduling (书安OS). The framework owns the grammar + serializable data contracts (AgentStepSpec / StepOutcome) and ships a local default; a host implements AgentExecutor to place steps on remote nodes. concurrency_hint() is a *hint*, not a hard local bound — that is what lets orchestration scale past one process. - `AgentExecutor` trait + `AgentStepSpec`/`StepOutcome` (serializable for transport + future workflow checkpoints). - `execute_steps_parallel` — the barrier (parallel) primitive over the seam, reusing run_ordered_parallel_with_limit (order-preserving, per-branch panic isolation). Later combinators (pipeline, phases) build on this. - `impl AgentExecutor for TaskExecutor` — the in-box local executor (runs each step as a child AgentLoop on this node's tokio runtime). - TaskExecutor::execute_parallel now routes through the seam, so the model- driven fan-out and the programmable grammar share one execution path (load-bearing, not vestigial). Additive + zero-regression: model-driven delegation behavior is unchanged. Tests: 4 orchestration unit tests (mock executor — fan-out, order, concurrency hint, failure+panic isolation), all 59 task tests green, and the real-LLM parallel-delegation integration tests pass against .a3s/config.acl.
1 parent 63c9d27 commit a7c7dfa

4 files changed

Lines changed: 404 additions & 49 deletions

File tree

core/src/lib.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ pub mod llm;
9494
pub mod loop_checkpoint;
9595
pub mod mcp;
9696
pub mod memory;
97+
pub mod orchestration;
9798
pub(crate) mod ordered_parallel;
9899
pub mod permissions;
99100
pub mod planning;
@@ -134,6 +135,7 @@ pub use llm::{
134135
ContentBlock, HttpMetricsCallback, HttpMetricsRecord, ImageSource, LlmClient, LlmResponse,
135136
Message, OpenAiClient, TokenUsage,
136137
};
138+
pub use orchestration::{execute_steps_parallel, AgentExecutor, AgentStepSpec, StepOutcome};
137139
pub use prompts::{AgentStyle, DetectionConfidence, PlanningMode, SystemPromptSlots};
138140
pub use run::{
139141
ActiveToolSnapshot, InMemoryRunStore, RunEventRecord, RunHandle, RunRecord, RunSnapshot,

core/src/orchestration/executor.rs

Lines changed: 292 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,292 @@
1+
//! The `AgentExecutor` seam and its barrier (`parallel`) primitive.
2+
3+
use crate::agent::{AgentEvent, DEFAULT_MAX_PARALLEL_TASKS};
4+
use crate::ordered_parallel::run_ordered_parallel_with_limit;
5+
use async_trait::async_trait;
6+
use serde::{Deserialize, Serialize};
7+
use std::sync::Arc;
8+
use tokio::sync::broadcast;
9+
10+
/// A single unit of orchestrated agent work — *what* to run, independent of
11+
/// *where* it runs.
12+
///
13+
/// Serializable on purpose: a host (书安OS) may ship it to another node, and
14+
/// a future workflow checkpoint persists it. The orchestration layer assigns
15+
/// `task_id`; everything else mirrors a delegated task.
16+
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
17+
pub struct AgentStepSpec {
18+
/// Stable id for this step. Flows into lifecycle events (and, later,
19+
/// workflow checkpoints) so a step can be correlated and resumed.
20+
pub task_id: String,
21+
/// Registry key of the agent to run (e.g. `"explore"`, `"review"`).
22+
pub agent: String,
23+
/// Short human label for display/tracking.
24+
pub description: String,
25+
/// The instruction handed to the child agent.
26+
pub prompt: String,
27+
/// Optional per-step tool-round cap.
28+
#[serde(default, skip_serializing_if = "Option::is_none")]
29+
pub max_steps: Option<usize>,
30+
/// Parent session id, for lifecycle-event correlation.
31+
#[serde(default, skip_serializing_if = "Option::is_none")]
32+
pub parent_session_id: Option<String>,
33+
}
34+
35+
impl AgentStepSpec {
36+
/// A step that runs `agent` with `prompt`, identified by `task_id`.
37+
pub fn new(
38+
task_id: impl Into<String>,
39+
agent: impl Into<String>,
40+
description: impl Into<String>,
41+
prompt: impl Into<String>,
42+
) -> Self {
43+
Self {
44+
task_id: task_id.into(),
45+
agent: agent.into(),
46+
description: description.into(),
47+
prompt: prompt.into(),
48+
max_steps: None,
49+
parent_session_id: None,
50+
}
51+
}
52+
53+
pub fn with_max_steps(mut self, max_steps: usize) -> Self {
54+
self.max_steps = Some(max_steps);
55+
self
56+
}
57+
58+
pub fn with_parent_session_id(mut self, parent_session_id: impl Into<String>) -> Self {
59+
self.parent_session_id = Some(parent_session_id.into());
60+
self
61+
}
62+
}
63+
64+
/// The result of running one [`AgentStepSpec`] to completion.
65+
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
66+
pub struct StepOutcome {
67+
pub task_id: String,
68+
pub session_id: String,
69+
pub agent: String,
70+
pub output: String,
71+
pub success: bool,
72+
}
73+
74+
impl StepOutcome {
75+
/// A failed outcome for a step that could not start (e.g. unknown agent)
76+
/// or whose fan-out branch panicked. `session_id` mirrors the id the
77+
/// local executor would have derived, so failed steps remain addressable.
78+
pub fn failed(
79+
task_id: impl Into<String>,
80+
agent: impl Into<String>,
81+
message: impl Into<String>,
82+
) -> Self {
83+
let task_id = task_id.into();
84+
let session_id = format!("task-run-{task_id}");
85+
Self {
86+
task_id,
87+
session_id,
88+
agent: agent.into(),
89+
output: message.into(),
90+
success: false,
91+
}
92+
}
93+
}
94+
95+
/// Runs agent steps — the seam between the framework's orchestration grammar
96+
/// and the host's placement / transport / scheduling.
97+
///
98+
/// The in-box [`TaskExecutor`](crate::tools::TaskExecutor) runs every step
99+
/// locally (in-process, tokio). A host such as 书安OS implements this trait to
100+
/// place steps on remote nodes; the orchestration combinators are written
101+
/// purely against the trait and never observe where a step actually ran. The
102+
/// framework deliberately does **not** own placement, transport, or
103+
/// cross-node scheduling — those are the host's.
104+
#[async_trait]
105+
pub trait AgentExecutor: Send + Sync {
106+
/// Run one step to completion.
107+
///
108+
/// Failures are reported as `StepOutcome { success: false, .. }` rather
109+
/// than a hard error, so a fan-out can continue when one branch fails.
110+
/// `event_tx`, when present, receives the step's lifecycle/progress
111+
/// [`AgentEvent`]s.
112+
async fn execute_step(
113+
&self,
114+
spec: AgentStepSpec,
115+
event_tx: Option<broadcast::Sender<AgentEvent>>,
116+
) -> StepOutcome;
117+
118+
/// Advisory ceiling on how many steps the orchestration layer should run
119+
/// concurrently. The local default returns its `max_parallel_tasks`; a
120+
/// scheduler-backed host may return its cluster-wide target. It is a
121+
/// *hint*, not a hard local bound — that is what lets orchestration scale
122+
/// past a single process.
123+
fn concurrency_hint(&self) -> usize {
124+
DEFAULT_MAX_PARALLEL_TASKS
125+
}
126+
}
127+
128+
/// Fan `specs` out across the executor, bounded by its
129+
/// [`concurrency_hint`](AgentExecutor::concurrency_hint), preserving input
130+
/// order. A panicked branch becomes a failed [`StepOutcome`] without dropping
131+
/// the others.
132+
///
133+
/// This is the barrier (`parallel`) primitive — it awaits every step. Later
134+
/// combinators (pipeline, phases) build on the same seam.
135+
pub async fn execute_steps_parallel(
136+
executor: Arc<dyn AgentExecutor>,
137+
specs: Vec<AgentStepSpec>,
138+
event_tx: Option<broadcast::Sender<AgentEvent>>,
139+
) -> Vec<StepOutcome> {
140+
let limit = executor.concurrency_hint();
141+
// Keep (task_id, agent) by index so a panicked branch still yields a
142+
// correctly-labelled failed outcome (mirrors TaskExecutor's fallback).
143+
let labels: Vec<(String, String)> = specs
144+
.iter()
145+
.map(|s| (s.task_id.clone(), s.agent.clone()))
146+
.collect();
147+
148+
let results = run_ordered_parallel_with_limit(specs, limit, move |_idx, spec| {
149+
let executor = Arc::clone(&executor);
150+
let event_tx = event_tx.clone();
151+
async move { executor.execute_step(spec, event_tx).await }
152+
})
153+
.await;
154+
155+
results
156+
.into_iter()
157+
.map(|result| match result.output {
158+
Ok(outcome) => outcome,
159+
Err(error) => {
160+
let (task_id, agent) = labels
161+
.get(result.index)
162+
.cloned()
163+
.unwrap_or_else(|| ("unknown".to_string(), "unknown".to_string()));
164+
StepOutcome::failed(task_id, agent, error.to_string())
165+
}
166+
})
167+
.collect()
168+
}
169+
170+
#[cfg(test)]
171+
mod tests {
172+
use super::*;
173+
use std::sync::atomic::{AtomicUsize, Ordering};
174+
use std::time::Duration;
175+
176+
/// Executor with no LLM — records peak concurrency and synthesizes
177+
/// outcomes from the spec, so the combinator can be tested in isolation.
178+
struct MockExecutor {
179+
hint: usize,
180+
active: Arc<AtomicUsize>,
181+
max_active: Arc<AtomicUsize>,
182+
}
183+
184+
impl MockExecutor {
185+
fn new(hint: usize) -> Self {
186+
Self {
187+
hint,
188+
active: Arc::new(AtomicUsize::new(0)),
189+
max_active: Arc::new(AtomicUsize::new(0)),
190+
}
191+
}
192+
}
193+
194+
#[async_trait]
195+
impl AgentExecutor for MockExecutor {
196+
async fn execute_step(
197+
&self,
198+
spec: AgentStepSpec,
199+
_event_tx: Option<broadcast::Sender<AgentEvent>>,
200+
) -> StepOutcome {
201+
let now = self.active.fetch_add(1, Ordering::SeqCst) + 1;
202+
self.max_active.fetch_max(now, Ordering::SeqCst);
203+
tokio::time::sleep(Duration::from_millis(20)).await;
204+
self.active.fetch_sub(1, Ordering::SeqCst);
205+
206+
// `boom` panics (exercise branch-panic isolation); `fail`
207+
// returns an unsuccessful outcome.
208+
assert!(spec.agent != "boom", "boom");
209+
StepOutcome {
210+
task_id: spec.task_id.clone(),
211+
session_id: format!("task-run-{}", spec.task_id),
212+
agent: spec.agent.clone(),
213+
output: format!("ran: {}", spec.prompt),
214+
success: spec.agent != "fail",
215+
}
216+
}
217+
fn concurrency_hint(&self) -> usize {
218+
self.hint
219+
}
220+
}
221+
222+
fn spec(id: &str, agent: &str) -> AgentStepSpec {
223+
AgentStepSpec::new(id, agent, "d", format!("prompt-{id}"))
224+
}
225+
226+
#[tokio::test]
227+
async fn fans_out_in_input_order() {
228+
let exec: Arc<dyn AgentExecutor> = Arc::new(MockExecutor::new(8));
229+
let specs = vec![spec("a", "explore"), spec("b", "review"), spec("c", "plan")];
230+
let out = execute_steps_parallel(exec, specs, None).await;
231+
assert_eq!(
232+
out.iter().map(|o| o.task_id.as_str()).collect::<Vec<_>>(),
233+
vec!["a", "b", "c"],
234+
"results preserve input order"
235+
);
236+
assert!(out.iter().all(|o| o.success));
237+
assert_eq!(out[0].output, "ran: prompt-a");
238+
}
239+
240+
#[tokio::test]
241+
async fn respects_concurrency_hint() {
242+
let mock = MockExecutor::new(2);
243+
let max_active = Arc::clone(&mock.max_active);
244+
let exec: Arc<dyn AgentExecutor> = Arc::new(mock);
245+
let specs = (0..6).map(|i| spec(&i.to_string(), "explore")).collect();
246+
let _ = execute_steps_parallel(exec, specs, None).await;
247+
assert_eq!(
248+
max_active.load(Ordering::SeqCst),
249+
2,
250+
"never more than concurrency_hint steps run at once"
251+
);
252+
}
253+
254+
#[tokio::test]
255+
async fn isolates_failed_and_panicked_steps() {
256+
let exec: Arc<dyn AgentExecutor> = Arc::new(MockExecutor::new(8));
257+
let specs = vec![
258+
spec("ok", "explore"),
259+
spec("bad", "fail"),
260+
spec("crash", "boom"),
261+
spec("ok2", "review"),
262+
];
263+
let out = execute_steps_parallel(exec, specs, None).await;
264+
assert_eq!(out.len(), 4, "every step yields a result");
265+
assert!(out[0].success);
266+
assert!(
267+
!out[1].success,
268+
"explicit failure surfaces as success=false"
269+
);
270+
assert!(
271+
!out[2].success && out[2].agent == "boom",
272+
"a panicked branch becomes a labelled failed outcome, not a drop"
273+
);
274+
assert!(out[3].success, "later steps unaffected by an earlier panic");
275+
}
276+
277+
#[tokio::test]
278+
async fn default_concurrency_hint_is_the_framework_default() {
279+
struct Bare;
280+
#[async_trait]
281+
impl AgentExecutor for Bare {
282+
async fn execute_step(
283+
&self,
284+
spec: AgentStepSpec,
285+
_tx: Option<broadcast::Sender<AgentEvent>>,
286+
) -> StepOutcome {
287+
StepOutcome::failed(spec.task_id, spec.agent, "unused")
288+
}
289+
}
290+
assert_eq!(Bare.concurrency_hint(), DEFAULT_MAX_PARALLEL_TASKS);
291+
}
292+
}

core/src/orchestration/mod.rs

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
//! Programmable, deterministic multi-agent orchestration.
2+
//!
3+
//! Today an agent fans work out only by *model-driven* delegation (the LLM
4+
//! decides to call the `task` / `parallel_task` tool). This module adds a
5+
//! *programmable* layer: a developer expresses fan-out / pipelines /
6+
//! verification panels as code, so the orchestration is reproducible,
7+
//! testable, budget-bounded, and (later) resumable — independent of what the
8+
//! model chooses to do.
9+
//!
10+
//! ## The framework / host boundary
11+
//!
12+
//! Everything here is written against one seam, [`AgentExecutor`]: "run this
13+
//! step, give me the result." The framework owns the **grammar** (which steps,
14+
//! how they compose, the concurrency *hint*, the data contracts
15+
//! [`AgentStepSpec`] / [`StepOutcome`]) and ships a default executor that runs
16+
//! every step locally. The **placement** of those steps across a cluster —
17+
//! transport, scheduling, where a step actually executes — belongs to the
18+
//! host (书安OS), which implements [`AgentExecutor`]. The combinators never
19+
//! observe where a step ran, so the same orchestration scales from one process
20+
//! to a cluster without change.
21+
//!
22+
//! The in-box implementation is [`crate::tools::TaskExecutor`].
23+
24+
mod executor;
25+
26+
pub use executor::{execute_steps_parallel, AgentExecutor, AgentStepSpec, StepOutcome};

0 commit comments

Comments
 (0)