Skip to content

Add structured agent result callbacks #932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
willwashburn wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Add structured agent result callbacks #932

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
839d0cd
Add structured agent result callbacks
willwashburn May 20, 2026
c8299b7
Address PR #932 review feedback
willwashburn May 21, 2026
6dea769
Merge branch 'main' into codex/structured-agent-results
willwashburn May 21, 2026
5b4005e
fix: resolve clippy regressions for structured result callbacks
Copilot May 21, 2026
3332e83
Merge main and align agent result listener
willwashburn May 22, 2026
4be45b4
Avoid persisting result callback tokens
willwashburn May 22, 2026
7577197
Merge main into structured agent results
willwashburn May 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .trajectories/completed/2026-05/traj_78ytpicts778.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"id": "traj_78ytpicts778",
"version": 1,
"task": {
"title": "Address PR 932 result callback review findings"
},
"status": "completed",
"startedAt": "2026-05-22T15:59:25.187Z",
"completedAt": "2026-05-22T16:05:12.320Z",
"agents": [
{
"name": "default",
"role": "lead",
"joinedAt": "2026-05-22T16:04:55.453Z"
}
],
"chapters": [
{
"id": "chap_fjkgwh3kotyj",
"title": "Work",
"agentName": "default",
"startedAt": "2026-05-22T16:04:55.453Z",
"endedAt": "2026-05-22T16:05:12.320Z",
"events": [
{
"ts": 1779465895454,
"type": "decision",
"content": "Do not persist result callback tokens in shared MCP config: Do not persist result callback tokens in shared MCP config",
"raw": {
"question": "Do not persist result callback tokens in shared MCP config",
"chosen": "Do not persist result callback tokens in shared MCP config",
"alternatives": [],
"reasoning": "The result callback token is per spawn. Gemini/Droid mcp add, opencode.json, and .cursor/mcp.json are shared or persistent surfaces, so those paths now omit AGENT_RELAY_RESULT_* and rely on inline configs or worker process env instead."
},
"significance": "high"
}
]
}
],
"retrospective": {
"summary": "Verified PR 932 review findings. Fixed the valid shared-config token persistence issue by omitting AGENT_RELAY_RESULT_* from Gemini/Droid mcp add, opencode.json, and .cursor/mcp.json while retaining inline Codex result env. Skipped the skip_relay_prompt finding because current worker spawn already sets AGENT_RELAY_RESULT_* on the worker process even when MCP prompt injection is skipped.",
"approach": "Standard approach",
"confidence": 0.88
},
"commits": [],
"filesChanged": [],
"projectId": "/private/tmp/relay-pr-932-review",
"tags": [],
"_trace": {
"startRef": "3332e83756f0e905b77ae7f6a111e9787a66cf99",
"endRef": "3332e83756f0e905b77ae7f6a111e9787a66cf99"
}
}
31 changes: 31 additions & 0 deletions .trajectories/completed/2026-05/traj_78ytpicts778.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Trajectory: Address PR 932 result callback review findings

> **Status:** ✅ Completed
> **Confidence:** 88%
> **Started:** May 22, 2026 at 11:59 AM
> **Completed:** May 22, 2026 at 12:05 PM

---

## Summary

Verified PR 932 review findings. Fixed the valid shared-config token persistence issue by omitting AGENT_RELAY_RESULT_* from Gemini/Droid mcp add, opencode.json, and .cursor/mcp.json while retaining inline Codex result env. Skipped the skip_relay_prompt finding because current worker spawn already sets AGENT_RELAY_RESULT_* on the worker process even when MCP prompt injection is skipped.

**Approach:** Standard approach

---

## Key Decisions

### Do not persist result callback tokens in shared MCP config
- **Chose:** Do not persist result callback tokens in shared MCP config
- **Reasoning:** The result callback token is per spawn. Gemini/Droid mcp add, opencode.json, and .cursor/mcp.json are shared or persistent surfaces, so those paths now omit AGENT_RELAY_RESULT_* and rely on inline configs or worker process env instead.

---

## Chapters

### 1. Work
*Agent: default*

- Do not persist result callback tokens in shared MCP config: Do not persist result callback tokens in shared MCP config
53 changes: 53 additions & 0 deletions .trajectories/completed/2026-05/traj_ceo5q9bh2od3.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"id": "traj_ceo5q9bh2od3",
"version": 1,
"task": {
"title": "Add structured spawned-agent results"
},
"status": "completed",
"startedAt": "2026-05-20T21:24:17.929Z",
"completedAt": "2026-05-20T21:43:51.936Z",
"agents": [
{
"name": "default",
"role": "lead",
"joinedAt": "2026-05-20T21:43:48.187Z"
}
],
"chapters": [
{
"id": "chap_7lq3uodpwrbr",
"title": "Work",
"agentName": "default",
"startedAt": "2026-05-20T21:43:48.187Z",
"endedAt": "2026-05-20T21:43:51.936Z",
"events": [
{
"ts": 1779313428188,
"type": "decision",
"content": "Implemented structured agent results as a broker-mediated callback token plus MCP tool: Implemented structured agent results as a broker-mediated callback token plus MCP tool",
"raw": {
"question": "Implemented structured agent results as a broker-mediated callback token plus MCP tool",
"chosen": "Implemented structured agent results as a broker-mediated callback token plus MCP tool",
"alternatives": [],
"reasoning": "The SDK can declare a result contract at spawn time, the broker mints a per-agent callback token, and the injected Relaycast MCP server exposes submit_result without requiring the spawned agent to know broker credentials."
Comment on lines +28 to +33
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remove duplicated decision text in trajectory event payload.

The decision text is repeated ("... plus MCP tool: ... plus MCP tool"), which makes generated history noisy and harder to scan.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.trajectories/completed/2026-05/traj_ceo5q9bh2od3.json around lines 28 - 33,
The trajectory event payload duplicates the decision text in the "content" field
and in "raw.question"/"raw.chosen" (e.g., "... plus MCP tool: ... plus MCP
tool"); update the code that assembles the trajectory payload (where "content",
"raw.question", and "raw.chosen" are set) to avoid repeating the same phrase
twice—either use the canonical string once (prefer raw.chosen or raw.question)
or run a small dedupe/normalize step that collapses immediately repeated
substrings (detect "X ... : X ..." patterns or consecutive identical phrases)
before assigning to "content" so the event payload contains the decision text
only once.

},
"significance": "high"
}
]
}
],
"retrospective": {
"summary": "Added structured spawned-agent result contracts across the broker, MCP tool, and TypeScript SDK, with focused Rust and Vitest coverage plus docs.",
"approach": "Standard approach",
"confidence": 0.82
},
"commits": [],
"filesChanged": [],
"projectId": "/private/tmp/relay-structured-agent-results",
"tags": [],
"_trace": {
"startRef": "db037b0557e7353e169e92293f4adde06e48a6c6",
"endRef": "db037b0557e7353e169e92293f4adde06e48a6c6"
}
}
33 changes: 33 additions & 0 deletions .trajectories/completed/2026-05/traj_ceo5q9bh2od3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Trajectory: Add structured spawned-agent results

> **Status:** ✅ Completed
> **Confidence:** 82%
> **Started:** May 20, 2026 at 05:24 PM
> **Completed:** May 20, 2026 at 05:43 PM

---

## Summary

Added structured spawned-agent result contracts across the broker, MCP tool, and TypeScript SDK, with focused Rust and Vitest coverage plus docs.

**Approach:** Standard approach

---

## Key Decisions

### Implemented structured agent results as a broker-mediated callback token plus MCP tool

- **Chose:** Implemented structured agent results as a broker-mediated callback token plus MCP tool
- **Reasoning:** The SDK can declare a result contract at spawn time, the broker mints a per-agent callback token, and the injected Relaycast MCP server exposes submit_result without requiring the spawned agent to know broker credentials.

---

## Chapters

### 1. Work

_Agent: default_

- Implemented structured agent results as a broker-mediated callback token plus MCP tool: Implemented structured agent results as a broker-mediated callback token plus MCP tool
16 changes: 15 additions & 1 deletion .trajectories/index.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"version": 1,
"lastUpdated": "2026-05-21T19:00:06.973Z",
"lastUpdated": "2026-05-22T16:05:12.489Z",
"trajectories": {
"traj_05xg7j388bc4": {
"title": "Add browser workflow step integration",
Expand Down Expand Up @@ -1125,6 +1125,13 @@
"completedAt": "2026-05-20T11:46:17.506Z",
"path": "/Users/khaliqgant/Projects/AgentWorkforce/.msd-autofix-1bdf6c0b/.trajectories/completed/2026-05/traj_af7iew24eiip.json"
},
"traj_ceo5q9bh2od3": {
"title": "Add structured spawned-agent results",
"status": "completed",
"startedAt": "2026-05-20T21:24:17.929Z",
"completedAt": "2026-05-20T21:43:51.936Z",
"path": "/private/tmp/relay-structured-agent-results/.trajectories/completed/2026-05/traj_ceo5q9bh2od3.json"
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Avoid temp-machine absolute paths in trajectory index entries.

Storing /private/tmp/... makes index metadata non-portable and brittle across environments. Prefer a repo-relative canonical path (for example, .trajectories/completed/...) or a stable trajectory ID reference.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.trajectories/index.json at line 1133, The index entry currently contains an
absolute temp-machine path in the "path" field (e.g.
"/private/tmp/relay-structured-agent-results/.trajectories/completed/2026-05/traj_ceo5q9bh2od3.json"),
which should be replaced with a portable value; update the producer that writes
the index so the "path" value is a repo-relative canonical path like
".trajectories/completed/2026-05/traj_ceo5q9bh2od3.json" or a stable trajectory
ID reference, and ensure any code that reads the index resolves that relative
path (or looks up by ID) instead of relying on absolute temp directories—locate
the code that sets the "path" property in the index writer and change it
accordingly.

},
"traj_dcl9hgoiuac5": {
"title": "Verify --broker-name override for agent-relay up",
"status": "completed",
Expand All @@ -1138,6 +1145,13 @@
"startedAt": "2026-05-21T04:14:44.815Z",
"completedAt": "2026-05-21T04:14:45.063Z",
"path": "/private/tmp/relay-quiet-broker-logs/.trajectories/completed/2026-05/traj_u3loicehnwb4.json"
},
"traj_78ytpicts778": {
"title": "Address PR 932 result callback review findings",
"status": "completed",
"startedAt": "2026-05-22T15:59:25.187Z",
"completedAt": "2026-05-22T16:05:12.320Z",
"path": "/private/tmp/relay-pr-932-review/.trajectories/completed/2026-05/traj_78ytpicts778.json"
}
}
}
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `agent-relay rm <name>` releases broker-owned agents.
- Broker `/api/spawned/{name}/delivery-mode`, `/pending`, and `/flush` routes manage per-agent inbound queues.
- Broker `/api/input/{name}/stream` and SDK `openInputStream()` provide ordered websocket PTY input without one HTTP request per keystroke.
- Broker and TypeScript SDK structured result contracts add the `submit_result` MCP tool, `agent.waitForResult()`, per-spawn `result.onResult`, and `relay.addListener('agentResult', ...)` for typed JSON worker outcomes.
- TypeScript SDK clients can read snapshots, stream worker output, set delivery mode, inspect pending queues, and flush queued messages.
- `agent-relay replies <agent>` reads worker DM replies with JSON, unread, mark-read, sender identity, and cursor options.
- `agent-relay history` and `agent-relay replies` accept message-id `--since` cursors for incremental reads.
Expand Down Expand Up @@ -118,6 +119,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Pin next via overrides to dedupe duplicate install

### Technical Perspective

#### Dependencies & Tooling
- Upgrade @posthog/next 0.1.0 → 0.4.61 (#937)

Expand Down
Loading
Loading