feat(v1): the exchange — chat()/turn(), harness segments, resume, one user mechanism

[hallerite]

Stacked on #1939 (the run half — merge that first; this diff is against it). This PR is the exchange half split out of #1939, restacked onto the run half's synthesis surface (make_agent + Agents, flat traces with EpisodeInfo stamps, per-agent retries).

What

The exchange: one mechanism for every multi-turn conversation — the chat session — replacing the old in-boundary vf.User machinery. Any harness becomes a conversational participant; who computes the user's turns is control flow, never framework machinery.

The exchange

agent.chat(task) holds a rollout open turn-by-turn, and the caller is the run's user. Its unit is the harness segment — one launch of the program, running assistant → tools → … until it exits. turn() runs exactly one segment; the harness's own tool loop lives inside a segment, the exchange lives between segments, so turn-taking and tool-use cannot interfere. A prompted task speaks first (a bare turn() takes its opening reply); a prompt-less task is opened by the first turn(message); chat(mask_prompt=True) hides a scenario prompt from the wire while the task still scores the real row; leaving the chat() context closes the exchange (user_closed) and runs scoring. A scripted user is a loop, a modeled user is another agent's session relayed in, a game engine is neither — just Python:

async with agents.player.chat(task) as session:      # textarena: the engine plays the user
    reply = await session.turn()                     # prompted task: the model speaks first
    while not reply.stopped:
        game.step(reply.text)
        if game.state.done:
            break
        reply = await session.turn(feedback(game))

The seam on Harness is two methods: launch() runs the first segment, resume(messages) every continuation. The default resume relaunches the program on the accreted conversation as a Messages prompt — correct for any stateless chat program, free for every SUPPORTS_MESSAGE_PROMPT harness. A harness with its own session state overrides it natively: codex resumes with codex exec resume --last in a per-trace CODEX_HOME, going from refused-for-multi-turn to a first-class conversational participant. Multi-turn capability is derived from the contract (a native resume or a Messages prompt), not declared by a flag.

The interception server correspondingly becomes a pure model boundary — one request, one recorded turn; the user loop moves out of it entirely, to the rollout layer between segments.

On the run half's engine this is exactly the staged seam: ChatSession.turn(messages) is run.step(messages), and Agent.run stays the one-call special case open(); step(); close().

user-sim and the proof envs

The user-sim bundled env is the whole user-simulation mechanism, readable in one screen — a two-session relay where the user is just another agent (tool-less null harness, untrainable via brief()), the taskset row's prompt becomes the user's scenario, and the task's own rewards and judges still score the real row:

async with (
    agents.user.chat(user_task) as sim,                          # persona task: scenario in its system prompt
    agents.assistant.chat(task, mask_prompt=True) as assistant,  # same task, prompt hidden from the wire
):
    ask = await sim.turn("Hello! How can I help you today?")
    while not ask.stopped and self.config.done_marker not in ask.text:
        reply = await assistant.turn(ask.text)
        if reply.stopped:
            break
        ask = await sim.turn(reply.text)
# rollout() returns nothing — both sessions joined the episode at close,
# stamped with their agent name and the shared EpisodeInfo

Because tools live inside segments and the exchange between them, the tau-bench shape — a tool-using assistant plus a modeled user — composes by construction, pinned by a live e2e (test_env_id_user_sim_with_tools: user-sim over a tool taskset, tool call crossing mid-conversation, reward 1.0, wire masked). kuhn_poker_v1 is the self-play reference: both seats live chat sessions of the run's own model, refereed host-side, paid out zero-sum onto agent-stamped traces. The textarena taskset (wordle) and the alphabet-sort/color-codeword envs are re-ported from vf.User servers to session loops.

Restacked onto the synthesis (#1939)

The port to the reworked run half is behavior-preserving, with these deliberate mappings:

• _EpisodeAgent.chat stamps agent.name/trainable and the shared EpisodeInfo at mint and captures the trace in the episode at close — a chat driven from rollout() stays crash-safe under the flat-trace schema.
• Chat sessions take no eval gate: two of them interleave inside one episode (user-sim, kuhn), so gating them like one-shot runs would deadlock at -c 1.
• Per-agent retries (--env.<agent>.retries.*) never apply to chat() — an exchange is caller-driven, and the user's side can't be replayed.
• With mask_prompt, the masked wire view is what persists on the trace; provenance of the real row is its idx (the pre-restack episode envelope carried the unmasked task; the flat schema deliberately doesn't).
• user-sim/kuhn declare agents as config fields (no roles()/vf.Role — task × agent fit validates per run, on the task each agent actually receives).

Removed

• vf.User/UserConfig/serve_user/user_respond, Task.user, per-rollout user-server provisioning + placement config, vf init -U, and the user-placement e2e matrix — the user lives in env control flow, in the eval process; nothing to declare, place, or serve.
• The interception server's user machinery (opening cache, consult loop, dialect.extend, RolloutSession.user, UserError) and the SUPPORTS_USER_SIM flag — the exchange moved to the rollout layer, capability is derived.

Breaking / downstream

• bfcl in research-environments still declares the removed Task.user and needs a re-port (a session loop) before a verifiers bump there.

Validation

Offline: full suite (168 tests — the run half's 150 plus the exchange's session-protocol, resume-seam, user-sim pairing, and kuhn-logic pins), ruff, ty check verifiers, scripts/sync.py --check — all green on the restacked tree. The pre-split tree validated the live behavior (tau shape with a mid-conversation tool call, codex multi-turn via native resume, kuhn self-play with forfeit handling); the restacked live matrix rides this PR's CI (-m "not prime and not modal").

🤖 Generated with Claude Code

Note

Replace user simulator architecture with chat()-based exchange API for multi-turn agent environments

• Introduces Agent.chat() context manager and ChatSession with turn()/close() that let environment run() methods drive multi-turn exchanges directly, replacing the previous MCP-based user simulator server model.
• Adds Harness.resume() to continue a rollout with accumulated messages, and updates RolloutRun to support segment-based lifecycle; Harness.run() dispatches to launch() or resume() based on whether messages are provided.
• Removes all user simulator infrastructure: vf.User, UserConfig, UserError, SUPPORTS_USER_SIM, serve_user, Dialect.extend, and user-related MCP launch helpers are deleted from the public API.
• Adds UserSimEnv as a built-in environment that relays two chat sessions (assistant + modeled user agent) to replace the old user simulator pattern.
• Adds KuhnPokerEnv as a new two-player self-play environment with seeded card dealing, action validation, retry policy, and zero-sum payoff computation.
• Migrates all existing environments (TextArena, OpenEnv, AlphabetSort, ColorCodeword, Wordle) from user simulator hooks to env-driven Agent.chat() loops.
• Risk: removes previously exported symbols (User, UserConfig, UserError, serve_user, SUPPORTS_USER_SIM, Dialect.extend) — any consumer code importing these will break at runtime.

^{Macroscope summarized 4d8f9a3.}

---

Note

High Risk
Breaking API removal (vf.User, Task.user, SUPPORTS_USER_SIM) and a core change to multi-turn rollout, harness launch/resume, and interception; many example envs and downstream packages must migrate.

Overview
Replaces the vf.User server/simulator with a single exchange API: Agent.chat(task) / ChatSession.turn(), where each turn runs one harness segment (launch or resume) until the program yields; tool loops stay inside a segment, user turns between segments.

Harness contract changes: launch() takes explicit TaskData; default resume() relaunches on the accreted conversation (Messages prompt); Codex overrides with native codex exec resume. Removes SUPPORTS_USER_SIM, dialect extend(), interception user loop, UserError, and uv run init -U.

Environments: Alphabet sort, color codeword, echo fixtures, and TextArena/wordle configs move to Env.run() chat loops; adds bundled user-sim (modeled user + masked assistant) and kuhn-poker-v1 (dual-seat self-play). Config axes shift from env.agent to env.player where applicable.

Docs/skills/tests updated for the new model; e2e adds chat(), user-sim, tau-bench-with-tools, and Kuhn poker coverage.

^{Reviewed by Cursor Bugbot for commit 4d8f9a3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

🟡 Medium kuhn_poker_v1/taskset.py:121

When invalid_retries is set to -1 (a valid int config value), range(self.config.invalid_retries + 1) evaluates to range(0), so the ask loop never calls session.turn and immediately returns None. This forfeits the acting player without ever prompting them for a move, producing a forfeit result on a hand that was never played. Consider clamping invalid_retries to a nonnegative value (e.g. max(self.config.invalid_retries, 0)) before computing the loop bound.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @environments/kuhn_poker_v1/kuhn_poker_v1/taskset.py around line 121:

When `invalid_retries` is set to `-1` (a valid `int` config value), `range(self.config.invalid_retries + 1)` evaluates to `range(0)`, so the `ask` loop never calls `session.turn` and immediately returns `None`. This forfeits the acting player without ever prompting them for a move, producing a forfeit result on a hand that was never played. Consider clamping `invalid_retries` to a nonnegative value (e.g. `max(self.config.invalid_retries, 0)`) before computing the loop bound.

[macroscopeapp]

🟡 Medium v1/harness.py:185

resume strips every system-role message from the branch before relaunching, but only resolve_prompt re-emits data.system_prompt. When a task's prompt is a Messages value that includes a system message (or a prior turn supplied one) while data.system_prompt is None, that system instruction is present for the first segment but silently dropped from every resumed segment, changing model behavior. Only the system message synthesized from data.system_prompt should be de-duplicated; messages the task or user explicitly provided should survive.

Suggested change

	conversation = [*(m for m in branch if m.role != "system"), *messages]
	system_prompt = data.system_prompt
	conversation = [
	*(m for m in branch if m.role != "system" or system_prompt is None),
	*messages,
	]

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/harness.py around line 185:

`resume` strips every system-role message from the branch before relaunching, but only `resolve_prompt` re-emits `data.system_prompt`. When a task's `prompt` is a `Messages` value that includes a system message (or a prior turn supplied one) while `data.system_prompt` is `None`, that system instruction is present for the first segment but silently dropped from every resumed segment, changing model behavior. Only the system message synthesized from `data.system_prompt` should be de-duplicated; messages the task or user explicitly provided should survive.

[macroscopeapp]

🟡 Medium user_sim/env.py:64

When the task's prompt_text contains the literal {done}, the second .replace("{done}", ...) overwrites it with the done marker inside the user's system prompt, silently altering the scenario. This happens because both replacements run sequentially over the same string, so the {done} substitution processes the scenario text that was just inserted. Consider replacing {done} before {scenario}, or otherwise ensuring the scenario text is not subject to placeholder substitution.

Suggested change

	system_prompt=self.config.persona.replace(
	"{scenario}", scenario
	).replace("{done}", self.config.done_marker),
	)
	system_prompt=self.config.persona.replace(
	"{done}", self.config.done_marker
	).replace("{scenario}", scenario),

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/envs/user_sim/env.py around lines 64-67:

When the task's `prompt_text` contains the literal `{done}`, the second `.replace("{done}", ...)` overwrites it with the done marker inside the user's system prompt, silently altering the scenario. This happens because both replacements run sequentially over the same string, so the `{done}` substitution processes the scenario text that was just inserted. Consider replacing `{done}` before `{scenario}`, or otherwise ensuring the scenario text is not subject to placeholder substitution.

[macroscopeapp]

🟠 High codex/harness.py:146

resume always runs codex exec resume --last, but when a rollout is opened by turn(message) with no prior segment, the per-trace CODEX_HOME has no recorded session to resume — Codex exits immediately instead of answering the opening user message. --last requires an existing session in CODEX_HOME; on the first segment there is none. Consider detecting the first segment (e.g., empty trace/prior segments) and launching a fresh codex exec session instead of exec resume --last in that case.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/harnesses/codex/harness.py around line 146:

`resume` always runs `codex exec resume --last`, but when a rollout is opened by `turn(message)` with no prior segment, the per-trace `CODEX_HOME` has no recorded session to resume — Codex exits immediately instead of answering the opening user message. `--last` requires an existing session in `CODEX_HOME`; on the first segment there is none. Consider detecting the first segment (e.g., empty trace/prior segments) and launching a fresh `codex exec` session instead of `exec resume --last` in that case.

[macroscopeapp]

🟡 Medium interception/server.py:336

A retried streaming (SSE) request that arrives while the original stream is still active gets HTTP 503 even though the original exchange succeeds. The in-flight slot (session.inflight[req_hash]) is now claimed before the streaming branch, so a duplicate streaming request enters coalesced(), awaits the original's future, and receives None — because _stream() never calls serve() and the outer finally resolves fut to None. The duplicate then waits for the entire successful stream and is still failed with 503. Previously, streaming requests returned before the in-flight slot was claimed. Consider claiming the in-flight slot only after the streaming check, or resolving fut with the streamed completion inside _stream().

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/interception/server.py around line 336:

A retried streaming (SSE) request that arrives while the original stream is still active gets HTTP 503 even though the original exchange succeeds. The in-flight slot (`session.inflight[req_hash]`) is now claimed *before* the streaming branch, so a duplicate streaming request enters `coalesced()`, awaits the original's future, and receives `None` — because `_stream()` never calls `serve()` and the outer `finally` resolves `fut` to `None`. The duplicate then waits for the entire successful stream and is still failed with 503. Previously, streaming requests returned before the in-flight slot was claimed. Consider claiming the in-flight slot only after the streaming check, or resolving `fut` with the streamed completion inside `_stream()`.

[macroscopeapp]

🟡 Medium v1/agent.py:161

ChatSession.turn() documents a strict one-consumer request/response alternation but has no lock guarding it. Two concurrent turn() calls can both pass the _over/_started/prompted state checks before either awaits _run.step(), so both then execute RolloutRun.step() concurrently against the same harness, trace, and interception session. This interleaves user turns and segment writes, returning replies for the wrong message or corrupting conversation ordering. Add an asyncio.Lock (or an in-progress guard) around the entire turn() body so concurrent callers are serialized.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/agent.py around line 161:

`ChatSession.turn()` documents a strict one-consumer request/response alternation but has no lock guarding it. Two concurrent `turn()` calls can both pass the `_over`/`_started`/`prompted` state checks before either awaits `_run.step()`, so both then execute `RolloutRun.step()` concurrently against the same harness, trace, and interception session. This interleaves user turns and segment writes, returning replies for the wrong message or corrupting conversation ordering. Add an `asyncio.Lock` (or an in-progress guard) around the entire `turn()` body so concurrent callers are serialized.

[macroscopeapp]

🟡 Medium openenv/taskset.py:119

An empty reply.text is sent to client.step, where parse_action raises ValueError for multi-field schemas (or sends an invalid empty value for single-field schemas), aborting the rollout. The previous respond implementation skipped stepping when the message was empty. Consider skipping the step and re-sending the current observation when reply.text.strip() is empty.

                reply = await session.turn(payload())
-                while not reply.stopped and not result.done:
-                    result = await client.step(parse_action(reply.text, action_schema))
+                while not reply.stopped and not result.done:
+                    if reply.text.strip():
+                        result = await client.step(parse_action(reply.text, action_schema))
+                        total += result.reward or 0.0
+                        if result.done:
+                            break
+                    reply = await session.turn(payload())

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/tasksets/openenv/taskset.py around lines 119-125:

An empty `reply.text` is sent to `client.step`, where `parse_action` raises `ValueError` for multi-field schemas (or sends an invalid empty value for single-field schemas), aborting the rollout. The previous `respond` implementation skipped stepping when the message was empty. Consider skipping the step and re-sending the current observation when `reply.text.strip()` is empty.

[macroscopeapp]

🟡 Medium codex/harness.py:191

_env creates a per-trace CODEX_HOME under /tmp but never removes it, so each rollout leaves Codex's recorded session and state on disk. On shared/borrowed runtimes (which the docstring explicitly supports), this accumulates across many traces and can exhaust /tmp, eventually causing subsequent rollouts to fail. Consider cleaning up the home directory after the rollout completes, similar to how launch removes image_dir.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/harnesses/codex/harness.py around line 191:

`_env` creates a per-trace `CODEX_HOME` under `/tmp` but never removes it, so each rollout leaves Codex's recorded session and state on disk. On shared/borrowed runtimes (which the docstring explicitly supports), this accumulates across many traces and can exhaust `/tmp`, eventually causing subsequent rollouts to fail. Consider cleaning up the home directory after the rollout completes, similar to how `launch` removes `image_dir`.

[macroscopeapp]

🟡 Medium user_sim/env.py:79

If done_marker is set to an empty string, self.config.done_marker not in ask.text is always False, so the relay loop exits after the user's opening turn and the assistant never runs. Since done_marker is an unrestricted str config field with no validation, an empty value silently produces unusable evaluations. Consider validating that done_marker is non-empty at config/init time.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/envs/user_sim/env.py around line 79:

If `done_marker` is set to an empty string, `self.config.done_marker not in ask.text` is always `False`, so the relay loop exits after the user's opening turn and the assistant never runs. Since `done_marker` is an unrestricted `str` config field with no validation, an empty value silently produces unusable evaluations. Consider validating that `done_marker` is non-empty at config/init time.

[macroscopeapp]

Macroscope has since reviewed this pull request. An earlier review was skipped by a cost limit; a review has now completed, so that notice no longer applies.

[macroscopeapp]

Approvability

Verdict: Needs human review

10 blocking correctness issues found. Major new feature introducing chat()/turn() multi-turn conversation API, removing the old user simulator mechanism, and adding new environments. Multiple unresolved review comments identify high and medium severity bugs affecting runtime behavior. The scope and complexity of architectural changes to core framework abstractions warrants careful human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4d8f9a3. Configure here.}

[cursor]

Codex resume breaks opening turns

High Severity

CodexHarness.resume always runs codex exec resume --last, but a user-opened exchange (prompt=None or mask_prompt=True) sends the first turn(message) through resume with no prior session. That first segment needs a normal launch (or equivalent) so Codex can create a session before --last can continue it.

Additional Locations (1)

• verifiers/v1/harness.py#L118-L127

 

^{Reviewed by Cursor Bugbot for commit 4d8f9a3. Configure here.}

[cursor]

One-shot runs leave stop unset

Medium Severity

Removing trace.stop("agent_completed") from Harness.run is required for multi-segment chat, but Agent.run’s one-shot open/step/close path never stamps a stop condition. Successful single-shot traces now finish with stop_condition is None, so dashboards and anything keying on natural completion lose that signal.

Additional Locations (1)

• verifiers/v1/agent.py#L362-L383

 

^{Reviewed by Cursor Bugbot for commit 4d8f9a3. Configure here.}

[macroscopeapp]

🟡 Medium user_sim/env.py:79

The loop terminates as soon as the user's reply merely contains done_marker as a substring, but the persona instructs the user to reply with exactly the marker and nothing else. If the user's message embeds ###DONE### alongside other text (e.g., quoting it from the scenario or adding commentary), the loop exits without forwarding that message to the assistant, truncating the conversation prematurely. Consider checking ask.text.strip() == self.config.done_marker so only a standalone marker ends the exchange.

Also found in 1 other location(s)

environments/kuhn_poker_v1/kuhn_poker_v1/taskset.py:61

parse_action converts matches to a set, so repeated occurrences of the same action collapse to one. A reply such as [bet] [bet] is accepted even though the prompt and docstring require exactly one bracketed action, bypassing the invalid-move retry path.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/envs/user_sim/env.py around line 79:

The loop terminates as soon as the user's reply merely contains `done_marker` as a substring, but the persona instructs the user to reply with exactly the marker and nothing else. If the user's message embeds `###DONE###` alongside other text (e.g., quoting it from the scenario or adding commentary), the loop exits without forwarding that message to the assistant, truncating the conversation prematurely. Consider checking `ask.text.strip() == self.config.done_marker` so only a standalone marker ends the exchange.

Also found in 1 other location(s):
- environments/kuhn_poker_v1/kuhn_poker_v1/taskset.py:61 -- `parse_action` converts matches to a set, so repeated occurrences of the same action collapse to one. A reply such as `[bet] [bet]` is accepted even though the prompt and docstring require exactly one bracketed action, bypassing the invalid-move retry path.