feat(v1): the exchange — chat()/turn(), segments, resume, one user mechanism

The turn half of the agent-programs work, restacked onto the synthesis run
half (make_agent + Agents, flat traces with EpisodeInfo stamps, verdict-file
judging, per-agent retries, the session seal).

- Agent.chat(task) -> ChatSession: the caller IS the run's user; one turn()
  per harness segment (the program runs to its exit, the caller answers, the
  next segment resumes the exchange with the answer). mask_prompt hides a
  scenario prompt from the wire while the task still scores the real row.
  _EpisodeAgent.chat stamps agent name/trainable + the shared EpisodeInfo at
  mint, so env-driven sessions land in the episode exactly like plain runs.
  An exchange is caller-driven, so per-agent retries never apply to chat().
- Harness.resume: relaunch on the accreted conversation by default
  (SUPPORTS_MESSAGE_PROMPT); codex overrides it with a native continuation.
- The user loop moves OUT of the model boundary: vf.User/Task.user (session
  injection, dialect extend, serve_user, UserError, the -U scaffold, the
  placement matrix) is deleted — chat sessions in the env's rollout() are the
  one exchange mechanism. The interception drive loop flattens to one turn
  per request, keeping per-call records and the concluded-trace seal.
- textarena becomes a recipe env stepping the real engine host-side;
  alphabet-sort/color-codeword script their users through SingleAgentEnv
  sessions; openenv drives its client from rollout() (no more served user);
  user-sim bundled env (two-session relay, the tau-style substrate, null
  harness for the modeled user, untrainable via brief()) and kuhn_poker_v1
  (self-play reference) added.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: hallerite

AGENTS.md

@@ -20,7 +20,7 @@ Use these rules when shaping public v1 APIs, configs, and environment files.
 - Expose as few knobs in the configs as possible, but as many as needed.
 - Use strict Pydantic models for structured config, tasks, messages, and state.
 - A basic taskset should fit in a few dozen idiomatic lines: typed data/task/config classes, `load()`, and decorated scoring on the task.
-- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__` or `User.__init__`; use `setup()`.
+- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__`; use `setup()`.
 - Refer to the code as the source of truth.
 - Avoid mutable module globals. Process-level locks/rate limiters and immutable constants are the narrow exceptions.
 

assets/agents/common_best_practices.md

@@ -16,6 +16,6 @@ Use these rules when shaping public v1 APIs, configs, and environment files.
 - Expose as few knobs in the configs as possible, but as many as needed.
 - Use strict Pydantic models for structured config, tasks, messages, and state.
 - A basic taskset should fit in a few dozen idiomatic lines: typed data/task/config classes, `load()`, and decorated scoring on the task.
-- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__` or `User.__init__`; use `setup()`.
+- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__`; use `setup()`.
 - Refer to the code as the source of truth.
 - Avoid mutable module globals. Process-level locks/rate limiters and immutable constants are the narrow exceptions.

assets/lab/AGENTS.md

@@ -23,7 +23,7 @@ Use these rules when shaping public v1 APIs, configs, and environment files.
 - Expose as few knobs in the configs as possible, but as many as needed.
 - Use strict Pydantic models for structured config, tasks, messages, and state.
 - A basic taskset should fit in a few dozen idiomatic lines: typed data/task/config classes, `load()`, and decorated scoring on the task.
-- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__` or `User.__init__`; use `setup()`.
+- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__`; use `setup()`.
 - Refer to the code as the source of truth.
 - Avoid mutable module globals. Process-level locks/rate limiters and immutable constants are the narrow exceptions.
 

assets/lab/environments/AGENTS.md

@@ -28,12 +28,10 @@ The command also supports:
 - `-p`, `--path <dir>` — parent directory, default: `./environments`
 - `-T`, `--add-tool` — also scaffold a `vf.Toolset` tool server at `servers/tool.py`
   - Use this to create custom tools which are installed into supported harnesses via MCP.
-- `-U`, `--add-user` — also scaffold a `vf.User` simulator at `servers/user.py`
-  - Use this to simulate a user interacting with the model. Not all harnesses support user simulation.
 - `-H`, `--add-harness` — also scaffold a custom `vf.Harness` at `harness.py`, selectable via `--env.agent.harness.id <name>`
   - Prefer a built-in harness unless the model needs to run inside a custom program.
 
-Most tasksets do not need specific tools, user simulations or custom harnesses.
+Most tasksets do not need specific tools or custom harnesses. (To simulate a user interacting with the model, open a chat session from an env's `rollout()` and script the user's turns — see the [Agent docs](https://github.com/PrimeIntellect-ai/verifiers/blob/main/docs/v1/agent.md).)
 
 > For a production-scale catalog of tasksets, see the companion [`research-environments`](https://github.com/PrimeIntellect-ai/research-environments) repository.
 
@@ -336,12 +334,41 @@ checked in as `configs/agentic_judge.toml` (`uv run eval @ configs/agentic_judge
 exporting an `Environment` subclass via `__all__`, or a Hub `org/name[@version]` —
 and its `EnvConfig` surface typed on the CLI (`--env.<agent>.*`, `-h` renders
 them). Empty (the default) keeps the taskset's own story: the env its package
-ships (a *recipe* env like `code_golf_v1`, where the interaction is intrinsic to
-the data), else `SingleAgentEnv`. An explicit id wins over a bundled recipe env.
+ships (a *recipe* env like `code_golf_v1` or `kuhn_poker_v1`, where the
+interaction is intrinsic to the data), else `SingleAgentEnv`. An explicit id wins
+over a bundled recipe env.
 
 Bundled envs (`verifiers/v1/envs/`):
 
 | id | agents | what it does |
 | --- | --- | --- |
 | `best-of-n` | `agent` | `--env.n` independent attempts per rollout; its metrics mark the argmax-reward sibling (`best`) and whether any reached `--env.threshold` (`pass_at_n`) — rejection sampling and pass@k. A single-agent env keeps the single-agent name, so `--env.agent.*` flags compose unchanged. |
 | `agentic-judge` | `solver`, `judge` | agent-as-judge: the solver plays the task; a code-executing judge agent verifies the finished attempt with real execution, always in its own sandbox, never on the host. The judge's task mirrors the solver task's world (same image, a fresh box in its original state) with the graded transcript uploaded (`/tmp/transcript.md`/`.json`). The verdict channel is a file: the judge writes `{"score": 0-10, "reasoning": ...}` to `/tmp/verdict.json` in its box, scraped onto its trace while the box is alive and validated STRICTLY onto the solver's trace as the `judge` reward — a missing, malformed, or off-scale verdict fails the rollout instead of clamping. The judge must land in a container: pin `--env.judge.harness.runtime.type docker\|prime`, or construction refuses. A judgement that needs no execution belongs on the plugged tier (`env.taskset.task.judges`), not on an agent. |
+| `user-sim` | `assistant`, `user` | a modeled user (an untrainable `user` agent — `brief()` — on the `null` harness by default) opens and drives the conversation from the task's prompt-as-scenario (`--env.persona`); the assistant plays the same task through a masked chat session (`mask_prompt`) — the prompt is hidden from its harness while the task's own rewards and judges still score the real row. The substrate for tau-bench-style evals. |
+
+### User simulation: the user is just another agent
+
+There is exactly one exchange mechanism: the chat session (`agents.<name>.chat(task)`)
+— whoever calls `turn()` is the run's user, and each turn runs one harness segment
+(the program runs until it yields, the caller answers its final message, the next
+segment resumes the exchange with the answer). A prompt-less task is opened by the
+first `turn(message)`; a prompted task speaks first (take its opening reply with a
+bare `turn()`). Who computes the turns is the env's control flow, not framework
+machinery:
+
+```python
+class SortEnv(vf.SingleAgentEnv):
+    async def rollout(self, task, agents):
+        # a pre-scripted episode: the task is prompt-less, so the first turn opens
+        async with agents.agent.chat(task) as session:
+            for prompt in task.data.info["user_turns"]:
+                if (await session.turn(prompt)).stopped:
+                    break                        # a limit or @stop ended the run
+```
+
+A *scripted* user is a plain loop like this (a game engine stepping in-process works
+the same way — see the bundled `textarena` taskset). A *modeled* user is another
+agent: open both sessions and relay their `turn()`s into each other — see the bundled
+`user-sim` env, `environments/kuhn_poker_v1` (the same relay as self-play: both seats
+chat sessions of the run's own model), and [chat() in the Agent docs](https://github.com/PrimeIntellect-ai/verifiers/blob/main/docs/v1/agent.md). The
+user runs in the eval process, so there is nothing to declare, place, or serve.

configs/textarena.toml

@@ -1,7 +1,7 @@
-# textarena — single-player TextArena games driven by a user simulator (vf.User).
+# textarena — single-player TextArena games, the engine playing the user.
 # Supported (--env.taskset.game): Wordle-v0, Wordle-v0-long, Hangman-v0, WordLadder-v0,
-# WordSearch-v0. The framework's interception server plays the game; the harness never sees
-# it. The user sim runs colocated in the harness's runtime.
+# WordSearch-v0. The env's rollout() steps the real engine host-side as the run's user;
+# the harness only ever sees the conversation.
 #
 #   uv run eval @ configs/textarena.toml --model <small-instruct-model>
 num_tasks = 5
@@ -11,8 +11,8 @@ num_rollouts = 2
 id = "textarena"
 game = "Wordle-v0"
 
-[env.agent]
+[env.player]
 max_turns = 15
 
-[env.agent.harness]
+[env.player.harness]
 id = "null"

configs/wordle.toml

@@ -1,4 +1,4 @@
-# wordle-v1 — the textarena taskset pinned to Wordle, driven by a user simulator.
+# wordle-v1 — the textarena taskset pinned to Wordle, the engine playing the user.
 #
 #   uv run eval @ configs/wordle.toml --model <small-instruct-model>
 num_tasks = 5
@@ -7,8 +7,8 @@ num_rollouts = 2
 [env.taskset]
 id = "wordle-v1"
 
-[env.agent]
+[env.player]
 max_turns = 8
 
-[env.agent.harness]
+[env.player.harness]
 id = "null"

docs/v1/agent.md

@@ -42,7 +42,57 @@ trainable. Everything beyond the arrow is a parameter, not a concept:
 - **retries are the agent's own**: `run` reruns its rollout while the trace ends
   with a retryable error (`--env.<agent>.retries.max_retries`, flat
   include/exclude), with backoff — never into a borrowed box, whose state is no
-  longer the task's start state.
+  longer the task's start state. (A `chat()` exchange is caller-driven, so it is
+  never auto-retried.)
+- **the exchange is `chat()`**, the one multi-turn surface: whoever calls `turn()`
+  is the run's user, and each turn runs one harness SEGMENT — the program runs
+  until it yields (exits), the caller answers its final message, and the next
+  segment resumes the exchange with the answer (`Harness.resume`: a relaunch on
+  the accreted conversation by default; codex continues its own recorded session
+  natively). The harness's own tool loop runs entirely inside a segment, so tools
+  and a simulated user compose freely. Needs a harness that can resume (a
+  Messages prompt, or a native `resume()`).
+- **`chat(mask_prompt=True)`** says the task's `prompt` is the USER's side of the
+  story — a scenario the caller pursues, not the assistant's seed. The wire hides
+  the prompt (the caller opens instead), while the task's hooks, rewards, and
+  judges still score the real row. This is how the bundled `user-sim` env runs
+  the assistant on the original task.
+
+## chat(): be the user yourself
+
+`agent.chat(task)` is the caller side of the same channel — a full run of the task
+where YOU (or another agent's program) supply each user turn live:
+
+```python
+async with agent.chat(task) as session:          # task has no prompt: chat opens it
+    reply = await session.turn("hello there")    # one user message -> one vf.Reply
+    if not reply.stopped:
+        followup = await session.turn(f"you said: {reply.text}")
+    print(session.trace.num_turns)               # the trace is live mid-exchange
+# leaving the context ends the exchange (user_closed) and finishes the run:
+# hooks and scoring included — session.trace is the final, scored trace
+```
+
+`turn()` returns a `vf.Reply` (`text`, `stopped`); a `stopped` reply means the run
+ended (a limit, a `@stop`, the harness finishing) instead of answering. And because
+both sides speak the same channel, "the user is just another agent" — a modeled
+user is a chat session relayed into another agent's run:
+
+```python
+async with (
+    user_agent.chat(user_task) as sim,             # the modeled user (its own trace!)
+    assistant.chat(task, mask_prompt=True) as helper,
+):
+    ask = await sim.turn("Hello! How can I help you today?")
+    while not ask.stopped:
+        reply = await helper.turn(ask.text)
+        if reply.stopped:
+            break
+        ask = await sim.turn(reply.text)
+```
+
+Modeled-user runs stay cheap with the tool-less `null` harness: a bare chat-loop
+program, no tools, no container — the rollout is essentially just the model calls.
 
 Chaining is a `Task` classmethod: mint the next task from earlier traces with
 `YourTask.from_trace(...)` and hand it to the next agent.

docs/v1/environments.md

@@ -85,12 +85,41 @@ checked in as `configs/agentic_judge.toml` (`uv run eval @ configs/agentic_judge
 exporting an `Environment` subclass via `__all__`, or a Hub `org/name[@version]` —
 and its `EnvConfig` surface typed on the CLI (`--env.<agent>.*`, `-h` renders
 them). Empty (the default) keeps the taskset's own story: the env its package
-ships (a *recipe* env like `code_golf_v1`, where the interaction is intrinsic to
-the data), else `SingleAgentEnv`. An explicit id wins over a bundled recipe env.
+ships (a *recipe* env like `code_golf_v1` or `kuhn_poker_v1`, where the
+interaction is intrinsic to the data), else `SingleAgentEnv`. An explicit id wins
+over a bundled recipe env.
 
 Bundled envs (`verifiers/v1/envs/`):
 
 | id | agents | what it does |
 | --- | --- | --- |
 | `best-of-n` | `agent` | `--env.n` independent attempts per rollout; its metrics mark the argmax-reward sibling (`best`) and whether any reached `--env.threshold` (`pass_at_n`) — rejection sampling and pass@k. A single-agent env keeps the single-agent name, so `--env.agent.*` flags compose unchanged. |
 | `agentic-judge` | `solver`, `judge` | agent-as-judge: the solver plays the task; a code-executing judge agent verifies the finished attempt with real execution, always in its own sandbox, never on the host. The judge's task mirrors the solver task's world (same image, a fresh box in its original state) with the graded transcript uploaded (`/tmp/transcript.md`/`.json`). The verdict channel is a file: the judge writes `{"score": 0-10, "reasoning": ...}` to `/tmp/verdict.json` in its box, scraped onto its trace while the box is alive and validated STRICTLY onto the solver's trace as the `judge` reward — a missing, malformed, or off-scale verdict fails the rollout instead of clamping. The judge must land in a container: pin `--env.judge.harness.runtime.type docker\|prime`, or construction refuses. A judgement that needs no execution belongs on the plugged tier (`env.taskset.task.judges`), not on an agent. |
+| `user-sim` | `assistant`, `user` | a modeled user (an untrainable `user` agent — `brief()` — on the `null` harness by default) opens and drives the conversation from the task's prompt-as-scenario (`--env.persona`); the assistant plays the same task through a masked chat session (`mask_prompt`) — the prompt is hidden from its harness while the task's own rewards and judges still score the real row. The substrate for tau-bench-style evals. |
+
+## User simulation: the user is just another agent
+
+There is exactly one exchange mechanism: the chat session (`agents.<name>.chat(task)`)
+— whoever calls `turn()` is the run's user, and each turn runs one harness segment
+(the program runs until it yields, the caller answers its final message, the next
+segment resumes the exchange with the answer). A prompt-less task is opened by the
+first `turn(message)`; a prompted task speaks first (take its opening reply with a
+bare `turn()`). Who computes the turns is the env's control flow, not framework
+machinery:
+
+```python
+class SortEnv(vf.SingleAgentEnv):
+    async def rollout(self, task, agents):
+        # a pre-scripted episode: the task is prompt-less, so the first turn opens
+        async with agents.agent.chat(task) as session:
+            for prompt in task.data.info["user_turns"]:
+                if (await session.turn(prompt)).stopped:
+                    break                        # a limit or @stop ended the run
+```
+
+A *scripted* user is a plain loop like this (a game engine stepping in-process works
+the same way — see the bundled `textarena` taskset). A *modeled* user is another
+agent: open both sessions and relay their `turn()`s into each other — see the bundled
+`user-sim` env, `environments/kuhn_poker_v1` (the same relay as self-play: both seats
+chat sessions of the run's own model), and [chat() in the Agent docs](agent.md). The
+user runs in the eval process, so there is nothing to declare, place, or serve.

docs/v1/tasksets.md

@@ -21,12 +21,10 @@ The command also supports:
 - `-p`, `--path <dir>` — parent directory, default: `./environments`
 - `-T`, `--add-tool` — also scaffold a `vf.Toolset` tool server at `servers/tool.py`
   - Use this to create custom tools which are installed into supported harnesses via MCP.
-- `-U`, `--add-user` — also scaffold a `vf.User` simulator at `servers/user.py`
-  - Use this to simulate a user interacting with the model. Not all harnesses support user simulation.
 - `-H`, `--add-harness` — also scaffold a custom `vf.Harness` at `harness.py`, selectable via `--env.agent.harness.id <name>`
   - Prefer a built-in harness unless the model needs to run inside a custom program.
 
-Most tasksets do not need specific tools, user simulations or custom harnesses.
+Most tasksets do not need specific tools or custom harnesses. (To simulate a user interacting with the model, open a chat session from an env's `rollout()` and script the user's turns — see the [Agent docs](agent.md).)
 
 > For a production-scale catalog of tasksets, see the companion [`research-environments`](https://github.com/PrimeIntellect-ai/research-environments) repository.