feat(v1): the exchange — chat()/turn(), segments, resume, one user mechanism

The turn half of the agent-programs work, rebased onto the restructured run
half (abstract Environment + SingleAgentEnv, one EnvConfig per env, agentic
judging, the session seal).

- Agent.chat(task) -> ChatSession: the caller IS the run's user; one turn()
  per harness segment (the program runs to its exit, the caller answers, the
  next segment resumes the exchange with the answer). mask_prompt hides a
  scenario prompt from the wire while the task still scores the real row.
  _RoleAgent.chat stamps roles/trainable at mint, so env-driven sessions land
  role-stamped exactly like plain runs.
- Harness.resume: relaunch on the accreted conversation by default
  (SUPPORTS_MESSAGE_PROMPT); codex overrides it with a native continuation.
- The user loop moves OUT of the model boundary: vf.User/Task.user (session
  injection, dialect extend, serve_user, UserError, the -U scaffold, the
  placement matrix) is deleted — chat sessions in the env's rollout() are the
  one exchange mechanism. The interception drive loop flattens to one turn
  per request, keeping per-call records and the concluded-trace seal.
- textarena becomes a recipe env stepping the real engine host-side;
  alphabet-sort/color-codeword script their users through SingleAgentEnv
  sessions; openenv drives its client from rollout() (no more served user);
  user-sim bundled env (two-session relay, the tau-style substrate, null
  harness for the modeled user) and kuhn_poker_v1 (self-play reference)
  added.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: hallerite

AGENTS.md

@@ -20,7 +20,7 @@ Use these rules when shaping public v1 APIs, configs, and environment files.
 - Expose as few knobs in the configs as possible, but as many as needed.
 - Use strict Pydantic models for structured config, tasks, messages, and state.
 - A basic taskset should fit in a few dozen idiomatic lines: typed data/task/config classes, `load()`, and decorated scoring on the task.
-- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__` or `User.__init__`; use `setup()`.
+- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__`; use `setup()`.
 - Refer to the code as the source of truth.
 - Avoid mutable module globals. Process-level locks/rate limiters and immutable constants are the narrow exceptions.
 

assets/agents/common_best_practices.md

@@ -16,6 +16,6 @@ Use these rules when shaping public v1 APIs, configs, and environment files.
 - Expose as few knobs in the configs as possible, but as many as needed.
 - Use strict Pydantic models for structured config, tasks, messages, and state.
 - A basic taskset should fit in a few dozen idiomatic lines: typed data/task/config classes, `load()`, and decorated scoring on the task.
-- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__` or `User.__init__`; use `setup()`.
+- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__`; use `setup()`.
 - Refer to the code as the source of truth.
 - Avoid mutable module globals. Process-level locks/rate limiters and immutable constants are the narrow exceptions.

assets/lab/AGENTS.md

@@ -23,7 +23,7 @@ Use these rules when shaping public v1 APIs, configs, and environment files.
 - Expose as few knobs in the configs as possible, but as many as needed.
 - Use strict Pydantic models for structured config, tasks, messages, and state.
 - A basic taskset should fit in a few dozen idiomatic lines: typed data/task/config classes, `load()`, and decorated scoring on the task.
-- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__` or `User.__init__`; use `setup()`.
+- Do not override `Taskset.__init__`; implement `load()`. Do not override `Harness.__init__`; use `setup()`.
 - Refer to the code as the source of truth.
 - Avoid mutable module globals. Process-level locks/rate limiters and immutable constants are the narrow exceptions.
 

assets/lab/environments/AGENTS.md

@@ -28,12 +28,10 @@ The command also supports:
 - `-p`, `--path <dir>` — parent directory, default: `./environments`
 - `-T`, `--add-tool` — also scaffold a `vf.Toolset` tool server at `servers/tool.py`
   - Use this to create custom tools which are installed into supported harnesses via MCP.
-- `-U`, `--add-user` — also scaffold a `vf.User` simulator at `servers/user.py`
-  - Use this to simulate a user interacting with the model. Not all harnesses support user simulation.
 - `-H`, `--add-harness` — also scaffold a custom `vf.Harness` at `harness.py`, selectable via `--env.agent.harness.id <name>`
   - Prefer a built-in harness unless the model needs to run inside a custom program.
 
-Most tasksets do not need specific tools, user simulations or custom harnesses.
+Most tasksets do not need specific tools or custom harnesses. (To simulate a user interacting with the model, open a chat session from an env's `rollout()` and script the user's turns — see the [Agent docs](https://github.com/PrimeIntellect-ai/verifiers/blob/main/docs/v1/agent.md).)
 
 > For a production-scale catalog of tasksets, see the companion [`research-environments`](https://github.com/PrimeIntellect-ai/research-environments) repository.
 
@@ -400,7 +398,7 @@ uv run eval my-task-v1 --env.id agentic-judge --env.judge.harness.runtime.type d
 exporting an `Environment` subclass via `__all__`, or a Hub `org/name[@version]` —
 and its `EnvConfig` surface typed on the CLI (`--env.<role>.*`, `-h` renders them).
 Empty (the default) keeps the taskset's own story: the env its package ships (a
-*recipe* env like `code_golf_v1`, where the interaction is
+*recipe* env like `code_golf_v1` or `kuhn_poker_v1`, where the interaction is
 intrinsic to the data), else `SingleAgentEnv`. An explicit id wins over a
 bundled recipe env.
 
@@ -410,3 +408,32 @@ Bundled envs (`verifiers/v1/envs/`):
 | --- | --- | --- |
 | `best-of-n` | `solver` | `--env.n` independent attempts per rollout; `score()` marks the argmax-reward sibling (`best`) and whether any reached `--env.threshold` (`pass_at_n`) — rejection sampling and pass@k. |
 | `agentic-judge` | `solver`, `judge` | agent-as-judge: the solver plays the task; a code-executing judge agent verifies the finished attempt with real execution, always in its own sandbox, never on the host. The verdict spec is a **judge plugin** (`--env.spec.id score\|rubric\|reference`, the same registry and format as `env.taskset.task.judges`) — write your grading criteria once; the parsed verdict + per-criterion metrics land on the solver's trace exactly as the plugged tier records them. The judge's verdict task mirrors the solver task's world (same image, a fresh box in its original state) with the graded transcript uploaded (`/tmp/transcript.md`/`.json`); the judge seat defaults to the taskset's default harness and must land in a container: pin `--env.judge.harness.runtime.type docker\|prime`, or construction refuses. A judgement that needs no execution belongs on the plugged tier, not on an agent. |
+| `user-sim` | `assistant`, `user` | a modeled user (an untrainable `user` seat, `null` harness by default) opens and drives the conversation from the task's prompt-as-scenario (`--env.persona`); the assistant plays the same task through a masked chat session (`mask_prompt`) — the prompt is hidden from its harness while the task's own rewards and judges still score the real row. The substrate for tau-bench-style evals. |
+
+### User simulation: the user is just another agent
+
+There is exactly one exchange mechanism: the chat session (`agents[...].chat(task)`)
+— whoever calls `turn()` is the run's user, and each turn runs one harness segment
+(the program runs until it yields, the caller answers its final message, the next
+segment resumes the exchange with the answer). A prompt-less task is opened by the
+first `turn(message)`; a prompted task speaks first (take its opening reply with a
+bare `turn()`). Who computes the turns is the env's control flow, not framework
+machinery:
+
+```python
+class SortEnv(vf.SingleAgentEnv):
+    async def rollout(self, task, agents):
+        # a pre-scripted episode: the task is prompt-less, so the first turn opens
+        async with agents["agent"].chat(task) as session:
+            for prompt in task.data.info["user_turns"]:
+                if (await session.turn(prompt)).stopped:
+                    break                        # a limit or @stop ended the run
+        return {"agent": session.trace}
+```
+
+A *scripted* user is a plain loop like this (a game engine stepping in-process works
+the same way — see the bundled `textarena` taskset). A *modeled* user is another agent
+role: open both sessions and relay their `turn()`s into each other — see the bundled
+`user-sim` env, `environments/kuhn_poker_v1` (the same relay as self-play: both seats
+chat sessions of the run's own model), and [chat() in the Agent docs](https://github.com/PrimeIntellect-ai/verifiers/blob/main/docs/v1/agent.md). The
+user runs in the eval process, so there is nothing to declare, place, or serve.

configs/textarena.toml

@@ -1,7 +1,7 @@
-# textarena — single-player TextArena games driven by a user simulator (vf.User).
+# textarena — single-player TextArena games, the engine playing the user.
 # Supported (--env.taskset.game): Wordle-v0, Wordle-v0-long, Hangman-v0, WordLadder-v0,
-# WordSearch-v0. The framework's interception server plays the game; the harness never sees
-# it. The user sim runs colocated in the harness's runtime.
+# WordSearch-v0. The env's rollout() steps the real engine host-side as the run's user;
+# the harness only ever sees the conversation.
 #
 #   uv run eval @ configs/textarena.toml --model <small-instruct-model>
 num_tasks = 5
@@ -14,5 +14,5 @@ max_turns = 15
 id = "textarena"
 game = "Wordle-v0"
 
-[env.agent.harness]
+[env.player.harness]
 id = "null"

configs/wordle.toml

@@ -1,4 +1,4 @@
-# wordle-v1 — the textarena taskset pinned to Wordle, driven by a user simulator.
+# wordle-v1 — the textarena taskset pinned to Wordle, the engine playing the user.
 #
 #   uv run eval @ configs/wordle.toml --model <small-instruct-model>
 num_tasks = 5
@@ -10,5 +10,5 @@ max_turns = 8
 [env.taskset]
 id = "wordle-v1"
 
-[env.agent.harness]
+[env.player.harness]
 id = "null"

docs/v1/agent.md

@@ -41,6 +41,54 @@ Everything beyond the arrow is a parameter, not a concept:
   program a box to place runs into. A different model is a different agent — construct
   another `Agent` (sharing the client, and the interception pool via `interception=`)
   rather than swapping contexts per run.
+- **the exchange** is `chat()`, the one multi-turn surface: whoever calls `turn()` is the
+  run's user, and each turn runs one harness SEGMENT — the program runs until it yields
+  (exits), the caller answers its final message, and the next segment resumes the
+  exchange with the answer (`Harness.resume`: a relaunch on the accreted conversation by
+  default; codex continues its own recorded session natively). The harness's own tool
+  loop runs entirely inside a segment, so tools and a simulated user compose freely.
+  Needs a harness that can resume (a Messages prompt, or a native `resume()` — default /
+  null / direct / codex).
+- **`chat(mask_prompt=True)`** says the task's `prompt` is the USER's side of the story —
+  a scenario the caller pursues, not the assistant's seed. The wire hides the prompt (the
+  caller opens instead), while the task's hooks, rewards, and judges still score the real
+  row. This is how the bundled `user-sim` env runs the assistant on the original task.
+
+## chat(): be the user yourself
+
+`agent.chat(task)` is the caller side of the same channel — a full run of the task where
+YOU (or another agent's program) supply each user turn live:
+
+```python
+async with agent.chat(task) as session:          # task has no prompt: chat opens it
+    reply = await session.turn("hello there")    # one user message -> one vf.Reply
+    if not reply.stopped:
+        followup = await session.turn(f"you said: {reply.text}")
+    print(session.trace.num_turns)               # the trace is live mid-exchange
+# leaving the context ends the exchange (user_closed) and finishes the run:
+# hooks and scoring included — session.trace is the final, scored trace
+```
+
+`turn()` returns a `vf.Reply` (`text`, `stopped`); a `stopped` reply means the run ended
+(a limit, a `@stop`, the harness finishing) instead of answering. And because both sides
+speak the same channel, "the user is just another agent" — a modeled user is a chat
+session relayed into another agent's run:
+
+```python
+async with (
+    user_agent.chat(user_task) as sim,             # the modeled user (its own trace!)
+    assistant.chat(task, mask_prompt=True) as helper,
+):
+    ask = await sim.turn("Hello! How can I help you today?")
+    while not ask.stopped:
+        reply = await helper.turn(ask.text)
+        if reply.stopped:
+            break
+        ask = await sim.turn(reply.text)
+```
+
+Modeled-user runs stay cheap with the tool-less `null` harness: a bare chat-loop
+program, no tools, no container — the rollout is essentially just the model calls.
 
 Interception follows the same borrowing story as runtimes: pass a live, already-entered
 `Interception` at construction (`vf.Agent(..., interception=pool)`) and several agents

docs/v1/environments.md

@@ -150,7 +150,7 @@ uv run eval my-task-v1 --env.id agentic-judge --env.judge.harness.runtime.type d
 exporting an `Environment` subclass via `__all__`, or a Hub `org/name[@version]` —
 and its `EnvConfig` surface typed on the CLI (`--env.<role>.*`, `-h` renders them).
 Empty (the default) keeps the taskset's own story: the env its package ships (a
-*recipe* env like `code_golf_v1`, where the interaction is
+*recipe* env like `code_golf_v1` or `kuhn_poker_v1`, where the interaction is
 intrinsic to the data), else `SingleAgentEnv`. An explicit id wins over a
 bundled recipe env.
 
@@ -160,3 +160,32 @@ Bundled envs (`verifiers/v1/envs/`):
 | --- | --- | --- |
 | `best-of-n` | `solver` | `--env.n` independent attempts per rollout; `score()` marks the argmax-reward sibling (`best`) and whether any reached `--env.threshold` (`pass_at_n`) — rejection sampling and pass@k. |
 | `agentic-judge` | `solver`, `judge` | agent-as-judge: the solver plays the task; a code-executing judge agent verifies the finished attempt with real execution, always in its own sandbox, never on the host. The verdict spec is a **judge plugin** (`--env.spec.id score\|rubric\|reference`, the same registry and format as `env.taskset.task.judges`) — write your grading criteria once; the parsed verdict + per-criterion metrics land on the solver's trace exactly as the plugged tier records them. The judge's verdict task mirrors the solver task's world (same image, a fresh box in its original state) with the graded transcript uploaded (`/tmp/transcript.md`/`.json`); the judge seat defaults to the taskset's default harness and must land in a container: pin `--env.judge.harness.runtime.type docker\|prime`, or construction refuses. A judgement that needs no execution belongs on the plugged tier, not on an agent. |
+| `user-sim` | `assistant`, `user` | a modeled user (an untrainable `user` seat, `null` harness by default) opens and drives the conversation from the task's prompt-as-scenario (`--env.persona`); the assistant plays the same task through a masked chat session (`mask_prompt`) — the prompt is hidden from its harness while the task's own rewards and judges still score the real row. The substrate for tau-bench-style evals. |
+
+## User simulation: the user is just another agent
+
+There is exactly one exchange mechanism: the chat session (`agents[...].chat(task)`)
+— whoever calls `turn()` is the run's user, and each turn runs one harness segment
+(the program runs until it yields, the caller answers its final message, the next
+segment resumes the exchange with the answer). A prompt-less task is opened by the
+first `turn(message)`; a prompted task speaks first (take its opening reply with a
+bare `turn()`). Who computes the turns is the env's control flow, not framework
+machinery:
+
+```python
+class SortEnv(vf.SingleAgentEnv):
+    async def rollout(self, task, agents):
+        # a pre-scripted episode: the task is prompt-less, so the first turn opens
+        async with agents["agent"].chat(task) as session:
+            for prompt in task.data.info["user_turns"]:
+                if (await session.turn(prompt)).stopped:
+                    break                        # a limit or @stop ended the run
+        return {"agent": session.trace}
+```
+
+A *scripted* user is a plain loop like this (a game engine stepping in-process works
+the same way — see the bundled `textarena` taskset). A *modeled* user is another agent
+role: open both sessions and relay their `turn()`s into each other — see the bundled
+`user-sim` env, `environments/kuhn_poker_v1` (the same relay as self-play: both seats
+chat sessions of the run's own model), and [chat() in the Agent docs](agent.md). The
+user runs in the eval process, so there is nothing to declare, place, or serve.

docs/v1/tasksets.md

@@ -21,12 +21,10 @@ The command also supports:
 - `-p`, `--path <dir>` — parent directory, default: `./environments`
 - `-T`, `--add-tool` — also scaffold a `vf.Toolset` tool server at `servers/tool.py`
   - Use this to create custom tools which are installed into supported harnesses via MCP.
-- `-U`, `--add-user` — also scaffold a `vf.User` simulator at `servers/user.py`
-  - Use this to simulate a user interacting with the model. Not all harnesses support user simulation.
 - `-H`, `--add-harness` — also scaffold a custom `vf.Harness` at `harness.py`, selectable via `--env.agent.harness.id <name>`
   - Prefer a built-in harness unless the model needs to run inside a custom program.
 
-Most tasksets do not need specific tools, user simulations or custom harnesses.
+Most tasksets do not need specific tools or custom harnesses. (To simulate a user interacting with the model, open a chat session from an env's `rollout()` and script the user's turns — see the [Agent docs](agent.md).)
 
 > For a production-scale catalog of tasksets, see the companion [`research-environments`](https://github.com/PrimeIntellect-ai/research-environments) repository.