feat(v1): task-authored interception of model exchanges (@vf.intercept)

[xeophon]

Summary

Adds task-authored interception of every model exchange to v1: an @vf.intercept decorator (mirroring @vf.stop/@vf.reward) that can block, rewrite, strip, or terminate exchanges — for all harnesses at once, at the one choke point all model traffic funnels through (the interception server).

• Inbound (harness → model): tool results, provider-tool definitions, and provider-tool history items can be stripped or rewritten before the model sees them.
• Outbound (model → harness): tool calls can be blocked before the harness executes them — the handler returns a vf.Message whose text answers the model instead, so it can do something else. Provider-side tool output (Codex/Claude web search) can be stripped from the response. Streaming responses are buffered when handlers are registered, then re-emitted (byte-identical when untouched).
• Terminate: vf.Terminate(reason=..., reward=-1.0) ends the rollout on the spot and records the reward — direct punishment for reward hacking.
• Everything is recorded on the trace (trace.interceptions); in-place mutations of exchange.raw are auto-detected and logged, so handlers only ever return a message, a Terminate, or nothing.

@vf.intercept
async def no_git(self, exchange):
    for call in (exchange.message.tool_calls if exchange.message else []) or []:
        if match_tool(call.name, "bash") and "git " in call.arguments:
            return vf.UserMessage(content="git is not allowed in this environment")

Prebuilt helpers (verifiers/v1/intercepts/, importable like judges)

• match_tool / match_tool_calls — fuzzy tool-name matching, so one pattern (e.g. web_search) covers Claude's web_search_20250305 and Codex's web_search_preview
• judge_tools(exchange, judge=...) -> bool — an LLM judge (ToolCallJudge, or any custom Judge subclass with its own prompt) vets each tool call; the handler decides what a rejection means
• strip_provider_tools(exchange, ...) -> list[str] — strips provider-side tools in both directions and records what was removed

Showcase environment

environments/intercept_v1/ demonstrates all four behaviors on a 4-row inline dataset: blocking git commands (the model gets an error and continues), a custom judge gating tool calls, provider-side web-search stripping, and terminating with reward −1 on reward-hacking attempts.

Verification

• uv run pytest tests/v1 tests/test_imports.py ... — 93 passed (e2e skips need API keys); ruff, pre-commit, and ty clean
• Temp-scripted end-to-end checks against a live InterceptionServer: blocked tool calls never reach the harness (HTTP + SSE, with passthrough controls), strips/terminates/replays behave, and streaming is byte-identical when no handlers are registered

---

Note

High Risk
Changes core interception-server request/response paths and streaming behavior for all harnesses; incorrect handling could block legitimate tool use or alter provider traffic silently.

Overview
Introduces task-authored interception at the single model-traffic choke point (the interception server), via a new @vf.intercept decorator parallel to @vf.stop / @vf.reward.

Handlers run on every exchange in priority order on both request (mutate wire JSON before upstream) and response (before the turn commits). They can return a Message to block tool calls and answer the model in text, Terminate to end the rollout (optional negative reward), or mutate exchange.raw in place (rewrites are auto-logged). Actions are stored on trace.interceptions.

The interception server wires this through RolloutSession.run_intercepts: request-side termination refuses the turn; response-side blocking drops tool calls via dialect-aware wire helpers. Streaming buffers the full SSE when handlers exist, then replays original chunks or dialect.stream_events after mutations. Parsers now attach response.raw for in-place edits.

Prebuilt helpers (match_tool, judge_tools, strip_provider_tools) cover fuzzy tool names, LLM judge gating, and stripping provider-side web search across chat / Anthropic / Responses shapes.

Adds intercept-v1 (four demo rows: terminate on answer-file reads, block git, judge gate, strip web search) and registers it in the repo examples / uv workspace.

^{Reviewed by Cursor Bugbot for commit 601abec. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add @vf.intercept decorator for task-authored interception of model exchanges

• Introduces a new @vf.intercept decorator in decorators.py that tasks can use to register handler functions with an optional priority, discovered at rollout time and passed into RolloutSession.
• RolloutSession.run_intercepts in session.py runs handlers in priority order on each request/response exchange; handlers can silently mutate the wire payload, return a Message to block a tool call, or return Terminate to end the rollout with an optional reward.
• Each dialect (anthropic.py, chat.py, responses.py) now stores the native response on Response.raw and implements stream_events to re-synthesize SSE from a mutated raw object for replay.
• The interception server in server.py buffers streaming responses when intercepts are registered, runs handlers after the full response is assembled, and either replays buffered events or synthesizes new SSE from mutated raw; non-streaming paths run intercepts before forwarding and after receiving.
• Helper utilities in verifiers/v1/intercepts/ provide tool name matching (match_tool, TOOL_SYNONYMS), provider tool stripping (strip_provider_tools), and an LLM-based tool-call gate (judge_tools/ToolCallJudge).
• A reference implementation in environments/intercept_v1/ demonstrates block, judge-gate, strip, and terminate intercept patterns.
• Risk: streaming responses are fully buffered when any intercept handler is registered, bypassing live relay to the client.

^{Macroscope summarized 601abec.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 601abece6a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve Responses terminal failure state

When a streamed Responses call is mutated by any response-side interceptor, this replay path always emits response.completed, even if the raw response being replayed has status == "incomplete" or "failed" (the parser above explicitly treats response.incomplete and response.failed as terminal events). In those intercepted streaming cases, clients such as Codex will see a successful completion event instead of the original truncation/failure signal, so they can continue as if the turn completed normally while the trace records a length/failed response.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Do not strip model-invoked Responses tools

For Responses output this matches on both type and name, so strip_provider_tools(exchange, "web_search") will also delete a normal model-invoked function_call item whose function is named web_search, not just provider-side items like web_search_call. In a task that exposes its own function/MCP tool with a matching name, the model's legitimate tool call is removed before the harness can execute it, and the turn is re-derived as if no tool call was made.

Useful? React with 👍 / 👎.