Add pi_env: Pi coding-agent environment (HF sandbox + transparent-proxy logprobs) - #999
Conversation
…roxy PiSessionFactory/PiSession/PiConfig mirror the OpenCode primitive: install the pi CLI (Node bootstrap + npm), write models.json pointing at an OpenAI-compatible endpoint, run pi --print --mode json headless, and (transparent_proxy mode) capture per-token logprobs via the shared interception proxy for GRPO. The sandbox backend (HFSandboxBackend) and interception proxy are imported from opencode_env for now; the plan is to consolidate both into openenv.core in a follow-up. Includes builder + factory-lifecycle unit tests.
Mirrors opencode_env's second layer so pi_env is both an in-process harness primitive and a deployable OpenEnv environment: PiEnv MCP client + FastAPI server (PiEnvironment with the run_rollout tool, task catalog, Gradio UI) + Dockerfile + openenv.yaml. HF sandbox (image, HF_TOKEN) replaces E2B (template, E2B_API_KEY); in-sandbox paths are /root; the sandbox backend and interception proxy are imported from opencode_env. Adds uv.lock and the pi-env server-image entry to docker-build.yml.
sync_env_docs.py --fix generates the docs/source/environments/pi.md stub from the README; the HTML catalog card in environments.md and the _toctree.yml nav entry are added manually (the script does not manage those). check-env-docs now passes.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Pull request overview
Adds a new pi_env environment (patterned after opencode_env) to run the Pi coding agent inside a Hugging Face sandbox, with an optional transparent-proxy mode that captures token-level logprobs for GRPO-style training. The PR also extends opencode_env with a Hugging Face sandbox backend and updates related runtime/path handling, plus docs + CI docker builds.
Changes:
- Introduces
envs/pi_env(harness primitive + FastAPI/MCP server + Gradio UI + Docker build). - Adds
HFSandboxBackendtoopencode_envand updates proxy/opencode path helpers to respectsandbox_home. - Adds/updates tests, docs navigation, and docker-build workflow entries.
Reviewed changes
Copilot reviewed 35 out of 37 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/envs/test_pi_runtime.py | Unit tests for pi_env runtime builders (commands/paths/models.json). |
| tests/envs/test_pi_factory_lifecycle.py | Ensures PiSessionFactory.create() tears down sandboxes on post-provision failure. |
| tests/envs/test_opencode_sandbox_home.py | Regression tests for sandbox_home-derived path helpers in opencode_env. |
| tests/envs/test_opencode_hf_sandbox.py | Unit tests for HF sandbox background-job polling semantics. |
| tests/envs/test_opencode_factory_lifecycle.py | Ensures OpenCodeSessionFactory.create() tears down sandboxes on post-provision failure. |
| envs/pi_env/task.py | Adds PiTask payload model and coercion helper. |
| envs/pi_env/server/pi_environment.py | Implements MCP environment with run_rollout tool orchestrating a full rollout. |
| envs/pi_env/server/gradio_ui.py | Adds /web Gradio UI for running rollouts and inspecting results/logprobs. |
| envs/pi_env/server/Dockerfile | Multi-stage build for deployable pi_env server image; installs opencode_env for shared sandbox/proxy. |
| envs/pi_env/server/catalog.py | Endpoint shorthand resolution (vllm/openai/hf_router) via env vars + defaults. |
| envs/pi_env/server/app.py | FastAPI app wiring + Gradio mount under /web. |
| envs/pi_env/server/init.py | Declares the server package. |
| envs/pi_env/README.md | End-user documentation for harness + server usage. |
| envs/pi_env/pyproject.toml | Defines openenv-pi-env package deps and server entrypoint. |
| envs/pi_env/pi_runtime.py | Pure builders for Pi install/run commands and sandbox path helpers. |
| envs/pi_env/openenv.yaml | OpenEnv spec for deploying pi_env as a Space. |
| envs/pi_env/models.py | Pydantic models for server results/state (RolloutResult, RolloutTurn, etc.). |
| envs/pi_env/harness.py | PiSessionFactory/PiSession harness implementation, incl. transparent proxy startup. |
| envs/pi_env/config.py | PiConfig configuration model for harness runs. |
| envs/pi_env/client.py | Typed PiEnv MCP client wrapper for run_rollout. |
| envs/pi_env/.gitignore | Env-local ignores for dev artifacts. |
| envs/pi_env/.dockerignore | Build-context exclusions for server image. |
| envs/pi_env/init.py | Public re-exports for client/harness/models + shared sandbox backend. |
| envs/opencode_env/uv.lock | Lockfile updates for dependency set (incl. newer huggingface_hub). |
| envs/opencode_env/sandbox/hf.py | Adds HF sandbox backend implementation (SDK Sandbox + polling wait()). |
| envs/opencode_env/sandbox/hf_image/Dockerfile | Adds a pre-baked image for faster HF-sandbox rollouts. |
| envs/opencode_env/sandbox/init.py | Exposes HFSandboxBackend (with stubs when hub version is too old). |
| envs/opencode_env/README.md | Documents switching OpenCode harness from E2B to HF sandboxes. |
| envs/opencode_env/pyproject.toml | Adds huggingface_hub>=1.22 dependency. |
| envs/opencode_env/opencode_runtime.py | Adds sandbox_home-aware helpers for proxy/opencode paths. |
| envs/opencode_env/harness.py | Uses new path helpers; ensures sandbox teardown on post-provision failures. |
| docs/source/environments/pi.md | Adds docs page for pi_env. |
| docs/source/environments/opencode.md | Adds HF sandbox backend docs for opencode_env. |
| docs/source/environments.md | Adds Pi environment card to catalog page. |
| docs/source/_toctree.yml | Adds pi_env docs page to navigation. |
| .github/workflows/docker-build.yml | Adds docker-build entries for opencode-sandbox and pi-env. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Pi harness primitive — sandbox + proxy + agent driver | ||
| "httpx>=0.27.0", | ||
| # >=1.22 ships huggingface_hub.Sandbox (HFSandboxBackend); core only floors 0.20. | ||
| "huggingface_hub>=1.22", | ||
| # Shared sandbox backend + interception proxy live in opencode_env for now | ||
| # (plan: consolidate into openenv.core). Kept a peer dep — install from the | ||
| # same repo (the server Dockerfile installs it explicitly): | ||
| # pip install "openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env" | ||
| ] |
There was a problem hiding this comment.
Intentional for now: pi_env reuses the sandbox backend + interception proxy from opencode_env instead of duplicating them (this PR is stacked on #998). The plan is to consolidate both into openenv.core (tracked with #940 / the harness RFC), at which point the peer dependency goes away. Documented in the README and pyproject.toml.
| setup: Bash commands run sequentially **before** the agent starts. | ||
| Each command runs in the sandbox; non-zero exit aborts setup. | ||
| verify: Bash commands run sequentially **after** the agent exits. |
There was a problem hiding this comment.
The docs were reworded in 7798e3ce: setup runs "at rollout start", not strictly before the agent. It races the agent for ~1-2s, which is fine in practice because Pi takes >=20s to make its first model call. Kept consistent with opencode_env; a strict before/after ordering is a shared follow-up for both envs.
| # NEVER ship secrets into the image. The container reads E2B_API_KEY etc. | ||
| # from runtime env vars (HF Space secrets, ``docker run -e``). |
There was a problem hiding this comment.
Fixed in 271f94a6 — the comment now references HF_TOKEN.
| class HFSandboxBackend: | ||
| """Creates Hugging Face sandboxes for OpenCode rollouts. | ||
|
|
||
| ``image`` must ship the ``opencode`` CLI, node, and the in-sandbox proxy. | ||
| """ |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 37 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (2)
envs/pi_env/pyproject.toml:37
pyproject.tomldescribesopencode_envas a peer dependency, butpi_envimports it at module import time (e.g.pi_env/__init__.pyandpi_env/harness.py). As a result,pip install openenv-pi-envwill fail at import-time unlessopenenv-opencode-envis also installed. Either declare the dependency explicitly (hard dep or extra) or make these imports optional and raise a clear error only when sandbox/proxy features are used.
# Shared sandbox backend + interception proxy live in opencode_env for now
# (plan: consolidate into openenv.core). Kept a peer dep — install from the
# same repo (the server Dockerfile installs it explicitly):
# pip install "openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env"
]
envs/opencode_env/sandbox/hf.py:126
- The
HFSandboxBackenddocstring says the image "must ship the opencode CLI, node, and the in-sandbox proxy", but elsewhere (and in this PR) the harness supports cold-installing these dependencies on a plainpython:3.12image. This docstring is misleading and may cause unnecessary coupling to pre-baked images.
"""Creates Hugging Face sandboxes for OpenCode rollouts.
``image`` must ship the ``opencode`` CLI, node, and the in-sandbox proxy.
"""
| f"mkdir -p {home}/.pi/agent {home}/logs/agent {home}/logs/verifier {home}/task {home}/workdir && " | ||
| # Bootstrap Node 22 when the image ships none new enough (pi needs >=22.19). | ||
| 'if ! command -v node >/dev/null 2>&1 || ' | ||
| '! node -e "process.exit(+process.versions.node.split(\'.\')[0]>=22?0:1)"; then ' |
There was a problem hiding this comment.
Fixed in 271f94a6 — the guard now checks major+minor (>=22.19), not just the major version.
| # Run setup commands one at a time, *before* the agent starts. | ||
| # The factory has already started the agent in start_agent() | ||
| # during create(); to keep the order "setup → agent → verify" | ||
| # we'd need to restructure. As a pragmatic compromise we run | ||
| # setup IMMEDIATELY after create(), which races with the agent | ||
| # for ~1-2s but is fine for typical pip/git/download work | ||
| # because pi itself takes >=20s to make its first model call. |
There was a problem hiding this comment.
The docs were reworded in 7798e3ce: setup runs "at rollout start", not strictly before the agent. It races the agent for ~1-2s, which is fine in practice because Pi takes >=20s to make its first model call. Kept consistent with opencode_env; a strict before/after ordering is a shared follow-up for both envs.
| # The shared sandbox backend + interception proxy live in opencode_env (kept a | ||
| # peer dep in pyproject, not a hard dep, so local dev / training installs stay | ||
| # lean). Install it explicitly into the venv so the server can import | ||
| # ``opencode_env.sandbox``. TEMPORARY: repoint to the branch carrying | ||
| # HFSandboxBackend until it lands in main; removed once the shared sandbox | ||
| # consolidates into openenv.core. | ||
| RUN --mount=type=cache,target=/root/.cache/uv \ | ||
| uv pip install --python .venv/bin/python \ | ||
| "openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env" |
There was a problem hiding this comment.
Known and temporary: it points at main, which resolves once #998 merges. It disappears when the shared sandbox moves to openenv.core. Noted in the PR description.
| from pi_env import ( | ||
| HFSandboxBackend, | ||
| PiConfig, | ||
| PiSessionFactory, | ||
| PiTask, | ||
| ) |
There was a problem hiding this comment.
Fixed in 7798e3ce — the server now imports the primitive from its defining modules, not the top-level package (which re-exports the client), per the server/client-separation invariant.
…, pipefail - README: correct the HF sandbox docs URL, drop redundant/circular links - .dockerignore: HF_TOKEN, not E2B_API_KEY - pi_runtime: gate Node >=22.19 (not just major 22); use the full model id and select by --provider so a slashed id isn't split; set -o pipefail so the piped tee doesn't mask Pi's exit code
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 37 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (3)
envs/pi_env/server/pi_environment.py:337
setupis documented/treated as running before the agent, but the current flow callsfactory.create()(which starts Pi) and only then runs the setup commands. That can race with the agent and violates the advertised(instruction, setup, verify)ordering (README + tool signature comments).
# Run setup commands one at a time, *before* the agent starts.
# The factory has already started the agent in start_agent()
# during create(); to keep the order "setup → agent → verify"
# we'd need to restructure. As a pragmatic compromise we run
# setup IMMEDIATELY after create(), which races with the agent
envs/opencode_env/sandbox/hf.py:126
HFSandboxBackenddocstring says the image must ship opencode + proxy, but the harness supports cold-install (and docs/README explicitly recommendimage="python:3.12"+ runtime install). This is misleading for users choosing between cold-install vs the prebaked image.
class HFSandboxBackend:
"""Creates Hugging Face sandboxes for OpenCode rollouts.
``image`` must ship the ``opencode`` CLI, node, and the in-sandbox proxy.
"""
envs/pi_env/server/Dockerfile:59
- The Docker build installs
openenv-opencode-envfrom the movingmainbranch with no ref/SHA pin. That makes builds non-reproducible and can break unexpectedly whenmainchanges (especially important for HF Spaces rebuilds).
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install --python .venv/bin/python \
"openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env"
| from opencode_env.sandbox import ( | ||
| HFSandboxBackend, | ||
| SandboxBackend, | ||
| SandboxHandle, | ||
| ) | ||
|
|
||
| from .client import PiEnv | ||
| from .config import PiConfig | ||
| from .harness import PiSession, PiSessionFactory |
There was a problem hiding this comment.
Intentional for now: pi_env reuses the sandbox backend + interception proxy from opencode_env instead of duplicating them (this PR is stacked on #998). The plan is to consolidate both into openenv.core (tracked with #940 / the harness RFC), at which point the peer dependency goes away. Documented in the README and pyproject.toml.
…docs Import the primitive from its defining modules instead of the top-level pi_env package (which re-exports the client), per the server/client-separation invariant. Reword the setup docs: it runs at rollout start, not strictly before the agent.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 37 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
envs/pi_env/pyproject.toml:37
pi_envimports and re-exportsopencode_env.sandbox(andpi_env.harnessimportsopencode_env) butopenenv-opencode-envis not declared as an install dependency. As written,pip install openenv-pi-envwill fail at import/runtime unless users manually installopenenv-opencode-env, which contradicts the current “peer dep” comment. Consider making the dependency explicit (or making the import truly optional with stubs).
# Pi harness primitive — sandbox + proxy + agent driver
"httpx>=0.27.0",
# >=1.22 ships huggingface_hub.Sandbox (HFSandboxBackend); core only floors 0.20.
"huggingface_hub>=1.22",
# Shared sandbox backend + interception proxy live in opencode_env for now
# (plan: consolidate into openenv.core). Kept a peer dep — install from the
# same repo (the server Dockerfile installs it explicitly):
# pip install "openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env"
]
envs/pi_env/server/Dockerfile:58
- The Docker build installs
openenv-opencode-envfrom the GitHub default branch with no ref/sha pin, which makes builds non-reproducible and makes stacked-PR validation harder (it will silently pull whatevermainis at build time). It would be safer to add an ARG so CI / developers can pin a branch, tag, or commit SHA when needed.
# The shared sandbox backend + interception proxy live in opencode_env (kept a
# peer dep in pyproject, not a hard dep, so local dev / training installs stay
# lean). Install it explicitly into the venv so the server can import
# ``opencode_env.sandbox``. TEMPORARY: repoint to the branch carrying
# HFSandboxBackend until it lands in main; removed once the shared sandbox
# consolidates into openenv.core.
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install --python .venv/bin/python \
"openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env"
envs/opencode_env/sandbox/hf.py:126
HFSandboxBackendis now reused bypi_envas a generic HF sandbox backend, and the harnesses can also cold-install required binaries. The docstring currently reads as OpenCode-specific (“image must ship opencode CLI, node, and proxy”), which is misleading for other consumers and for the cold-install path.
class HFSandboxBackend:
"""Creates Hugging Face sandboxes for OpenCode rollouts.
``image`` must ship the ``opencode`` CLI, node, and the in-sandbox proxy.
"""
…dead field - harness/server: size the sandbox lifetime and MCP-tool timeout to the full Node+Pi cold install, not just the agent - README: note the opencode_env install prerequisite for the primitive path; correct the server Dockerfile comment - config: drop unused request_timeout_ms - client: document the endpoint shorthand
| if proc is None: | ||
| return 0 | ||
| if not proc.running: | ||
| return int(proc.exit_code) if proc.exit_code is not None else 0 |
There was a problem hiding this comment.
Reaped process reports exit zero
Medium Severity
HFBgJob.wait treats a missing pid in processes() as exit code 0. If the agent exits non-zero and is reaped between polls, the harness reports success, so agent_exit_code and any logic that depends on it become wrong.
Reviewed by Cursor Bugbot for commit 82d3c19. Configure here.
There was a problem hiding this comment.
This is opencode_env's hf.py (part of #998), shown here because #999 is stacked on it. Verified against a real sandbox that finished processes stay listed, so a vanished pid is a torn-down sandbox — fixed in #998 (acf3143a): wait() now raises instead of returning 0. Propagates to #999 on rebase once #998 lands.
| self._exec_with_retry( | ||
| sandbox, | ||
| "pip install --quiet 'fastapi>=0.104' 'uvicorn[standard]>=0.24' " | ||
| "'httpx>=0.27' 2>&1 | tail -20", | ||
| timeout=180, | ||
| attempts=3, | ||
| backoff_s=2.0, | ||
| label="proxy deps install", | ||
| ) |
There was a problem hiding this comment.
Fixed in e1e3c27f — set -o pipefail on the proxy-deps install so a failed pip isn't masked by tail.
| def _command_rows(items: list[dict[str, Any]]) -> list[list[str]]: | ||
| rows: list[list[str]] = [] | ||
| for it in items or []: | ||
| cmd = it.get("cmd", "") | ||
| rows.append( | ||
| [ | ||
| cmd if len(cmd) <= 80 else cmd[:77] + "...", | ||
| str(it.get("exit_code", "")), | ||
| f"{it.get('duration_s', 0):.2f}s", | ||
| (it.get("stderr") or "").splitlines()[-1][:80] if it.get("exit_code") else "", | ||
| ] | ||
| ) | ||
| return rows |
There was a problem hiding this comment.
Fixed in e1e3c27f — the UI now uses next(reversed(...), "") so an empty stderr split no longer raises IndexError.
| if not proxy_already_present: | ||
| # Install proxy deps (idempotent on retries). |
There was a problem hiding this comment.
Fixed in e1e3c27f — set -o pipefail on the proxy-deps install so a failed pip isn't masked by tail.
| if not (base_url and api_key and model): | ||
| raise ValueError( | ||
| "must provide either ``endpoint`` (one of " | ||
| f"{ENDPOINT_KINDS}) or all of base_url + api_key + model" | ||
| ) |
There was a problem hiding this comment.
Fixed in e1e3c27f — the message now renders ', '.join(ENDPOINT_KINDS), so no duplicated paren.
Mirror opencode_env's pre-baked image: bake Node + Pi + proxy deps so rollouts skip the cold install (the proxy source is uploaded at runtime since it lives in opencode_env). Also drop the stale '(default)' on transparent_proxy in the harness docstring (the factory default is black_box).
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 36 out of 38 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
envs/pi_env/server/gradio_ui.py:148
_command_rowscan raiseIndexErrorwhen a command fails with an emptystderr(because.splitlines()[-1]is applied to an empty list). This would break the Gradio UI rendering for some failures (e.g., SIGKILL/network blips that yield no stderr).
def _command_rows(items: list[dict[str, Any]]) -> list[list[str]]:
rows: list[list[str]] = []
for it in items or []:
cmd = it.get("cmd", "")
rows.append(
[
cmd if len(cmd) <= 80 else cmd[:77] + "...",
str(it.get("exit_code", "")),
f"{it.get('duration_s', 0):.2f}s",
(it.get("stderr") or "").splitlines()[-1][:80] if it.get("exit_code") else "",
]
envs/opencode_env/sandbox/hf.py:126
- The
HFSandboxBackenddocstring says the sandboximagemust already include opencode + node + proxy, but the harness code/docs also support cold-installing these at runtime (e.g.image="python:3.12"). This docstring is misleading for users choosing an image.
class HFSandboxBackend:
"""Creates Hugging Face sandboxes for OpenCode rollouts.
``image`` must ship the ``opencode`` CLI, node, and the in-sandbox proxy.
"""
| # Inside-sandbox paths the server writes/reads (HF sandbox execs as root). | ||
| HOME = "/root" | ||
| WORKDIR = f"{HOME}/workdir" | ||
| INSTRUCTION_PATH = f"{HOME}/task/instruction.md" | ||
| REWARD_FILE = f"{HOME}/logs/verifier/reward.txt" | ||
| PROXY_LOG = f"{HOME}/logs/agent/proxy.log" | ||
| AGENT_LOG = f"{HOME}/logs/agent/pi.jsonl" | ||
| VERIFY_TIMEOUT_S = 120 |
There was a problem hiding this comment.
Fixed in 8bb0c3b8 (and 1c466584 for opencode) — setup commands now get SETUP_TIMEOUT_S=300 while verify keeps 120, so pip installs / downloads in setup aren't cut off.
|
Three usability blockers:
|
Addresses review: run_rollout ran setup after create() had already started the agent, so setup raced the agent over the workspace. create() now takes start_agent (default True, so the loop-owning training path is unchanged), the server passes start_agent=False, runs setup, then calls session.start_agent(), giving the correct setup then agent then verify order. A non-zero agent exit is now treated like a timeout, so a crashed agent no longer runs verify or earns a reward. AI-assisted.
e59b3b9 to
01f9dca
Compare
| except ImportError: # pragma: no cover | ||
| from config import PiConfig # type: ignore | ||
| from harness import PiSessionFactory # type: ignore | ||
| from task import PiTask # type: ignore |
There was a problem hiding this comment.
Docker import fallback breaks harness
High Severity
The Docker entrypoint runs uvicorn server.app:app with PYTHONPATH=/app/env, so server loads as a top-level package and the from ..harness import fails. The fallback then does from harness import …, but harness.py itself uses relative imports (from .config, from .pi_runtime, …), which raise ImportError: attempted relative import with no known parent package. That crashes on the first PiEnvironment() construction (/reset, run_rollout, Gradio Run), while /health still looks fine. The installed pi_env package in the venv would import cleanly if the fallback used pi_env.harness (or the CMD used pi_env.server.app:app).
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 01f9dca. Configure here.
that's the shared sandbox layer (backend + interception proxy) that
fixed.
fixed. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 4 total unresolved issues (including 3 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e96444c. Configure here.
| # (plan: consolidate into openenv.core). Kept a peer dep — install from the | ||
| # same repo (the server Dockerfile installs it explicitly): | ||
| # pip install "openenv-opencode-env @ git+https://github.com/huggingface/OpenEnv.git#subdirectory=envs/opencode_env" | ||
| ] |
There was a problem hiding this comment.
Undeclared opencode_env peer dependency
Medium Severity
pi_env eagerly imports opencode_env from __init__.py and harness.py, but openenv-opencode-env is only mentioned in a comment inside dependencies, not declared. A clean pip install of openenv-pi-env therefore fails on from pi_env import …, including the HTTP client path that does not need the sandbox backend.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit e96444c. Configure here.
- examples/pi_env_simple.py: inference demo hitting the deployed Space - docs tutorial pi-agent-grpo.md: black-box AsyncGRPO training recipe, links to TRL
Add
pi_env: Pi coding-agent environment (HF sandbox + transparent-proxy logprobs)What this adds
A new environment,
pi_env, that runs the Pi coding agent as an OpenEnv environment. It mirrorsopencode_envone-to-one and ships the same two layers:PiSessionFactory/PiSession/PiConfig/PiTask. Drives one Pi rollout inside a sandbox in-process (installs thepiCLI, writesmodels.jsonpointing at any OpenAI-compatible endpoint, runspi --print --mode jsonheadless). This is what a loop-owning trainer (e.g. TRLAsyncGRPOTrainer+HarnessRolloutWorker) calls directly.PiEnv(MCP client) + FastAPI server (PiEnvironmentwith a singlerun_rollouttool, task catalog, Gradio UI at/web) +Dockerfile, sopi_envcan also be deployed as an HF Space and driven over HTTP, like every other env in the repo.In
transparent_proxymode an in-sandbox proxy fronts the LLM endpoint, injectslogprobs=true, and captures per-turn(completion_token_ids, per_token_logps)toproxy_trace.jsonlfor GRPO.Pi uses its builtin tools (
read/bash/edit/write/…), so it is a self-contained coding agent — no external tool bridge. The default sandbox backend is the Hugging Face sandbox (HFSandboxBackend,huggingface_hub.Sandbox); the agent cold-installs Node + Pi per rollout on apython:3.12image.Relationship to
opencode_env/ #998 (stacked)pi_envis deliberately a thin sibling ofopencode_env: it imports the sandbox backend and the interception proxy fromopencode_envrather than duplicating them (from opencode_env.sandbox import HFSandboxBackend, …, and the proxy source is uploaded from theopencode_envpackage). Concretely it reusesHFSandboxBackend+interception.py, both introduced in #998.HFSandboxBackendPR). This branch is stacked on it; it should be reviewed after / merged after Add HFSandboxBackend for the OpenCode harness #998 so the diff is Pi-only.opencode_envcross-dependency is temporary. The intended end state is to consolidate the shared sandbox backend + proxy intoopenenv.core(coordinated with the harness-consolidation work in Agentic RL with black-box harnesses in OpenEnv via LLM-call interception #940 / the RFC). At that point bothopencode_envandpi_envimport from core and the peer dependency disappears.Validation
The harness primitive is validated end-to-end; the HTTP layer is validated at the wiring level.
tests/envs/test_pi_runtime.py,test_pi_factory_lifecycle.py)interception.py→ fake OpenAI server, streaming)npm install+ black-box run + Mode-B proxy trace)/health,/schema,/reset)trainer exit=0,TRAIN_LOOP_DONETraining metrics (real gradient steps, non-zero reward — Pi actually solved MBPP in the remote sandbox):
Full self-contained training repro (vLLM + NCCL + cloudflared tunnel +
PiSessionFactory):https://gist.github.com/sergiopaniego/ed71514ff1ae53cf522b23c0b890d939
TRL note (not part of this PR)
The loop-owning training path is agent-agnostic and needed no reconciler changes for Pi. One latent TRL bug surfaced:
print_prompt_completions_sample'sformat_entryassumesstrmessage content, but Pi emits structured content ([{"type":"text",…}]), tripping arichTypeError. Worked around in the repro (disablelog_completions+ a one-line coerce patch); to be fixed upstream in TRL. The known empty-choicesguard (huggingface/trl#6420) is also patched in the repro.Caveats
server/Dockerfileinstallsopenenv-opencode-envfor the shared sandbox; it resolves once Add HFSandboxBackend for the OpenCode harness #998 is onmain. A localopenenv buildon this branch needs the ref repointed to the branch carryingHFSandboxBackend.hf jobs cancel(TRL async-worker teardown behavior, notpi_env).sandbox/hf_image.Docs
Adds
docs/source/environments/pi.md(viasync_env_docs.py --fix), a catalog card indocs/source/environments.md, and a_toctree.ymlnav entry.check-env-docspasses.Testing
AI-assisted PR.
Note
Medium Risk
Large new surface that provisions remote sandboxes, runs user-supplied bash setup/verify, and forwards LLM traffic with API keys; risk is mitigated by mirroring the proven
opencode_envpattern but it still depends on a cross-packageopencode_envinstall until shared code moves to core.Overview
Introduces
pi_env, a new OpenEnv environment that runs the Pi coding agent in a Hugging Face sandbox (default) with the same two-layer shape asopencode_env: in-processPiSessionFactory/PiSessionand a deployable FastAPI/MCP server with a singlerun_rollouttool,PiEnvclient, and Gradio UI at/web.Rollouts use the shared
(instruction, setup, verify)task contract; reward is verify pass rate unless/root/logs/verifier/reward.txtoverrides.transparent_proxymode reusesopencode_env’sHFSandboxBackendand uploadsinterception.pyinto the sandbox to capture per-turn token ids and logprobs for GRPO;black_boxskips the proxy.CI gains
pi-envandpi-sandboximage builds (hf_image/Dockerfilepre-bakes Node 22 + Pi CLI + proxy deps). Docs add the Pi environment page, catalog entry, and a Pi + TRL AsyncGRPO tutorial.Reviewed by Cursor Bugbot for commit 7193ad3. Bugbot is set up for automated code reviews on this repo. Configure here.