feat(sandbox): per-environment network egress policy

CHANGELOG.md

@@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- Sandbox environments now automatically allow and authenticate their repository's git platform (GitLab/GitHub) for git operations over HTTPS in the sandbox. DAIV injects an `Authorization: Basic` credential at the egress proxy, built from a short-lived token — the same project-scoped ephemeral clone token for GitLab, and a `contents:write` installation token for GitHub — so `git fetch`/clone/push of the repository works without configuring a per-environment host rule. Because DAIV pushes from inside the sandbox, this applies even to **Network Off** environments when a push token exists: the environment is opened solely for its git platform (everything else stays blocked) so publishing always works, while token-less eval/benchmark runs stay fully network-isolated. The rule is applied at runtime (never stored on the environment) and takes precedence over a user-defined rule for the same host.
+- Added a per-environment **network egress policy** for sandbox environments, configurable from the environment form when network is enabled. Restrict outbound traffic to an allow-list of hosts (glob patterns), attach per-host HTTP credentials (header name + value, encrypted at rest and never rendered back), scope each host to **all methods** or **read-only** (`GET, HEAD, OPTIONS`), and set the `default` (deny/allow) reachability for unlisted hosts. The proxy always TLS-inspects every reachable host, so per-host credentials and method rules are always enforced. Credentials are injected by the daiv-sandbox egress proxy and never enter the sandbox container. The egress proxy is mandatory for network-enabled sessions: if the sandbox deployment has no egress proxy configured, a network-enabled run is rejected rather than falling back to unrestricted network.
 - Added `release_orphan_queued_threads` management command to recover `QUEUED` activities left behind on threads with no active sibling (rare TOCTOU loss).
 - Added per-domain auth headers for the `web_fetch` tool to the configuration UI under **Web Fetch → Per-domain auth headers**. Each row pairs a domain (exact match) with an HTTP header name and a header value (encrypted at rest). Replaces the previous env-only `AUTOMATION_WEB_FETCH_AUTH_HEADERS` setting; the new env override is `DAIV_WEB_FETCH_AUTH_HEADERS` (same JSON shape). Operators using the old name must rename it.
 - Added a "Start a run" page at `/dashboard/runs/new/` for launching new agent runs from the UI, and a "Retry" button on terminal non-webhook activities that pre-fills the form with the original prompt, repository, ref, and max-mode flag.
@@ -36,10 +38,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added support for custom global skills available across all repositories. Mount skill directories to `/home/daiv/data/skills/` (configurable via `DAIV_AGENT_CUSTOM_SKILLS_PATH`). Custom global skills follow the same format as built-in skills and can override them.
 - Added deferred tool loading for the agent, reducing the number of tools visible to the model at startup and loading additional tools on demand based on the task context.
 
+### Changed
+
+- **Upgrade note — existing network-enabled sandbox environments become network-isolated.** The legacy `network_enabled` flag is replaced by the per-environment egress policy (network is now derived from the presence of a policy). There is no automatic conversion: the migration leaves every previously network-enabled environment without a policy, so after upgrade those environments run network-isolated. The repository's own git platform stays reachable (DAIV injects it at runtime, so runs still clone and publish), but any other outbound access — package registries, external APIs — is blocked until an operator re-enables it by setting **Network** to **On** and defining an egress allow-list. This is deliberate: the new model has no unrestricted-network mode, and auto-granting allow-all would fail closed on deployments without the egress proxy CA configured.
+
 ### Fixed
 
 - Hardened the agent's workspace file tools (`read_file`/`write_file`/`edit_file`/`ls`/`glob`/`grep`) against sandbox transport faults: a transient failure — most notably the `409 "Session is busy"` the sandbox returns when batched tool calls contend on the one-at-a-time session — now degrades to a retryable tool-result error instead of aborting the whole run, matching the `bash` tool. The `bash` tool's own classifier now also treats `409` as transient (retry once) rather than permanent.
 - Hardened MCP tool loading so a single broken or slow MCP server can no longer freeze chats and runs. Each server's tools are now loaded independently with a per-server timeout (`MCP_TOOL_LOAD_TIMEOUT`, default 30s); a server that times out or errors is skipped instead of blocking the whole agent.
+- GitLab clones now self-heal from a stale cached clone token. The short-lived project-scoped clone token is cached for a day, but the underlying project access token can die sooner (revoked, or the project/instance reset); previously the dead token was served to — and failed — every clone of that project until the cache window closed. A clone rejected for authentication while using the ephemeral token now drops it and retries once with a freshly minted one (the PAT fallback is not retried, since re-minting cannot change that credential).
 - Enforced "at most one active (`READY`/`RUNNING`) API/MCP Activity per `thread_id`" at the DB layer via a partial unique constraint. Concurrent submissions on the same thread cleanly fall back to `QUEUED` instead of both running.
 - Made the FIFO dispatcher race-safe: an atomic compare-and-swap (`UPDATE filter(status=QUEUED) → READY`) prevents two terminal events on the same thread from double-promoting the same queued sibling. Dispatch failures now set `finished_at` and iterate via a loop instead of recursive signal re-entry. The loop bails after `MAX_CONSECUTIVE_DISPATCH_FAILURES` (3) consecutive failures so a transient broker outage does not mass-fail an entire QUEUED backlog — remaining rows stay QUEUED for `release_orphan_queued_threads`.
 - Guarded the post-enqueue `task_result_id` save in both `asubmit_batch_runs` and `dispatch_next_in_thread`: a DB blip after a successful broker enqueue now marks the row FAILED (and releases queued siblings) instead of stranding it in READY with no task linkage.

daiv/accounts/templates/accounts/_sidebar.html

@@ -1,7 +1,7 @@
 {% load i18n icon_tags nav_tags %}
 
 <aside data-testid="app-sidebar"
-       class="hidden w-60 shrink-0 overflow-y-auto border-r border-white/[0.06] bg-[#030712] sm:flex sm:flex-col"
+       class="shrink-0 overflow-y-auto bg-[#030712] {{ sidebar_root_class|default:'hidden w-60 border-r border-white/[0.06] sm:flex sm:flex-col' }}"
        x-cloak>
   <div class="flex items-center gap-2.5 px-5 pt-5 pb-5">
     <span class="inline-block h-6 w-6 rounded-md bg-gradient-to-br from-violet-500 to-indigo-500"></span>

daiv/accounts/templates/base_app.html

@@ -17,7 +17,7 @@
          x-transition:leave-start="translate-x-0"
          x-transition:leave-end="-translate-x-full"
          class="relative flex h-full w-[85%] max-w-[260px] flex-col bg-[#030712] shadow-2xl">
-      {% include "accounts/_sidebar.html" %}
+      {% include "accounts/_sidebar.html" with sidebar_root_class="flex w-full flex-col" %}
     </div>
   </div>
 

daiv/automation/agent/git_manager.py

@@ -10,6 +10,7 @@
 from git import GitCommandError
 
 from automation.agent.constants import REPO_PATH
+from core.utils import is_git_auth_error_text
 
 if TYPE_CHECKING:
     from git import Repo
@@ -366,31 +367,14 @@ class GitPushPermissionError(RuntimeError):
 class GitPushNetworkError(RuntimeError):
     """Raised when pushing fails because the remote host is unreachable.
 
-    Typically a sandbox-authoritative run whose sandbox environment is configured with
-    ``network_enabled=False``: git runs (and therefore pushes) from inside the sandbox, so the
-    sandbox must have network access for the push to reach ``origin``.
+    git runs (and therefore pushes) from inside the sandbox, so the sandbox must be able to reach
+    ``origin``'s git platform. DAIV opens that host automatically whenever a platform token can be
+    minted — even on a network-off env — so this typically means the egress proxy is unavailable, or
+    the run is one that legitimately has no platform token (e.g. an eval/benchmark run) and so stays
+    fully network-isolated.
     """
 
 
-def _is_push_auth_error_text(output: str) -> bool:
-    """
-    Check if git push output indicates an authentication or permission failure.
-    """
-    text = output.lower()
-    return any(
-        marker in text
-        for marker in (
-            "returned error: 403",
-            "authentication failed",
-            "permission denied",
-            "access denied",
-            "http basic: access denied",
-            "could not read username",
-            "not authorized",
-        )
-    )
-
-
 def _is_push_network_error_text(output: str) -> bool:
     """Check if git push output indicates the remote host was unreachable (network failure).
 
@@ -420,7 +404,7 @@ def _raise_for_push_failure(push_args: list[str], result: _GitResult) -> None:
     Auth/permission → ``GitPushPermissionError``; an unreachable host → ``GitPushNetworkError``;
     anything else → the raw ``GitCommandError``. Auth is checked first so it always wins.
     """
-    if _is_push_auth_error_text(result.output):
+    if is_git_auth_error_text(result.output):
         logger.warning("git push auth failure: %s", result.output)
         raise GitPushPermissionError(
             "Failed to push changes to the remote repository due to authentication or permission issues. "
@@ -432,7 +416,7 @@ def _raise_for_push_failure(push_args: list[str], result: _GitResult) -> None:
         logger.warning("git push network failure: %s", result.output)
         raise GitPushNetworkError(
             "Failed to push changes: the remote host is unreachable. Sandbox-authoritative auto-commit "
-            "pushes from inside the sandbox, so the sandbox environment must run with network access "
-            "(network_enabled=True)."
+            "pushes from inside the sandbox, so the sandbox environment must run as an egress-enabled "
+            "sandbox."
         )
     raise GitCommandError(["git", *push_args], result.exit_code, result.output)

daiv/automation/agent/middlewares/sandbox.py

@@ -291,6 +291,21 @@ def _make_global_skills_archive() -> bytes | None:
     return buf.getvalue()
 
 
+class SandboxEgressUnavailableError(RuntimeError):
+    """Raised when a session's resolved egress policy cannot be provisioned because the sandbox has
+    no egress proxy configured (no shared egress CA). daiv-sandbox rejects such a session up front
+    with HTTP 400 and a detail naming the egress proxy (see its ``POST /session/`` handler).
+    Fail-closed: an environment that requires a restricted egress policy must not run without that
+    policy in force."""
+
+
+# Substring that distinguishes the "egress proxy not configured" 400 from any other create-time 400.
+# Couples to daiv-sandbox's FastAPI ``detail`` ("egress requires the egress proxy, which is not
+# configured on this deployment" — its ``POST /session/`` handler). The sandbox exposes no
+# machine-readable error code, so this is the single greppable point of that prose coupling.
+_EGRESS_PROXY_UNAVAILABLE_MARKER = "egress proxy"
+
+
 class BashFailure(Enum):
     """Why a bash invocation produced no result, mapped to the guidance the agent gets.
 
@@ -522,22 +537,40 @@ async def abefore_agent(self, state: StateT, runtime: Runtime[RuntimeCtx]) -> di
             return {"session_id": prior_session_id}
 
         sb = runtime.context.sandbox
-        session_id = await client.start_session(
-            StartSessionRequest(
-                base_image=sb.base_image,
-                network_enabled=sb.network_enabled,
-                memory_bytes=sb.memory_bytes,
-                cpus=sb.cpus,
-                environment=sb.env_vars or None,
+        try:
+            session_id = await client.start_session(
+                StartSessionRequest(
+                    base_image=sb.base_image,
+                    egress=sb.egress,
+                    memory_bytes=sb.memory_bytes,
+                    cpus=sb.cpus,
+                    environment=sb.env_vars or None,
+                )
             )
-        )
+        except httpx.HTTPStatusError as exc:
+            # A network-enabled env on a sandbox with no egress proxy (no shared CA) is rejected up
+            # front with HTTP 400 (see _EGRESS_PROXY_UNAVAILABLE_MARKER). Match that specific signal so
+            # an unrelated 400 (e.g. an invalid base image) re-raises as-is instead of being mislabelled.
+            detail = (exc.response.text or "").lower()
+            if exc.response.status_code == 400 and _EGRESS_PROXY_UNAVAILABLE_MARKER in detail:
+                logger.error(
+                    "Sandbox rejected egress-required session (400): the egress proxy/CA is not configured "
+                    "on the sandbox deployment. Aborting run (fail-closed)."
+                )
+                raise SandboxEgressUnavailableError(
+                    "The resolved sandbox environment requires the egress proxy, but the sandbox rejected "
+                    "the session (400). Configure the shared egress CA (DAIV_SANDBOX_EGRESS_CA_CERT_FILE + "
+                    "DAIV_SANDBOX_EGRESS_CA_KEY_FILE) on the sandbox deployment to enable the egress proxy."
+                ) from exc
+            raise
         try:
             working_dir = Path(runtime.context.gitrepo.working_dir)
             repo_archive, skills_archive = await asyncio.gather(
                 asyncio.to_thread(_make_repo_archive, str(working_dir)), asyncio.to_thread(_make_global_skills_archive)
             )
             await client.seed_session(session_id, repo_archive=repo_archive, skills_archive=skills_archive)
         except Exception:
+            # Build/seed failure on an already-created session (the egress-unavailable case fails earlier).
             logger.exception("Failed to build or seed sandbox session %s", session_id)
             try:
                 await client.close_session(session_id, force=True)

daiv/codebase/clients/base.py

@@ -1,13 +1,17 @@
 from __future__ import annotations
 
 import abc
+import base64
 import functools
 import logging
 from contextlib import contextmanager
+from dataclasses import dataclass
 from enum import StrEnum
 from functools import cached_property
 from typing import TYPE_CHECKING, Any
 
+from pydantic import SecretStr
+
 from codebase.base import (
     Discussion,
     GitPlatform,
@@ -41,6 +45,29 @@ class WebhookSetupResult(StrEnum):
     SKIPPED = "skipped"
 
 
+@dataclass(frozen=True)
+class GitEgressCredential:
+    """Egress-proxy contribution for a repo's git platform: which host to allow and the
+    ``Authorization`` header to inject so git-over-HTTPS in the sandbox is authenticated.
+
+    ``value`` is ``None`` when no token could be provisioned — the host is still returned so
+    reachability works (the repo's origin remote authenticates via its ``.git/config`` token)."""
+
+    host: str
+    header: str = "Authorization"
+    value: SecretStr | None = None
+
+    @classmethod
+    def for_token(cls, *, host: str, token: str | None) -> GitEgressCredential:
+        """Build a credential injecting ``Authorization: Basic base64("oauth2:<token>")`` —
+        the same shape DAIV's clone URL uses. ``value`` is ``None`` when ``token`` is falsy."""
+        value = None
+        if token:
+            encoded = base64.b64encode(f"oauth2:{token}".encode()).decode()
+            value = SecretStr(f"Basic {encoded}")
+        return cls(host=host, value=value)
+
+
 class RepoClient(abc.ABC):
     """
     Abstract class for repository clients.
@@ -110,6 +137,26 @@ def set_repository_webhooks(
     def load_repo(self, repository: Repository, sha: str) -> Iterator[Repo]:
         pass
 
+    def get_git_egress_credential(self, repository: Repository) -> GitEgressCredential | None:
+        """Egress allow-rule + credential for this repo's git platform, or ``None`` for platforms
+        that need none. Resolved per run because the host is repo-derived and the token is
+        short-lived — never stored on the environment.
+
+        The shared shape lives here — derive the host from the clone URL, then build a
+        ``Basic oauth2:<token>`` credential. Platforms supply only the token via
+        :meth:`_git_egress_token` (overriding this method is unnecessary)."""
+        from urllib.parse import urlparse
+
+        host = urlparse(repository.clone_url).hostname
+        if not host:
+            return None
+        return GitEgressCredential.for_token(host=host, token=self._git_egress_token(repository))
+
+    def _git_egress_token(self, repository: Repository) -> str | None:
+        """Short-lived token authenticating git-over-HTTPS for this repo's platform, or ``None`` for
+        platforms that need no credential (host-only reachability). Overridden by GitLab/GitHub."""
+        return None
+
     # Issue
     @abc.abstractmethod
     def get_issue(self, repo_id: str, issue_id: int) -> Issue:

daiv/codebase/clients/github/client.py

@@ -265,6 +265,20 @@ def load_repo(self, repository: Repository, sha: str) -> Iterator[Repo]:
             self._configure_commit_identity(repo)
             yield repo
 
+    def _git_egress_token(self, repository: Repository) -> str | None:
+        """Mint a short-lived installation token scoped to ``contents: write``.
+
+        The egress proxy injects this token over the ``Authorization`` header for *every* request to the
+        platform host (the platform rule matches all methods and forces interception), superseding the
+        token git embeds from ``.git/config``. DAIV's publish push runs from inside the sandbox and goes
+        through the same path, so the injected token must be **write**-capable or the push is rejected.
+        It matches the ``contents: write`` token used to clone. The agent is still prevented from pushing
+        ad-hoc from ``bash`` by the command policy, not by withholding write scope here."""
+        access_token = self._integration.get_access_token(
+            self.client_installation.id, permissions={"contents": "write"}
+        )
+        return access_token.token
+
     # Issue
     def get_issue(self, repo_id: str, issue_id: int) -> Issue:
         """