fix(egress): unblock SSE/chunked streaming through mitmproxy (#898)

* fix(egress): unblock SSE/chunked streaming through mitmproxy

The mitmdump --set stream_large_bodies=1m option (added to prevent OOM
on large downloads) buffers any response under 1 MiB before forwarding,
which stalls LLM SSE / chunked streams of small chunks until the stream
ends or the threshold is hit, producing perceptible stutter downstream.

Introduce a bundled system addon that is always loaded by the egress
launcher and forces flow.response.stream = True for responses with
content-type: text/event-stream or transfer-encoding: chunked. Large-
body OOM protection from stream_large_bodies stays intact for everything
else.

The previous example script add_header.py is renamed to system.py and
repurposed as the always-on system addon (wire-transparent: no headers
added or altered). User-supplied addons via OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT
are still loaded after the system addon and may observe or override its
hooks.

Refs: https://project.aone.alibaba-inc.com/v2/project/2135082/req/82131871

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(egress): case-insensitive header match and correct system addon docs

Normalize content-type and transfer-encoding to lowercase before substring
matching in the system addon, so legal mixed-case values like
Transfer-Encoding: Chunked or Content-Type: Text/Event-Stream still trigger
streaming instead of getting buffered by stream_large_bodies=1m.

Also fix the transparent-mode doc, which referenced a non-existent
OPENSANDBOX_EGRESS_MITMPROXY_SYSTEM_SCRIPT env var as a way to disable or
swap the system addon. The launcher always appends the bundled system addon
unconditionally; document that users override behavior via
OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT (loaded after the system addon).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

Author: Pangjiping

components/egress/docs/mitmproxy-transparent.md

@@ -30,8 +30,8 @@ By default, mitmproxy listens on `18081` and transparent redirect rules are set
 # Optional: change listening port (default: 18081)
 export OPENSANDBOX_EGRESS_MITMPROXY_PORT=18081
 
-# Optional: enable mitm addon script (e.g., inject request headers)
-export OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT=/opt/opensandbox/mitmscripts/add_header.py
+# Optional: load an additional user-defined mitm addon (loaded after the system addon)
+export OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT=/path/to/your/addon.py
 
 # Optional: bypass decryption for selected domains (semicolon-separated regex list)
 export OPENSANDBOX_EGRESS_MITMPROXY_IGNORE_HOSTS='.*\.log\.aliyuncs\.com;.*\.example\.internal'
@@ -43,7 +43,7 @@ export OPENSANDBOX_EGRESS_MITMPROXY_IGNORE_HOSTS='.*\.log\.aliyuncs\.com;.*\.exa
 |------|----------|------|--------|
 | `OPENSANDBOX_EGRESS_MITMPROXY_TRANSPARENT` | Yes | Enable transparent mitmproxy (`1/true/on`, etc.) | Disabled |
 | `OPENSANDBOX_EGRESS_MITMPROXY_PORT` | No | mitmdump listen port; `iptables` redirects `80/443` here | `18081` |
-| `OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT` | No | mitm addon script path (`-s`) | Empty |
+| `OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT` | No | Additional user mitm addon script path (`-s`); loaded after the system addon | Empty |
 | `OPENSANDBOX_EGRESS_MITMPROXY_IGNORE_HOSTS` | No | Host/IP regex list for TLS pass-through (`;` separated) | Empty |
 | `OPENSANDBOX_EGRESS_MITMPROXY_CONFDIR` | No | mitm config and CA directory (passed as `--set confdir=`, also used as `HOME`) | Default directory under `/var/lib/mitmproxy` |
 | `OPENSANDBOX_EGRESS_MITMPROXY_UPSTREAM_TRUST_DIR` | No | Trust directory for upstream TLS verification (OpenSSL style) | `/etc/ssl/certs` |
@@ -62,23 +62,31 @@ Notes:
 export OPENSANDBOX_EGRESS_MITMPROXY_TRANSPARENT=true
 ```
 
-### 2) Enable with Header Injection
+### 2) System Addon (Always On)
+
+The bundled system addon at `/var/egress/mitmscripts/system.py` is shipped in the egress image and loaded automatically whenever transparent mode is enabled. It stays wire-transparent (no headers added or altered) and currently provides:
+
+- Forces streaming (`flow.response.stream = True`) for SSE (`text/event-stream`) and chunked responses, so each chunk is forwarded immediately instead of being buffered up to the `stream_large_bodies=1m` threshold (critical for LLM streaming UX).
+
+The system addon is always loaded and cannot be disabled via configuration. To override its behavior, supply a user addon via `OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT`; user addons are loaded after the system addon and may observe or override its hooks.
+
+### 3) Add a User Addon Alongside the System Addon
 
 ```bash
 export OPENSANDBOX_EGRESS_MITMPROXY_TRANSPARENT=true
-export OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT=/opt/opensandbox/mitmscripts/add_header.py
+export OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT=/path/to/your/addon.py
 ```
 
-Built-in example script: `/opt/opensandbox/mitmscripts/add_header.py` (adds `X-OpenSandbox-Egress: 1`).
+The user addon is loaded after the system addon (`-s system.py -s user.py`), so user hooks observe and may override system behavior.
 
-### 3) Bypass Decryption for Specific Domains (e.g. log upload)
+### 4) Bypass Decryption for Specific Domains (e.g. log upload)
 
 ```bash
 export OPENSANDBOX_EGRESS_MITMPROXY_TRANSPARENT=true
 export OPENSANDBOX_EGRESS_MITMPROXY_IGNORE_HOSTS='.*\.log\.aliyuncs\.com'
 ```
 
-### 4) Use a Fixed CA (consistent fingerprint across replicas)
+### 5) Use a Fixed CA (consistent fingerprint across replicas)
 
 If CA files already exist in `confdir`, mitmproxy reuses them instead of regenerating on each startup. Typical paths:
 

components/egress/mitmscripts/add_header.py

@@ -0,0 +1,36 @@
+# Copyright 2026 Alibaba Group Holding Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# OpenSandbox egress system addon.
+#
+# Always loaded by the egress mitmproxy launcher. Stays transparent on the
+# wire (does not add or alter headers that would reveal the proxy to peers).
+#
+# Behavior:
+#   Forces streaming for SSE / chunked responses so each chunk is forwarded
+#   immediately, bypassing the stream_large_bodies=1m buffer set in launch.go
+#   (which otherwise stalls LLM-style small-chunk streams).
+#
+# User-defined addons can be loaded alongside this script via
+# OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT.
+from mitmproxy import http
+
+
+def responseheaders(flow: http.HTTPFlow) -> None:
+    if flow.response is None:
+        return
+    content_type = flow.response.headers.get("content-type", "").lower()
+    transfer_encoding = flow.response.headers.get("transfer-encoding", "").lower()
+    if "text/event-stream" in content_type or "chunked" in transfer_encoding:
+        flow.response.stream = True

components/egress/mitmscripts/system.py

@@ -34,11 +34,16 @@ const RunAsUser = "mitmproxy"
 // Loopback: transparent mode receives via REDIRECT; do not listen on 0.0.0.0 in the netns.
 const listenHostLoopback = "127.0.0.1"
 
+// systemScriptPath: bundled system addon shipped via the egress Dockerfile
+// (COPY components/egress/mitmscripts /var/egress/mitmscripts). Always loaded.
+const systemScriptPath = "/var/egress/mitmscripts/system.py"
+
 // Config: mitmdump --mode transparent; UserName must match iptables ! --uid-owner, ConfDir is mitm state/CA.
 type Config struct {
 	ListenPort int
 	UserName   string
 	ConfDir    string
+	// ScriptPath is an optional user-supplied addon, loaded after the system addon.
 	ScriptPath string
 	// OnExit is called (if non-nil) when mitmdump exits. Called from a background goroutine.
 	OnExit func(error)
@@ -120,8 +125,10 @@ func Launch(cfg Config) (*Running, error) {
 		args = append(args, "--set", "confdir="+cd)
 		homeEnv = cd
 	}
-	if strings.TrimSpace(cfg.ScriptPath) != "" {
-		args = append(args, "-s", strings.TrimSpace(cfg.ScriptPath))
+	// Load the system addon first so user addons can observe / override its hooks.
+	args = append(args, "-s", systemScriptPath)
+	if user := strings.TrimSpace(cfg.ScriptPath); user != "" {
+		args = append(args, "-s", user)
 	}
 
 	// Upstream passthrough: each pattern becomes --set ignore_hosts= (regex; IP ranges are practical in transparent mode).