feat(python-sdk): move the envd HTTP API client onto pyqwest - #1623
feat(python-sdk): move the envd HTTP API client onto pyqwest#1623mishushakov wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: 77e563b The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
PR SummaryMedium Risk Overview Streamed Reviewed by Cursor Bugbot for commit 77e563b. Bugbot is set up for automated code reviews on this repo. Configure here. |
Package ArtifactsBuilt from d7b10db. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-2.36.2-migrate-envd-api-to-pyqwest.0.tgzCLI ( npm install ./e2b-cli-2.16.1-migrate-envd-api-to-pyqwest.0.tgzPython SDK ( pip install ./e2b-2.35.0+migrate.envd.api.to.pyqwest-py3-none-any.whl |
c195691 to
d34b5b1
Compare
d34b5b1 to
b86e23b
Compare
b86e23b to
f9cf7f2
Compare
…nvd RPC stack With the envd RPC migration (#1558) on main, both stacks carried identical copies of the transport pieces. Make e2b.api the canonical home: proxy_to_url (with stack-neutral error messages) and the pool tuning move out of e2b.envd.client_shared, and the flavor modules gain a shared retrying_http_transport factory + ConnectionRetryTransport that e2b.envd.client_sync/client_async now import instead of defining their own. The stacks keep separate per-proxy pools (unification is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap the per-thread/per-loop envd httpx clients (file transfers, health checks) for shared pyqwest-backed ones built from the same transport pieces as the REST client and envd RPC. Streamed downloads default to a dedicated transport whose 60s idle read timeout bounds stalls (reqwest's read timer ticks during body send/TTFB, so it can't live on the shared transport); explicit request timeouts become whole-transfer deadlines and streamed uploads carry no client-side timeout, matching the JS SDK. Present httpx's dual sync/async MultipartStream as sync-only — the stock adapter matches AsyncByteStream first and kills multipart uploads mid-request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
f9cf7f2 to
77e563b
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 77e563b. Configure here.
| # global idle cap, but API traffic goes to a single host, so the values map | ||
| # directly; reqwest has no counterpart to `E2B_MAX_CONNECTIONS` (it does not | ||
| # cap concurrent connections). | ||
| # Mirror the httpx pool tuning above with pyqwest's equivalents, shared by |
There was a problem hiding this comment.
Unused httpx pool limits
Low Severity
limits is no longer referenced anywhere after the envd httpx transports were removed, so E2B_MAX_CONNECTIONS and the related httpx Limits tuning are dead. That makes the leftover env-var surface look effective when it is not.
Reviewed by Cursor Bugbot for commit 77e563b. Configure here.


What
Tracked in SDK-265 (part of the SDK-268 stack). Stacked on #1603, at the top of the pyqwest stack (#1601 → #1602 → #1603 → this). Migrate the envd HTTP API client — sandbox file transfers (
files.read/write), health checks — from httpx-native transports to pyqwest via the httpx adapter, and dedupe the transport plumbing that #1558 (envd RPC) and #1601 (REST) each carried a copy of. With this, all Python SDK traffic runs on pyqwest: REST control plane (#1601), envd RPC (#1558, connectrpc), envd HTTP API (this PR); the volume content client (#1602) and template build uploads (#1603) sit below this one in the stack.How
Shared plumbing (first commit):
e2b.apibecomes the canonical home for the proxy narrowing (proxy_to_config, with stack-neutral error messages), the pool tuning, and the flavorConnectionRetryTransport+ a newretrying_http_transport(proxy, read_timeout=None)factory;e2b.envd.client_sync/client_asyncimport them instead of defining their own (envd RPC behavior unchanged, pools stay separate — unification is SDK-291).envd HTTP API (second commit):
get_envd_transport(config, for_streaming=False)returns pyqwest-adapter transports cached per(proxy, streaming);get_envd_api(config, base_url, for_streaming=False)builds the httpx client with sandbox headers + logging hooks. The per-thread (sync) / per-loop (async) client caching inFilesystem/Commands/Pty/AsyncSandboxis gone — one shared client per module, same rationale as theApiClientsimplification in feat(python-sdk): move the REST API client onto pyqwest's httpx transport adapter #1601.Streamed downloads: the streaming transport carries a 60s
read_timeout— an idle bound that resets on every read, capping stalls without limiting total transfer time. It gets a dedicated pool because reqwest's read timer keeps ticking while a request body is sent and while waiting for the response head, so on the shared transport it would cut off uploads and slow unary responses. An explicitrequest_timeoutbecomes the whole-transfer deadline (adapter semantics) and is sent only when the caller set one;stream_idle_timeoutstays honored on the async client viawait_forper read (so values above 60s work and0disables), and is documented as ignored on the sync client, which cannot interrupt a blocking read. Mirrors feat(python-sdk): move the volume content client onto pyqwest #1602's volume design.Uploads: buffered uploads keep
request_timeoutas a whole-request deadline; streamed (file-like) uploads carry no client-side timeout and are bounded server-side (envd's idle read timeout) — both exactly the JS SDK's behavior (getSignalfor buffered, no signal for streams).Multipart:
files=uploads go out as httpx'sMultipartStream, which implements bothSyncByteStreamandAsyncByteStream. The pyqwest 0.7 adapter's sync content conversion matchedAsyncByteStreamfirst and raisedTypeError("unreachable")from inside the body iterator, surfacing as aWriteErrormid-request ("http2 error: stream error sent by user"). Fixed upstream in pyqwest#196, which matches the sync case first — so this PR needs pyqwest 0.8 (required by feat(python-sdk): move the REST API client onto pyqwest's httpx transport adapter #1601 already) and carries no workaround. The regression test stays, now covering the upstream fix.The stream readers map the transport's idle timeout (builtin
TimeoutErrorunder pyqwest) to the documentedhttpx.ReadTimeout;handle_envd_api_transport_exception's health-probe path keeps working because the adapter maps HTTP/2 stream resets tohttpx.RemoteProtocolError.RPC logging: the
LoggingInterceptordocstring no longer promises its own removal. pyqwest 0.8 does log requests (pyqwest#197), but on process-widepyqwest/pyqwest.accessloggers that can't carry the per-sandboxloggerand don't see streamed messages or the Connect error code of a stream that fails inside a200 OK— so the interceptor stays, with those loggers below it. pyqwest#192, the middleware it referenced, was closed in favor of Sync is da way #197.Testing
get_envd_apiwiring (headers, transports), multipart regression through a local server, stream-reader timeout mapping + per-read idle bound (tests/test_file_stream_reader.py), rewritten client-lifecycle tests (shared across threads). 236 unit tests green; lint + typecheck green.filessuites sync+async (123 tests — these caught the multipart bug),commands+ptysuites both flavors (57 tests). All green.Usage example
No API changes:
Only visible behavior shift: on the sync client,
files.read(..., format="stream", stream_idle_timeout=...)is now a documented no-op (the transport-wide 60s idle bound applies); the async client honors it as before.🤖 Generated with Claude Code