feat(python-sdk): adopt pyqwest 0.8's proxy object, multipart fix, and request loggers - #1626
feat(python-sdk): adopt pyqwest 0.8's proxy object, multipart fix, and request loggers#1626mishushakov wants to merge 3 commits into
Conversation
…nvd RPC stack With the envd RPC migration (#1558) on main, both stacks carried identical copies of the transport pieces. Make e2b.api the canonical home: proxy_to_url (with stack-neutral error messages) and the pool tuning move out of e2b.envd.client_shared, and the flavor modules gain a shared retrying_http_transport factory + ConnectionRetryTransport that e2b.envd.client_sync/client_async now import instead of defining their own. The stacks keep separate per-proxy pools (unification is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap the per-thread/per-loop envd httpx clients (file transfers, health checks) for shared pyqwest-backed ones built from the same transport pieces as the REST client and envd RPC. Streamed downloads default to a dedicated transport whose 60s idle read timeout bounds stalls (reqwest's read timer ticks during body send/TTFB, so it can't live on the shared transport); explicit request timeouts become whole-transfer deadlines and streamed uploads carry no client-side timeout, matching the JS SDK. Present httpx's dual sync/async MultipartStream as sync-only — the stock adapter matches AsyncByteStream first and kills multipart uploads mid-request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d loggers Require pyqwest 0.8 and take up the three features it adds over 0.7: - `Proxy` object: `proxy_to_url` becomes `proxy_to_config`, returning the proxy URL, credentials, and headers as a tuple that both keys the transport caches and builds a `pyqwest.Proxy`. An `httpx.Proxy`'s custom headers are no longer rejected, and its credentials pass through as a pair instead of being percent-encoded back into the URL userinfo. A per-proxy `ssl_context` stays unsupported. - Multipart: the adapter now matches dual sync/async request streams as sync first, so the `_SyncOnlyStream` rewrap of httpx's `MultipartStream` goes away; the regression test stays and now covers the upstream fix. - Loggers: pyqwest logs requests itself on `pyqwest.access`/`pyqwest` at DEBUG, restoring the transport-level diagnostics httpcore used to give. They are process-wide, so they complement rather than replace the SDK's `logger` option — noted where the transports are built and on the RPC logging interceptor, whose upstreaming (pyqwest#192) was superseded. Co-Authored-By: Claude <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: bf0bd6e The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
PR SummaryMedium Risk Overview
Removes sync Reviewed by Cursor Bugbot for commit bf0bd6e. Bugbot is set up for automated code reviews on this repo. Configure here. |
c195691 to
d34b5b1
Compare
What
Tracked in SDK-302 (part of the SDK-268 stack). Stacked on #1623. Adopts the three features merged into pyqwest after 0.7.0 and drops the workarounds the SDK carried while they were missing:
Proxyobject (auth, headers,no_proxy, scheme)httpx.Proxyheaders; folding credentials into the URL userinfo_SyncOnlyStream, the rewrap of httpx'sMultipartStreampyqwest/pyqwest.accessloggers, emitted from Rust atDEBUGImportant
Blocked on a pyqwest 0.8.0 release. All three are on
curioswitch/pyqwest@mainbut unreleased — the latest PyPI release is 0.7.0 (2026-07-19). Souv.lockcan't be regenerated and CI'suv sync --lockedfails until 0.8.0 is published. Everything below was verified against a wheel built locally frompyqwest@6d56923.How
Proxy —
proxy_to_url() -> Optional[str]becomesproxy_to_config() -> Optional[ProxyConfig], aNamedTupleof(url, auth, headers)that does double duty: it keys the transport caches (apyqwest.Proxycompares by identity, so caching on one would hand every call its own connection pool) and builds theProxyviato_pyqwest(). Consequences for theproxyoption:httpx.Proxy's custom headers are now honored instead of raisingInvalidArgumentException.(username, password)pair rather than being percent-encoded back into the URL userinfo — reqwest sendsProxy-Authorizationitself.ssl_contextis still rejected; pyqwest has no counterpart.It's a
NamedTuplerather than a frozen dataclass on purpose:tests/test_env_var_parsing.pyreloadse2b.api, which rebinds the class, and a dataclass's__eq__compares class identity — so cache keys created before and after a reload would silently stop matching. Tuples compare by value.Multipart —
ApiPyqwestTransport.handle_requestno longer rewraps request streams. httpx'sMultipartStream(files=uploads, i.e.files.write) implements bothSyncByteStreamandAsyncByteStream; the 0.7 adapter matched the async case first and raisedTypeError("unreachable")from inside the body iterator, surfacing as a mid-requestWriteError. Upstream now matches sync first. The regression test stays and covers the upstream fix.Loggers — no wiring needed (the loggers are process-wide and emitted from Rust), so this is documentation plus a test that pins the behavior. Two comments record how the layers relate:
retrying_http_transportnotes that pyqwest logs requests below the SDK'sloggeroption, andLoggingInterceptor's docstring replaces the "stays until pyqwest#192 ships" note — that PR was superseded by #197, whose loggers can't carry a per-sandbox logger nor see streamed messages or the Connect error code of a stream that fails inside a200 OK, so the interceptor is permanent.Testing
Unit suite: 240 passed, 1 skipped.
ruff check,ruff format --check,ty checkclean.New:
test_sync_transport_sends_proxy_credentials_and_headers(the echo server stands in for a proxy, asserting absolute-form request target +Proxy-Authorization+ the extra header actually reach it) andtest_transport_emits_pyqwest_access_log. Rewrote the nineproxy_to_urlnarrowing tests forproxy_to_config, plus one asserting equal configs share a pool while theProxyobjects they build do not compare equal.E2E against production on the local 0.8 wheel, sync and async: create →
files.write(single part) →write_files(multi-part body) → file-like write (octet-stream path) →read(format="stream")→commands.run→get_info→kill. All green, with the access log showing every REST, envd, and RPC request:Usage examples
A proxy that needs a custom header no longer has to be rewritten as a URL — it just works:
Transport-level HTTP logs, replacing the httpcore records that the move off httpx transports removed:
Both are opt-in and independent of the SDK's own
logger=option, which is unchanged.Follow-ups
proxy_to_url; they need the rename when they rebase onto this.no_proxyand per-scheme proxy routing, whichhttpx.Proxycan't express. Not exposed; would need a new SDK option and JS parity.🤖 Generated with Claude Code