Skip to content

feat(python-sdk): adopt pyqwest 0.8's proxy object, multipart fix, and request loggers - #1626

Closed
mishushakov wants to merge 3 commits into
migrate-envd-api-to-pyqwestfrom
adopt-pyqwest-0.8
Closed

feat(python-sdk): adopt pyqwest 0.8's proxy object, multipart fix, and request loggers#1626
mishushakov wants to merge 3 commits into
migrate-envd-api-to-pyqwestfrom
adopt-pyqwest-0.8

Conversation

@mishushakov

@mishushakov mishushakov commented Jul 30, 2026

Copy link
Copy Markdown
Member

What

Tracked in SDK-302 (part of the SDK-268 stack). Stacked on #1623. Adopts the three features merged into pyqwest after 0.7.0 and drops the workarounds the SDK carried while they were missing:

pyqwest PR What it adds What it lets us delete here
#194 Proxy object (auth, headers, no_proxy, scheme) rejecting httpx.Proxy headers; folding credentials into the URL userinfo
#196 multipart form support, and the httpx adapter matching dual sync/async streams sync-first _SyncOnlyStream, the rewrap of httpx's MultipartStream
#197 pyqwest / pyqwest.access loggers, emitted from Rust at DEBUG the "until pyqwest ships a logging middleware" note on the RPC interceptor (pyqwest#192 was superseded)

Important

Blocked on a pyqwest 0.8.0 release. All three are on curioswitch/pyqwest@main but unreleased — the latest PyPI release is 0.7.0 (2026-07-19). So uv.lock can't be regenerated and CI's uv sync --locked fails until 0.8.0 is published. Everything below was verified against a wheel built locally from pyqwest@6d56923.

How

Proxyproxy_to_url() -> Optional[str] becomes proxy_to_config() -> Optional[ProxyConfig], a NamedTuple of (url, auth, headers) that does double duty: it keys the transport caches (a pyqwest.Proxy compares by identity, so caching on one would hand every call its own connection pool) and builds the Proxy via to_pyqwest(). Consequences for the proxy option:

  • An httpx.Proxy's custom headers are now honored instead of raising InvalidArgumentException.
  • Credentials pass through as a (username, password) pair rather than being percent-encoded back into the URL userinfo — reqwest sends Proxy-Authorization itself.
  • A per-proxy ssl_context is still rejected; pyqwest has no counterpart.
  • Transports are keyed on the whole config, so the same proxy URL with different credentials or headers gets its own pool.

It's a NamedTuple rather than a frozen dataclass on purpose: tests/test_env_var_parsing.py reloads e2b.api, which rebinds the class, and a dataclass's __eq__ compares class identity — so cache keys created before and after a reload would silently stop matching. Tuples compare by value.

MultipartApiPyqwestTransport.handle_request no longer rewraps request streams. httpx's MultipartStream (files= uploads, i.e. files.write) implements both SyncByteStream and AsyncByteStream; the 0.7 adapter matched the async case first and raised TypeError("unreachable") from inside the body iterator, surfacing as a mid-request WriteError. Upstream now matches sync first. The regression test stays and covers the upstream fix.

Loggers — no wiring needed (the loggers are process-wide and emitted from Rust), so this is documentation plus a test that pins the behavior. Two comments record how the layers relate: retrying_http_transport notes that pyqwest logs requests below the SDK's logger option, and LoggingInterceptor's docstring replaces the "stays until pyqwest#192 ships" note — that PR was superseded by #197, whose loggers can't carry a per-sandbox logger nor see streamed messages or the Connect error code of a stream that fails inside a 200 OK, so the interceptor is permanent.

Testing

  • Unit suite: 240 passed, 1 skipped. ruff check, ruff format --check, ty check clean.

  • New: test_sync_transport_sends_proxy_credentials_and_headers (the echo server stands in for a proxy, asserting absolute-form request target + Proxy-Authorization + the extra header actually reach it) and test_transport_emits_pyqwest_access_log. Rewrote the nine proxy_to_url narrowing tests for proxy_to_config, plus one asserting equal configs share a pool while the Proxy objects they build do not compare equal.

  • E2E against production on the local 0.8 wheel, sync and async: create → files.write (single part) → write_files (multi-part body) → file-like write (octet-stream path) → read(format="stream")commands.runget_infokill. All green, with the access log showing every REST, envd, and RPC request:

    DEBUG pyqwest.access - HTTP Request: POST https://api.e2b.app/sandboxes "HTTP/2 201 Created"
    DEBUG pyqwest.access - HTTP Request: POST https://sandbox.e2b.app/files "HTTP/2 200 OK"
    DEBUG pyqwest.access - HTTP Request: POST https://sandbox.e2b.app/process.Process/Start "HTTP/2 200 OK"
    DEBUG pyqwest.access - HTTP Request: DELETE https://api.e2b.app/sandboxes/ickrgvawk1s9mn0963eta "HTTP/2 204 No Content"
    

Usage examples

A proxy that needs a custom header no longer has to be rewritten as a URL — it just works:

import httpx
from e2b import Sandbox

# Previously raised InvalidArgumentException
sbx = Sandbox.create(
    proxy=httpx.Proxy(
        "http://corp-proxy.internal:8030",
        auth=("user", "p@ss"),                 # sent as Proxy-Authorization
        headers={"X-Proxy-Token": "secret"},   # sent to the proxy
    )
)

Transport-level HTTP logs, replacing the httpcore records that the move off httpx transports removed:

import logging

logging.basicConfig()
logging.getLogger("pyqwest.access").setLevel(logging.DEBUG)   # one line per request
logging.getLogger("pyqwest").setLevel(logging.DEBUG)          # + request lifecycle

Sandbox.create()
# DEBUG pyqwest.access - HTTP Request: POST https://api.e2b.app/sandboxes "HTTP/2 201 Created"

Both are opt-in and independent of the SDK's own logger= option, which is unchanged.

Follow-ups

🤖 Generated with Claude Code

mishushakov and others added 3 commits July 29, 2026 19:37
…nvd RPC stack

With the envd RPC migration (#1558) on main, both stacks carried identical
copies of the transport pieces. Make e2b.api the canonical home: proxy_to_url
(with stack-neutral error messages) and the pool tuning move out of
e2b.envd.client_shared, and the flavor modules gain a shared
retrying_http_transport factory + ConnectionRetryTransport that
e2b.envd.client_sync/client_async now import instead of defining their own.
The stacks keep separate per-proxy pools (unification is a follow-up).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap the per-thread/per-loop envd httpx clients (file transfers, health
checks) for shared pyqwest-backed ones built from the same transport pieces
as the REST client and envd RPC. Streamed downloads default to a dedicated
transport whose 60s idle read timeout bounds stalls (reqwest's read timer
ticks during body send/TTFB, so it can't live on the shared transport);
explicit request timeouts become whole-transfer deadlines and streamed
uploads carry no client-side timeout, matching the JS SDK. Present httpx's
dual sync/async MultipartStream as sync-only — the stock adapter matches
AsyncByteStream first and kills multipart uploads mid-request.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d loggers

Require pyqwest 0.8 and take up the three features it adds over 0.7:

- `Proxy` object: `proxy_to_url` becomes `proxy_to_config`, returning the
  proxy URL, credentials, and headers as a tuple that both keys the transport
  caches and builds a `pyqwest.Proxy`. An `httpx.Proxy`'s custom headers are
  no longer rejected, and its credentials pass through as a pair instead of
  being percent-encoded back into the URL userinfo. A per-proxy `ssl_context`
  stays unsupported.
- Multipart: the adapter now matches dual sync/async request streams as sync
  first, so the `_SyncOnlyStream` rewrap of httpx's `MultipartStream` goes
  away; the regression test stays and now covers the upstream fix.
- Loggers: pyqwest logs requests itself on `pyqwest.access`/`pyqwest` at
  DEBUG, restoring the transport-level diagnostics httpcore used to give.
  They are process-wide, so they complement rather than replace the SDK's
  `logger` option — noted where the transports are built and on the RPC
  logging interceptor, whose upstreaming (pyqwest#192) was superseded.

Co-Authored-By: Claude <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: bf0bd6e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@e2b/python-sdk Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@cla-bot cla-bot Bot added the cla-signed label Jul 30, 2026
@cursor

cursor Bot commented Jul 30, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches shared HTTP/RPC transport and proxy configuration for all API traffic; behavior changes for proxied clients using httpx.Proxy headers, with broad test coverage mitigating regressions.

Overview
Requires pyqwest 0.8 and replaces proxy_to_url with proxy_to_config returning a ProxyConfig NamedTuple (URL, auth, headers) used as transport cache keys and converted via to_pyqwest() for pyqwest transports across REST, envd HTTP, and RPC.

httpx.Proxy custom headers and separate auth credentials are forwarded instead of raising on headers or encoding auth into the URL userinfo; per-proxy ssl_context still raises InvalidArgumentException. Different credentials or headers on the same proxy URL get separate connection pools.

Removes sync ApiPyqwestTransport _SyncOnlyStream rewrap for multipart files= uploads now that pyqwest 0.8 prefers the sync byte stream path. Notes that pyqwest.access / pyqwest DEBUG logging replaces httpcore-style transport diagnostics while the SDK logger option stays on the RPC interceptor. Changesets and tests cover proxy wiring, access logs, and pool caching.

Reviewed by Cursor Bugbot for commit bf0bd6e. Bugbot is set up for automated code reviews on this repo. Configure here.

@linear-code

linear-code Bot commented Jul 30, 2026

Copy link
Copy Markdown

SDK-302

@mishushakov
mishushakov force-pushed the migrate-envd-api-to-pyqwest branch from c195691 to d34b5b1 Compare July 30, 2026 13:53
@mishushakov

Copy link
Copy Markdown
Member Author

Superseded: the changes are folded into the stack they correct — the httpx.Proxypyqwest.Proxy mapping and the pyqwest 0.8 bump into #1601, the multipart-workaround removal and the envd renames into #1623, and the proxy_to_config rename into #1602/#1603. Tracking stays on SDK-302.

@mishushakov
mishushakov deleted the adopt-pyqwest-0.8 branch July 30, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant