feat: Enable zstd response compression for RPC providers#821
Conversation
Enables the reqwest `zstd` feature so every RPC client advertises `Accept-Encoding: zstd` and transparently decompresses responses from providers that support it (e.g. QuickNode), cutting transfer time on large responses that were causing request timeouts. Via cargo feature unification this also covers the reqwest clients inside alloy-transport-http (EVM) and solana-rpc-client. Stellar `getTransaction` — the high-volume status-polling call whose Soroban XDR payloads can reach hundreds of KB — is migrated off the stellar-rpc-client jsonrpsee client (which cannot decompress) onto the raw reqwest JSON-RPC path, mirroring upstream's response parsing. The raw path also gains proper error classification so retry/failover semantics hold: HTTP status errors (429/5xx) surface with their status code, and JSON-RPC error objects are mapped inside the retried operation so retriable codes (-32005, -32603) retry and fail over. Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughEnables the zstd feature for reqwest and adds a zstd dev-dependency. Documents and tests shared client zstd handling. Reworks Stellar provider's raw JSON-RPC path with centralized HTTP/JSON-RPC error mapping, migrates get_transaction and raw_request_dyn to the raw path, and adds retry/zstd tests. ChangesZstd decompression and Stellar raw JSON-RPC path
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Client as StellarProvider
participant Http as reqwest Client
participant Server as RPC Server
Client->>Http: raw_json_rpc_request(method, params)
Http->>Server: POST request (Accept-Encoding: zstd)
Server-->>Http: zstd-compressed JSON-RPC response
Http-->>Client: decompressed JSON body
alt HTTP error status
Client->>Client: error_for_status_ref -> ProviderError
else JSON-RPC "error" field present
Client->>Client: json_rpc_error_to_provider_error(code)
Client->>Client: retry_raw_request (if retriable)
else success
Client->>Client: extract "result" field
end
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. @@ Coverage Diff @@
## main #821 +/- ##
==========================================
+ Coverage 89.54% 89.59% +0.04%
==========================================
Files 303 303
Lines 128773 129442 +669
==========================================
+ Hits 115314 115972 +658
- Misses 13459 13470 +11
🚀 New features to boost your workflow:
|
tirumerla
left a comment
There was a problem hiding this comment.
Couple nits, otherwise lgtm. Thanks
There was a problem hiding this comment.
Pull request overview
This PR enables zstd response compression negotiation for RPC calls made via reqwest, reducing payload sizes and mitigating timeout risk for large RPC responses (notably Stellar getTransaction with large Soroban XDR).
Changes:
- Enable
reqwest’szstdfeature so relayer HTTP clients advertise zstd support and transparently decompress zstd responses. - Re-route Stellar
getTransactionthrough the existing rawreqwestJSON-RPC path (since the jsonrpsee transport cannot decompress), and improve retry classification by mapping HTTP status errors and JSON-RPC error objects inside the retried operation. - Add unit tests validating zstd negotiation/decompression and retry behavior for HTTP 429 and JSON-RPC error codes.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/services/provider/stellar/mod.rs |
Moves get_transaction to the raw reqwest JSON-RPC path and fixes retry classification for HTTP status + JSON-RPC errors; adds targeted tests. |
src/services/provider/mod.rs |
Documents zstd behavior for the shared reqwest client builder and adds a decompression test. |
Cargo.toml |
Enables reqwest zstd feature and adds zstd as a dev-dependency for tests. |
Cargo.lock |
Updates lockfile to include zstd-related dependencies. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Non-strict providers may return "error": null on success; skip null so a valid response is not misclassified as an error. On non-2xx, read the response body and surface it in RequestError instead of dropping it via error_for_status, keeping status-based retry/failover classification. Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
Summary
Providers such as QuickNode can compress RPC responses with zstd, but the relayer never asked for it. Large responses arrived uncompressed and slowly enough to hit request timeouts. The worst case is Stellar
getTransaction, whose Soroban XDR payloads can reach hundreds of kilobytes.This change enables zstd response negotiation across the relayer's RPC clients. Enabling reqwest's
zstdfeature makes every reqwest client sendAccept-Encoding: zstdand decompress responses transparently. Cargo feature unification extends this to the EVM and Solana clients, which use the same reqwest crate. StellargetTransactionneeded more than the feature flag. Thestellar-rpc-clientcrate sits on a jsonrpsee transport that cannot decompress responses, so this call now goes through the provider's existing raw reqwest JSON-RPC path instead. The remainingstellar-rpc-clientcalls stay uncompressed and would need the same migration if they ever show up in timeout logs.Reviewers should focus on the retry changes in the raw path. It previously treated non-2xx responses as JSON decode errors, and it mapped JSON-RPC error objects only after the retry loop finished. As a result, rate limits and transient server errors were never retried and never triggered failover. Both cases are now classified inside the retried operation. A side effect is that
get_transactionerrors change shape. They surface as typedProviderErrorvariants instead of jsonrpsee message strings, so log queries that match the old wording will stop matching for this call.Testing Process
New unit tests cover:
Formatting, the CI clippy gate, and the provider and Stellar domain test suites all pass locally.
Checklist