Skip to content

feat: Enable zstd response compression for RPC providers#821

Merged
dylankilkenny merged 4 commits into
mainfrom
feat/quicknode-zstd-compression
Jul 14, 2026
Merged

feat: Enable zstd response compression for RPC providers#821
dylankilkenny merged 4 commits into
mainfrom
feat/quicknode-zstd-compression

Conversation

@dylankilkenny

@dylankilkenny dylankilkenny commented Jul 9, 2026

Copy link
Copy Markdown
Member

Summary

Providers such as QuickNode can compress RPC responses with zstd, but the relayer never asked for it. Large responses arrived uncompressed and slowly enough to hit request timeouts. The worst case is Stellar getTransaction, whose Soroban XDR payloads can reach hundreds of kilobytes.

This change enables zstd response negotiation across the relayer's RPC clients. Enabling reqwest's zstd feature makes every reqwest client send Accept-Encoding: zstd and decompress responses transparently. Cargo feature unification extends this to the EVM and Solana clients, which use the same reqwest crate. Stellar getTransaction needed more than the feature flag. The stellar-rpc-client crate sits on a jsonrpsee transport that cannot decompress responses, so this call now goes through the provider's existing raw reqwest JSON-RPC path instead. The remaining stellar-rpc-client calls stay uncompressed and would need the same migration if they ever show up in timeout logs.

Reviewers should focus on the retry changes in the raw path. It previously treated non-2xx responses as JSON decode errors, and it mapped JSON-RPC error objects only after the retry loop finished. As a result, rate limits and transient server errors were never retried and never triggered failover. Both cases are now classified inside the retried operation. A side effect is that get_transaction errors change shape. They surface as typed ProviderError variants instead of jsonrpsee message strings, so log queries that match the old wording will stop matching for this call.

Testing Process

New unit tests cover:

  • End-to-end zstd negotiation and decompression, against a mockito server that returns pre-compressed bodies.
  • A 429 response is classified as rate limiting and retried.
  • A retriable JSON-RPC error (code -32005) is retried, and a non-retriable one (code -32601) fails fast without retrying.

Formatting, the CI clippy gate, and the provider and Stellar domain test suites all pass locally.

Checklist

  • Add a reference to related issues in the PR description.
  • Add unit tests if applicable.

Enables the reqwest `zstd` feature so every RPC client advertises
`Accept-Encoding: zstd` and transparently decompresses responses from
providers that support it (e.g. QuickNode), cutting transfer time on
large responses that were causing request timeouts. Via cargo feature
unification this also covers the reqwest clients inside
alloy-transport-http (EVM) and solana-rpc-client.

Stellar `getTransaction` — the high-volume status-polling call whose
Soroban XDR payloads can reach hundreds of KB — is migrated off the
stellar-rpc-client jsonrpsee client (which cannot decompress) onto the
raw reqwest JSON-RPC path, mirroring upstream's response parsing.

The raw path also gains proper error classification so retry/failover
semantics hold: HTTP status errors (429/5xx) surface with their status
code, and JSON-RPC error objects are mapped inside the retried
operation so retriable codes (-32005, -32603) retry and fail over.

Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
@dylankilkenny
dylankilkenny requested a review from a team as a code owner July 9, 2026 08:43
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 25d499c3-0a9a-4638-906f-9cf879f5326f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

Enables the zstd feature for reqwest and adds a zstd dev-dependency. Documents and tests shared client zstd handling. Reworks Stellar provider's raw JSON-RPC path with centralized HTTP/JSON-RPC error mapping, migrates get_transaction and raw_request_dyn to the raw path, and adds retry/zstd tests.

Changes

Zstd decompression and Stellar raw JSON-RPC path

Layer / File(s) Summary
Enable reqwest zstd feature
Cargo.toml
Adds zstd feature to reqwest and a zstd = "0.13" dev-dependency for compressing test fixtures.
Shared RPC client zstd documentation and test
src/services/provider/mod.rs
Documents that shared clients advertise Accept-Encoding: zstd and decompress responses; adds a mock-server test verifying this end-to-end.
Raw JSON-RPC request/error handling helpers
src/services/provider/stellar/mod.rs
Adds HTTP status checking in retry_raw_request, a new raw_json_rpc_request helper, and json_rpc_error_to_provider_error mapping.
get_transaction and raw_request_dyn migration
src/services/provider/stellar/mod.rs
Switches get_transaction and raw_request_dyn to use the new raw JSON-RPC path instead of the jsonrpsee client; updates error assertions.
Zstd and retry classification tests
src/services/provider/stellar/mod.rs
Adds tests for zstd-compressed responses, HTTP 429 retry classification, and retriable/non-retriable JSON-RPC error codes.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
    participant Client as StellarProvider
    participant Http as reqwest Client
    participant Server as RPC Server

    Client->>Http: raw_json_rpc_request(method, params)
    Http->>Server: POST request (Accept-Encoding: zstd)
    Server-->>Http: zstd-compressed JSON-RPC response
    Http-->>Client: decompressed JSON body

    alt HTTP error status
        Client->>Client: error_for_status_ref -> ProviderError
    else JSON-RPC "error" field present
        Client->>Client: json_rpc_error_to_provider_error(code)
        Client->>Client: retry_raw_request (if retriable)
    else success
        Client->>Client: extract "result" field
    end
Loading

Poem

A hop, a skip, past bytes compressed,
Zstd whispers "Accept" with zest!
Errors mapped with codes so neat,
Retries dance to their own beat.
🐰 Through JSON-RPC fields I dig,
No more jsonrpsee, just my rig!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: enabling zstd compression for RPC providers.
Description check ✅ Passed The description matches the template with Summary, Testing Process, and Checklist sections, and includes relevant implementation and test details.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/quicknode-zstd-compression

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
@dylankilkenny dylankilkenny changed the title feat: Zstd response compression for RPC providers feat: Enable zstd response compression for RPC providers Jul 9, 2026
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.57014% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.59%. Comparing base (b035c5a) to head (9ca9ec9).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/services/provider/stellar/mod.rs 94.57% 12 Missing ⚠️
Additional details and impacted files
Flag Coverage Δ
ai 0.00% <0.00%> (ø)
dev 89.59% <94.57%> (+0.04%) ⬆️
properties 0.01% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@            Coverage Diff             @@
##             main     #821      +/-   ##
==========================================
+ Coverage   89.54%   89.59%   +0.04%     
==========================================
  Files         303      303              
  Lines      128773   129442     +669     
==========================================
+ Hits       115314   115972     +658     
- Misses      13459    13470      +11     
Files with missing lines Coverage Δ
src/services/provider/mod.rs 93.63% <ø> (ø)
src/services/provider/stellar/mod.rs 91.31% <94.57%> (+1.04%) ⬆️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tirumerla tirumerla left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple nits, otherwise lgtm. Thanks

Comment thread src/services/provider/stellar/mod.rs Outdated
Comment thread src/services/provider/stellar/mod.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables zstd response compression negotiation for RPC calls made via reqwest, reducing payload sizes and mitigating timeout risk for large RPC responses (notably Stellar getTransaction with large Soroban XDR).

Changes:

  • Enable reqwest’s zstd feature so relayer HTTP clients advertise zstd support and transparently decompress zstd responses.
  • Re-route Stellar getTransaction through the existing raw reqwest JSON-RPC path (since the jsonrpsee transport cannot decompress), and improve retry classification by mapping HTTP status errors and JSON-RPC error objects inside the retried operation.
  • Add unit tests validating zstd negotiation/decompression and retry behavior for HTTP 429 and JSON-RPC error codes.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File Description
src/services/provider/stellar/mod.rs Moves get_transaction to the raw reqwest JSON-RPC path and fixes retry classification for HTTP status + JSON-RPC errors; adds targeted tests.
src/services/provider/mod.rs Documents zstd behavior for the shared reqwest client builder and adds a decompression test.
Cargo.toml Enables reqwest zstd feature and adds zstd as a dev-dependency for tests.
Cargo.lock Updates lockfile to include zstd-related dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Non-strict providers may return "error": null on success; skip null so a
valid response is not misclassified as an error. On non-2xx, read the
response body and surface it in RequestError instead of dropping it via
error_for_status, keeping status-based retry/failover classification.

Signed-off-by: Dylan Kilkenny <dylankilkenny95@gmail.com>
@dylankilkenny
dylankilkenny merged commit 48efd45 into main Jul 14, 2026
37 of 42 checks passed
@dylankilkenny
dylankilkenny deleted the feat/quicknode-zstd-compression branch July 14, 2026 14:30
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 14, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants