Skip to content

Reuse Faraday connection across requests#636

Open
khasinski wants to merge 2 commits into
alexrudall:mainfrom
khasinski:feature/connection-reuse
Open

Reuse Faraday connection across requests#636
khasinski wants to merge 2 commits into
alexrudall:mainfrom
khasinski:feature/connection-reuse

Conversation

@khasinski

@khasinski khasinski commented Mar 1, 2026

Copy link
Copy Markdown

Summary

  • Cache the Faraday connection object on the client instance instead of creating a new one per HTTP request
  • Enables TCP keep-alive and avoids rebuilding the middleware stack on every call
  • Duped clients (via beta/admin) get their own fresh connections through initialize_dup

Benchmark against live OpenAI API (10x models.list after warmup):

WITHOUT connection reuse: 7.352s (735.2ms avg)
WITH connection reuse:    6.365s (636.5ms avg)
~13% faster

Test plan

  • All 155 existing specs pass
  • Verified connection object is reused (same object_id across calls)
  • Verified dup creates a fresh connection (for beta/admin)
  • Benchmarked against live API with warmup to confirm real improvement

khasinski added 2 commits March 1, 2026 23:50
@khasinski
Reuse Faraday connection across requests
0131205 
Instead of creating a new Faraday connection object on every HTTP
request, cache it on the client instance. This enables TCP keep-alive
and avoids rebuilding the middleware stack each time.

Duped clients (via beta/admin) get their own fresh connections
through initialize_dup.
@khasinski
Fix Naming/MemoizedInstanceVariableName rubocop offense
3552a08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@khasinski