Skip to content

least request lb: include pending requests#45168

Open
benjawo wants to merge 1 commit into
envoyproxy:mainfrom
benjawo:least-request-pending-requests
Open

least request lb: include pending requests#45168
benjawo wants to merge 1 commit into
envoyproxy:mainfrom
benjawo:least-request-pending-requests

Conversation

@benjawo
Copy link
Copy Markdown

@benjawo benjawo commented May 20, 2026

Commit Message:

LeastRequestLoadBalancer ranks hosts by host.stats().rq_active_, which only counts streams attached in attachStreamToClient. Streams queued in pending_streams_ on a connection pool waiting for connection establishment are invisible, so a host with slow handshakes can keep winning picks until queued streams attach. See #44989.

Add a per-host rq_pending_active gauge tracked in PendingStream's ctor/dtor and have LeastRequestLoadBalancer fold it into the value it compares behind a runtime guard, default off
(envoy.reloadable_features.least_request_lb_count_pending_requests).

Fixes #44989

Additional Description: This was an AI assisted code change. I understand all code written.
Risk Level: Low
Testing: UTs. Local repro script against envoy process.
Docs Changes: N/A
Release Notes: New changelog file.
Platform Specific Features: N/A
[Optional Runtime guard:] envoy.reloadable_features.least_request_lb_count_pending_requests
[Optional Fixes #Issue] #44989
[Optional Fixes commit #PR or SHA] N/A
[Optional Deprecated:] N/A
[Optional API Considerations:] N/A

LeastRequestLoadBalancer ranks hosts by host.stats().rq_active_, which
only counts streams attached in attachStreamToClient. Streams queued in
pending_streams_ on a connection pool waiting for connection
establishment are invisible, so a host with slow handshakes can keep
winning picks until queued streams attach. See envoyproxy#44989.

Add a per-host rq_pending_active gauge tracked in PendingStream's
ctor/dtor and have LeastRequestLoadBalancer fold it into the value it
compares behind a runtime guard, default off
(envoy.reloadable_features.least_request_lb_count_pending_requests).

Fixes envoyproxy#44989

Signed-off-by: Ben Wood <wood.ben94@gmail.com>
@benjawo benjawo requested a deployment to external-contributors May 20, 2026 04:17 — with GitHub Actions Waiting
@repokitteh-read-only
Copy link
Copy Markdown

Hi @benjawo, welcome and thank you for your contribution.

We will try to review your Pull Request as quickly as possible.

In the meantime, please take a look at the contribution guidelines if you have not done so already.

🐱

Caused by: #45168 was opened by benjawo.

see: more, trace.

@repokitteh-read-only
Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #45168 was opened by benjawo.

see: more, trace.

@benjawo benjawo marked this pull request as ready for review May 20, 2026 04:23
@repokitteh-read-only
Copy link
Copy Markdown

CC @envoyproxy/runtime-guard-changes: FYI only for changes made to (source/common/runtime/runtime_features.cc).

🐱

Caused by: #45168 was ready_for_review by benjawo.

see: more, trace.

@nezdolik nezdolik self-assigned this May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LeastRequestLoadBalancer does not account for pending requests

2 participants