Skip to content

feat: Weighted Query Laning Strategy#19225

Open
mshahid6 wants to merge 8 commits into
apache:masterfrom
mshahid6:weighted-query-laning
Open

feat: Weighted Query Laning Strategy#19225
mshahid6 wants to merge 8 commits into
apache:masterfrom
mshahid6:weighted-query-laning

Conversation

@mshahid6

Copy link
Copy Markdown
Contributor

Description

Adds a new WeightedQueryLaningStrategy (strategy: "weighted") that scores queries by how many configured thresholds they breach and assigns them to graduated lanes with different capacity limits.

The existing HiLoQueryLaningStrategy uses a simple binary high/low split — any single threshold breach sends a query to the low lane. This misclassifies queries that are expensive on one dimension but cheap on others (e.g., many segments but cheap per-segment). The weighted strategy provides more nuanced lane assignment: a query breaching 1 threshold gets a different lane than one breaching 4.

Same threshold types as ThresholdBasedQueryPrioritizationStrategy:

  • periodThreshold — how far in the past the data is
  • durationThreshold — how large the query interval is
  • segmentCountThreshold — number of segments involved
  • segmentRangeThreshold — total segment time range

Each breach adds 1 to the query's score (this can be changed to make scoring different for each threshold). The query is assigned to the lane with the highest minScore it meets. Score 0 = no lane (runs in default pool). Existing caller-set lanes in query context are preserved.

Example config

{
"laning": {
"strategy": "weighted",
"periodThreshold": "P1M",
"durationThreshold": "P1D",
"segmentCountThreshold": 1000,
"segmentRangeThreshold": "P6M",
"lanes": {
"low": { "minScore": 1, "maxPercent": 30 },
"very-low": { "minScore": 3, "maxPercent": 10 }
}
}
}

A query breaching 2 thresholds (score=2) goes to low (30% capacity cap). A query breaching all 4 (score=4) goes to very-low (10% cap). When a lane is full, excess queries get HTTP 429.

Release note

New query laning strategy weighted that scores queries by how many thresholds they breach (segment count, interval duration, data age, segment range) and assigns them to configurable graduated lanes with different capacity limits, providing more nuanced lane assignment than the existing binary hilo strategy.


Key changed/added classes in this PR
  • WeightedQueryLaningStrategy
  • QueryLaningStrategy
  • WeightedQueryLaningStrategyTest
  • QuerySchedulerTest

[can add to docs once reviewed]

This PR has:

  • been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@jtuglu1 jtuglu1 self-requested a review April 8, 2026 19:59

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 4 of 4 changed files.


This is an automated review by Codex GPT-5.5

@jtuglu1 jtuglu1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, few minor comments

@jtuglu1 jtuglu1 self-requested a review May 22, 2026 19:20

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 4 of 4 changed files.


This is an automated review by Codex GPT-5.5

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 4 of 4 changed files.


This is an automated review by Codex GPT-5.5

@mshahid6 mshahid6 force-pushed the weighted-query-laning branch from 600d085 to 4a5d7e2 Compare June 22, 2026 20:27
@mshahid6 mshahid6 requested a review from FrankChen021 June 25, 2026 18:05
@jtuglu1 jtuglu1 added this to the 38.0.0 milestone Jun 26, 2026

@jtuglu1 jtuglu1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Largely LGTM. Could we add docs for this strategy as well?

@jtuglu1 jtuglu1 self-requested a review July 3, 2026 00:09

@jtuglu1 jtuglu1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good – one last thing then ✅

Comment thread docs/configuration/index.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants