Skip to content

[Feature] CostGuard scorer skeleton#243

Merged
github-actions[bot] merged 2 commits into
llm-d:mainfrom
davidbreitgand:costguard-scorer-skeleton
Jul 16, 2026
Merged

[Feature] CostGuard scorer skeleton#243
github-actions[bot] merged 2 commits into
llm-d:mainfrom
davidbreitgand:costguard-scorer-skeleton

Conversation

@davidbreitgand

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR is the first of a four-PR series that lands the CostGuard scorer plugin (see README for rationale, algorithm, and design).

Each PR compiles, runs, passes its own tests, and adds a bounded, well-scoped increment.

PR 1 (this PR): Plugin skeleton, config, factory, runner registration.
Lands the framework plumbing: PluginType, Config with full validation, factory, etc., and runner registration. Score returns a neutral 0.5 for every model, which is a valid "no opinion" state that keeps the scorer registerable and pipeline-safe. windowDuration parameter is parsed and validated but not consumed until PR 4 (marked with TODO in the code)

Future PRs outline (for convenience of reference and scope of work:

PR2 and PR3 are two halves of the same algorithm that must land together (eventually) to be useful.

PR 2: Exploitation branch (rank + sigmoid scoring).
Introduces the deterministic ranking half of the algorithm: reads each model's CostDigest from the datalayer, computes its intermediate rank, and applies a temperature-scaled sigmoid for obtaining the score in (0, 1). Models without cost history continue to receive the neutral 0.5. PR2 is independently reviewable but not deployable in isolation since the scorer requires the exploration branch from PR 3 to bootstrap its own cost data collection.

PR 3 — Exploration branch (epsilon-greedy).
Exploration branch (epsilon-greedy).
Adds the random exploration branch that drives sample collection on under-explored models. Together with PR 2, this completes the scoring algorithm. After PR 3 lands, the scorer is usable but not production-grade.

PR 4 — Epoch/window semantics (extractor owns the window).
Extends the requestcostmetadata extractor to archive and reset its per-model t-digest each windowDuration, so published snapshots reflect only the current epoch. Removes WindowDuration from the scorer's Config (the extractor becomes the single source of truth) and updates both plugin READMEs.

Refs #214

Release note (write NONE if no user-facing change):

NONE

@github-actions github-actions Bot added kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 15, 2026
@davidbreitgand
davidbreitgand force-pushed the costguard-scorer-skeleton branch from bc46d2c to 7773044 Compare July 15, 2026 19:06
@davidbreitgand

Copy link
Copy Markdown
Contributor Author

/cc @shmuelk

@github-actions
github-actions Bot requested a review from shmuelk July 15, 2026 19:08
Comment thread pkg/framework/plugins/modelselector/scorer/costguard/plugin.go Outdated
Comment thread pkg/framework/plugins/modelselector/scorer/costguard/plugin.go Outdated
Comment thread pkg/framework/plugins/modelselector/scorer/costguard/plugin.go Outdated
Comment thread pkg/framework/plugins/modelselector/scorer/costguard/plugin.go Outdated
Signed-off-by: David Breitgand <davidbreitgand@users.noreply.github.com>
Co-authored-by: Shmuel Kallner <kallner@il.ibm.com>
@davidbreitgand
davidbreitgand force-pushed the costguard-scorer-skeleton branch from 3f3c266 to 19739c5 Compare July 16, 2026 12:29
@shmuelk

shmuelk commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

/lgtm
/approve

@github-actions github-actions Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2026
Signed-off-by: David Breitgand <davidbreitgand@users.noreply.github.com>
@github-actions
github-actions Bot merged commit 454756a into llm-d:main Jul 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants