Skip to content

blog: measure semantic routing costs#733

Closed
danehans wants to merge 15 commits into
agentgateway:mainfrom
danehans:blog-semantic-routing-costs
Closed

blog: measure semantic routing costs#733
danehans wants to merge 15 commits into
agentgateway:mainfrom
danehans:blog-semantic-routing-costs

Conversation

@danehans

@danehans danehans commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • focus the post on a compact two-lane cost-based semantic routing example
  • include the validated catalog-priced cost, routing, latency, and cache-transition evaluation details
  • explain how the model mix and complex-prompt escalation demonstrate savings without a blanket downgrade

Observed run

  • 100 primary requests with exact agentgateway model-catalog lookups
  • $0.916940 always-expensive versus $0.375954 routed: 59.0% lower spend
  • 38 nano and 12 GPT-5.5 selections; 12 of 25 complex prompts escalated to GPT-5.5

Validation

  • hugo --minify --destination <temporary-directory> --cleanDestinationDir with the bundled Node runtime

@danehans danehans changed the title blog: measure semantic routing outcome quality blog: measure semantic routing costs Jul 13, 2026
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans
danehans force-pushed the blog-semantic-routing-costs branch from e72019d to f4aa0e1 Compare July 13, 2026 22:42
danehans added 5 commits July 14, 2026 10:06
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans
danehans marked this pull request as ready for review July 14, 2026 17:41
@danehans

danehans commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

Marking as "draft" status since this PR requires agentgateway/agentgateway#2532 (Codex client expects streaming API responses).

danehans added 9 commits July 14, 2026 12:00
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans

Copy link
Copy Markdown
Collaborator Author

Superseded by #750, which contains the cleaned final post and will receive the preview.

@danehans danehans closed this Jul 16, 2026
@danehans
danehans deleted the blog-semantic-routing-costs branch July 16, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant