Skip to content

Add LLM semantic routing example#2486

Merged
howardjohn merged 10 commits into
agentgateway:mainfrom
danehans:llm-semantic-routing-example
Jul 10, 2026
Merged

Add LLM semantic routing example#2486
howardjohn merged 10 commits into
agentgateway:mainfrom
danehans:llm-semantic-routing-example

Conversation

@danehans

@danehans danehans commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add an end-to-end example for cost-based semantic routing of OpenAI LLM traffic through agentgateway and vLLM Semantic Router.
  • Include Kubernetes manifests, router values, and tooling.
  • Document prereqs, deployment, and evaluation workflows.

danehans added 5 commits July 10, 2026 13:39
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans
danehans force-pushed the llm-semantic-routing-example branch from 9fe0326 to b9f78b2 Compare July 10, 2026 20:39
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans
danehans marked this pull request as ready for review July 10, 2026 21:39
@danehans
danehans requested a review from a team as a code owner July 10, 2026 21:39
danehans added 2 commits July 10, 2026 14:44
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Comment thread examples/llm-semantic-routing/README.md Outdated
Comment thread examples/llm-semantic-routing/README.md Outdated
Comment thread examples/llm-semantic-routing/README.md Outdated
Comment thread examples/llm-semantic-routing/README.md Outdated
Comment thread examples/llm-semantic-routing/README.md
Comment thread examples/llm-semantic-routing/k8s/semantic-router-values.yaml
danehans added 2 commits July 10, 2026 15:55
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@howardjohn
howardjohn merged commit d46bc5c into agentgateway:main Jul 10, 2026
10 checks passed
@danehans
danehans deleted the llm-semantic-routing-example branch July 10, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants