Skip to content

feat(recipes): add explainable-routing recipe and example (#115)#128

Merged
linfangw merged 16 commits into
mainfrom
feat/explainable-routing-recipe
Jul 7, 2026
Merged

feat(recipes): add explainable-routing recipe and example (#115)#128
linfangw merged 16 commits into
mainfrom
feat/explainable-routing-recipe

Conversation

@linfangw

@linfangw linfangw commented Jul 7, 2026

Copy link
Copy Markdown
Member

Closes #115. Advances the explainable-routing sub-task of #65.

What

A recipe that makes a transparent, cost-aware capability choice — the QVeris differentiator in practice. Instead of blindly taking the first result, the agent compares candidates on the routing signals QVeris returns (why_recommended, expected_cost, success rate, latency), selects one, and explains the decision before spending credits.

Contents

  • recipes/explainable-routing/README.md (CLI + Python SDK paths) and a qveris.manifest.json (validated by scripts/validate-ecosystem-manifests.mjs, 6/6 ok).
  • packages/python-sdk/examples/explainable_routing.py — prints a candidate comparison table, then applies two bounded overrides on top of the backend ranking:
    1. Cost saving — a much cheaper candidate (≤50% cost) that is no less reliable.
    2. Reliability upgrade — a candidate that costs no more but is meaningfully more reliable (≥5 pts higher success).
      It never trades a large cost increase for reliability. The choose(...) helper is copy-pasteable into a real agent loop.
  • Indexed in recipes/README.md and the python-sdk examples tables (en-US / zh-CN / cn/zh-CN) + package README.

Verification

Ran live against production (discovery only, no credits spent):

Selected: Quote
Reason:   chose a more reliable capability at no extra cost — 73.1% vs 59.2% success for the same ~1 credits.
  • python -m compileall qveris examples clean (matches the publish workflow step)
  • node scripts/validate-ecosystem-manifests.mjs — 6/6 ok
  • CLI path verified: qveris discover --json passes through why_recommended / expected_cost / stats for the jq comparison

Compatibility

Requires the routing fields shipped in #102: python_sdk >=0.2.1, cli >=0.6.0, mcp >=0.7.0 (declared in the manifest).

linfangw added 15 commits May 19, 2026 20:30
Adds a recipe that makes a transparent, cost-aware capability choice:
discover several candidates, compare them on the routing signals QVeris
returns (why_recommended, expected_cost, success rate, latency), select
one with a bounded cost/reliability rule, and explain the decision.

- recipes/explainable-routing/ (README.md + validated qveris.manifest.json)
- packages/python-sdk/examples/explainable_routing.py — prints a candidate
  comparison and applies two overrides on the backend ranking: a cost-saving
  pick (much cheaper, no less reliable) and a reliability upgrade (no more
  expensive, meaningfully more reliable); never trades a large cost increase
  for reliability
- indexed in recipes/README.md and the python-sdk examples tables (en/zh/cn)

Advances #65 (explainable routing sub-task); uses why_recommended and
expected_cost shipped in #102.
@linfangw linfangw requested a review from ax2 July 7, 2026 13:38

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an 'Explainable Routing' recipe and a runnable Python example (explainable_routing.py) that demonstrates how to discover, compare, and select candidate capabilities based on cost and reliability signals. It also updates the relevant documentation across multiple languages to include this new recipe. The review feedback highlights two important issues in the selection logic: first, the choose function should gracefully handle empty lists to prevent an IndexError; second, defaulting missing success rates to 0.0 can lead to incorrect routing decisions, so the algorithm should skip overrides when quality statistics are missing.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Both keep spend bounded: we never trade a large cost increase for
reliability. Anything else falls back to the backend's top pick.
"""
top = tools[0]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The choose function assumes that the tools list is non-empty. If an empty list is passed, it will raise an IndexError at top = tools[0]. Since this helper is designed to be copy-pasteable into real agent loops, it should handle the empty list case gracefully (e.g., by raising a clear ValueError or returning a default value).

Suggested change
top = tools[0]
if not tools:
raise ValueError("The tools list cannot be empty.")
top = tools[0]

Comment thread packages/python-sdk/examples/explainable_routing.py Outdated
…LI flags

- Add `from __future__ import annotations` so the list[]/tuple[] hints
  don't raise TypeError on Python 3.8/3.9 at import (requires-python is
  >=3.8; compileall + the 3.10-3.12 CI matrix missed it). Matches the
  stock_debate.py precedent.
- choose(): keep an unknown success_rate as None instead of coercing to
  0.0 — a None side now skips the cost/reliability comparison, so the
  explanation never claims '0.0% success' for a tool the table shows as
  'n/a', and an unknown top reliability no longer trips a false upgrade.
- README: usage summary is the default (dropped the ignored --summary
  flag); use the canonical --discovery-id for inspect/call instead of
  the deprecated --search-id alias.
@linfangw

linfangw commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Addressed the /review findings (commit 42120eb):

  1. [CRITICAL] Python 3.8 import break — the list[ToolInfo] / tuple[...] annotations are evaluated at def-time and would raise TypeError: 'type' object is not subscriptable on 3.8/3.9 (PEP 585 is 3.9+), which compileall (syntax-only) and the 3.10–3.12 CI matrix both miss. Added from __future__ import annotations (matching the stock_debate.py precedent) — annotations are now lazy strings, safe on the declared >=3.8 floor.
  2. [IMPORTANT] success_rate or 0.0 collapsed unknown→0.0 — the reason string could claim "0.0% success" for a tool the comparison table shows as "n/a", and an unknown top reliability could trip a false "reliability upgrade". choose() now keeps None distinct and skips the cost/reliability comparison when either side's success is unknown, falling back to the backend pick.
  3. [IMPORTANT] qveris usage --summary is a no-op (summary is the default; the flag is silently ignored) — dropped it.
  4. [SUGGESTION] --search-id on inspect/call is a deprecated alias — switched the CLI examples to the canonical --discovery-id (value still comes from discover's search_id).

Re-verified: compileall clean, validate-ecosystem-manifests 6/6, and a live production run still produces the correct explanation (73.1% vs 59.2% success for the same ~1 credits) with real percentages — no None contradiction.

@linfangw linfangw merged commit 22a8ba9 into main Jul 7, 2026
1 check passed
@linfangw linfangw deleted the feat/explainable-routing-recipe branch July 7, 2026 14:24
linfangw added a commit that referenced this pull request Jul 8, 2026
)

The CLI's usage/ledger commands select output shape via --mode
(summary|search|export_file), and summary is the default. --summary is
not a recognized flag: it falls through to positional args and is
silently ignored. All five recipes' CLI audit examples passed
--summary, teaching a flag that does nothing.

Drop it (summary is the default) across crypto-monitoring,
data-analysis, developer-automation, finance-research, and
risk-compliance. Surfaced by the #128 review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2][Ecosystem] Recipe: explainable routing with why_recommended

2 participants