Skip to content

Strengthen deterministic agent workflow routing#381

Open
michaelmwu wants to merge 6 commits into
mainfrom
michaelmwu/review-agent-smartness
Open

Strengthen deterministic agent workflow routing#381
michaelmwu wants to merge 6 commits into
mainfrom
michaelmwu/review-agent-smartness

Conversation

@michaelmwu

@michaelmwu michaelmwu commented Jul 9, 2026

Copy link
Copy Markdown
Member

Summary

  • route explicitly supported agent workflows through deterministic parsing before model planning
  • preserve structured planning for genuinely ambiguous requests
  • add regression coverage that clear account provisioning cannot be downgraded by a model proposal

Validation

  • uv run --locked pytest -q tests/unit/test_agent_gateway.py tests/unit/test_agent_evals.py tests/unit/test_agent_planner.py
  • uv run --locked python scripts/agent_eval.py --suite canonical --model primary --no-env-file --json
  • ./scripts/lint.sh
  • ./scripts/pyrefly.sh

Note

Medium Risk
Changes core request routing for provisioning and other regex-mapped workflows; wrong ordering could skip or mis-route high-impact account actions, though tests target the main regression.

Overview
Deterministic routing now wins when regex can map a message to a supported tool, so explicit flows like 508 account provisioning no longer go through the structured planner first (avoiding model proposals that could override a clear command).

AgentOrchestrator.plan() parses with _parse_action and returns via new _response_for_deterministic_action before calling _plan_with_model. Ambiguous messages still use the live model planner; intent normalization and clarification paths are unchanged for unparsed input.

Structured planner exceptions now emit a warning log (model_tier, error_type) before falling back, without exposing provider details to users.

Unit tests cover planner bypass for known workflows, observable planner failures, and structured planning for genuinely ambiguous onboarding-style requests.

Reviewed by Cursor Bugbot for commit 2ee1eff. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor

cursor Bot commented Jul 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_7770c897-b51a-4e91-8e43-2b412716448a)

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@michaelmwu, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 33 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0ff7064d-86df-4611-b742-146e078fece6

📥 Commits

Reviewing files that changed from the base of the PR and between 19c4a7d and 2ee1eff.

📒 Files selected for processing (2)
  • packages/shared/src/five08/agent/orchestrator.py
  • tests/unit/test_agent_gateway.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch michaelmwu/review-agent-smartness

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Jul 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_d0cae15a-af1f-46d4-94de-48af1910e99e)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 68f00d3ef3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +126 to +127
if action is not None:
return self._response_for_deterministic_action(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep explicit multi-action requests on the planner path

When a configured planner receives an explicit multi-step request where the first step matches the regex parser, e.g. Create a task to update docs and invite Sarah to Outline, this early return bypasses _plan_with_model after parsing only one AgentToolAction. The deterministic parser can only return a single action, so the later requested workflow is silently dropped instead of being planned as the multi-action confirmation path covered by the structured planner.

Useful? React with 👍 / 👎.

Comment on lines +126 to +127
if action is not None:
return self._response_for_deterministic_action(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clarify missing GitHub repos before executing searches

In deployments where GITHUB_DEFAULT_REPO is not configured, an explicit request like Search GitHub issues matching onboarding now hits this deterministic early return, so the read action executes immediately and fails from ToolRegistry._resolve_repository with “GitHub repository is required”. The structured planner path still has _planner_action_clarification to ask “Which GitHub repository should I search?”, but this bypass means configured-planner users get a failed execution instead of the intended clarification whenever they omit the repo.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant