Skip to content

Commit bdac59c

Browse files
authored
Merge pull request #66 from leonardo1229/issue/fix-65
feat: add rule engine integration, risk labels, load balancing, and missing risk signals
2 parents e783f1a + 1e40455 commit bdac59c

19 files changed

Lines changed: 3409 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,28 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
88

99
### Added
1010

11+
- **AI-powered reviewer recommendation** -- `/reviewers` slash command suggests
12+
the best reviewers for a PR based on CODEOWNERS ownership, commit history
13+
expertise, Watchflow rule severity, and current review load. Supports
14+
`--force` flag to bypass cooldown. Recommended reviewers are automatically
15+
assigned to the PR via the GitHub API.
16+
- **PR risk assessment** -- `/risk` slash command posts a detailed risk
17+
breakdown (size, sensitive paths, test coverage, contributor history, revert
18+
detection, dependency changes, breaking changes, and matched Watchflow rule
19+
severity). Applies `watchflow:risk-{level}` labels automatically.
20+
- **Contributor expertise profiles** -- reviewer expertise is persisted to
21+
`.watchflow/expertise.json` across PRs and used to boost candidates with
22+
cross-PR historical ownership.
23+
- **CODEOWNERS + rule integration** -- CODEOWNERS individual users and
24+
`@org/team` entries are handled separately; team slugs are passed to
25+
GitHub's `team_reviewers` API field to prevent 422 errors. When no
26+
CODEOWNERS exists, high/critical Watchflow rule path matches infer implicit
27+
ownership from commit history.
28+
- **Load balancing** -- reviewers with heavy recent review queues are
29+
penalised; reviewer count scales with risk level (low→1, medium→2,
30+
high/critical→3). Stale CODEOWNERS owners (no recent commits) receive a
31+
reduced score.
32+
1133
- **Description-diff alignment** -- `DescriptionDiffAlignmentCondition` uses
1234
the configured AI provider (OpenAI / Bedrock / Vertex AI) to verify that
1335
the PR description semantically matches the actual code changes. First

docs/getting-started/quick-start.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,46 @@ Parameter names must match the [supported conditions](configuration.md); see [Co
8787
|--------|--------|
8888
| `@watchflow acknowledge "reason"` / `@watchflow ack "reason"` | Record an acknowledgment for a violation (when the rule allows it). |
8989
| `@watchflow evaluate "rule in plain English"` | Ask whether a rule is feasible and get suggested YAML. |
90+
| `/risk` | Run a risk analysis on the PR and post a signal summary (file churn, ownership gaps, rule violations). |
91+
| `/reviewers` | Get AI-powered reviewer recommendations based on code ownership, commit history, and risk signals. |
9092
| `@watchflow help` | List commands. |
9193

9294
---
9395

96+
## Try it: risk analysis and reviewer recommendations
97+
98+
Once Watchflow is installed and `.watchflow/rules.yaml` is in place, open a pull request and post a comment:
99+
100+
```
101+
/risk
102+
```
103+
104+
Watchflow will reply with a breakdown of risk signals — for example:
105+
106+
> **Risk signals detected (2)**
107+
> - `src/auth/jwt.py` modified — no matching test file updated (medium)
108+
> - PR exceeds 500 lines changed (medium)
109+
>
110+
> **Active rules evaluated:** 7 · **Violations:** 2
111+
112+
Then ask for reviewer suggestions:
113+
114+
```
115+
/reviewers
116+
```
117+
118+
Watchflow analyses commit history, CODEOWNERS, and the risk signals, then replies with ranked recommendations:
119+
120+
> **Recommended reviewers**
121+
> 1. `@alice` — recent commits to `src/auth/jwt.py`, CODEOWNERS owner of `src/auth/`
122+
> 2. `@bob` — top contributor to `src/auth/` over the last 90 days
123+
>
124+
> *Tip: add a reviewer with `gh pr edit --add-reviewer alice`.*
125+
126+
You can see a working example of both commands against a real repo at [test-watchflow](https://github.com/warestack/test-watchflow).
127+
128+
---
129+
94130
## Next steps
95131

96132
- **Tune rules** — [Configuration](configuration.md) for parameter reference and examples.

src/agents/factory.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
from src.agents.extractor_agent import RuleExtractorAgent
1515
from src.agents.feasibility_agent import RuleFeasibilityAgent
1616
from src.agents.repository_analysis_agent import RepositoryAnalysisAgent
17+
from src.agents.reviewer_recommendation_agent import ReviewerRecommendationAgent
1718

1819
logger = logging.getLogger(__name__)
1920

@@ -51,6 +52,10 @@ def get_agent(agent_type: str, **kwargs: Any) -> BaseAgent:
5152
return AcknowledgmentAgent(**kwargs)
5253
elif agent_type == "repository_analysis":
5354
return RepositoryAnalysisAgent(**kwargs)
55+
elif agent_type == "reviewer_recommendation":
56+
return ReviewerRecommendationAgent()
5457
else:
55-
supported = ", ".join(["engine", "feasibility", "extractor", "acknowledgment", "repository_analysis"])
58+
supported = ", ".join(
59+
["engine", "feasibility", "extractor", "acknowledgment", "repository_analysis", "reviewer_recommendation"]
60+
)
5661
raise ValueError(f"Unsupported agent type: {agent_type}. Supported: {supported}")
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
from src.agents.reviewer_recommendation_agent.agent import ReviewerRecommendationAgent
2+
3+
__all__ = ["ReviewerRecommendationAgent"]
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# File: src/agents/reviewer_recommendation_agent/agent.py
2+
3+
from typing import Any
4+
5+
import structlog
6+
from langgraph.graph import END, StateGraph
7+
8+
from src.agents.base import AgentResult, BaseAgent
9+
from src.agents.reviewer_recommendation_agent import nodes
10+
from src.agents.reviewer_recommendation_agent.models import RecommendationState
11+
12+
logger = structlog.get_logger()
13+
14+
15+
class ReviewerRecommendationAgent(BaseAgent):
16+
"""
17+
Agent that recommends reviewers for a PR based on:
18+
1. CODEOWNERS ownership of changed files
19+
2. Commit history expertise (who recently touched the same files)
20+
3. Deterministic risk assessment (file count, sensitive paths, contributor status)
21+
4. LLM-powered ranking with natural-language reasoning
22+
23+
Outputs both a risk breakdown and ranked reviewer suggestions.
24+
"""
25+
26+
def __init__(self) -> None:
27+
super().__init__(agent_name="reviewer_recommendation")
28+
29+
def _build_graph(self) -> Any:
30+
workflow: StateGraph[RecommendationState] = StateGraph(RecommendationState)
31+
32+
llm = self.llm
33+
34+
async def _recommend_reviewers(state: RecommendationState) -> RecommendationState:
35+
return await nodes.recommend_reviewers(state, llm)
36+
37+
workflow.add_node("fetch_pr_data", nodes.fetch_pr_data)
38+
workflow.add_node("assess_risk", nodes.assess_risk)
39+
workflow.add_node("recommend_reviewers", _recommend_reviewers)
40+
41+
workflow.set_entry_point("fetch_pr_data")
42+
workflow.add_edge("fetch_pr_data", "assess_risk")
43+
workflow.add_edge("assess_risk", "recommend_reviewers")
44+
workflow.add_edge("recommend_reviewers", END)
45+
46+
return workflow.compile()
47+
48+
async def execute(self, **kwargs: Any) -> AgentResult:
49+
"""
50+
Args:
51+
repo_full_name: str — owner/repo
52+
pr_number: int — PR number
53+
installation_id: int — GitHub App installation ID
54+
"""
55+
repo_full_name: str | None = kwargs.get("repo_full_name")
56+
pr_number: int | None = kwargs.get("pr_number")
57+
installation_id: int | None = kwargs.get("installation_id")
58+
59+
if not repo_full_name or not pr_number or not installation_id:
60+
return AgentResult(success=False, message="repo_full_name, pr_number, and installation_id are required")
61+
62+
initial_state = RecommendationState(
63+
repo_full_name=repo_full_name,
64+
pr_number=pr_number,
65+
installation_id=installation_id,
66+
)
67+
68+
try:
69+
result = await self._execute_with_timeout(self.graph.ainvoke(initial_state), timeout=45.0)
70+
final_state = RecommendationState(**result) if isinstance(result, dict) else result
71+
72+
if final_state.error:
73+
return AgentResult(success=False, message=final_state.error)
74+
75+
return AgentResult(
76+
success=True,
77+
message="Recommendation complete",
78+
data={
79+
"risk_level": final_state.risk_level,
80+
"risk_score": final_state.risk_score,
81+
"risk_signals": [s.model_dump() for s in final_state.risk_signals],
82+
"candidates": [c.model_dump() for c in final_state.candidates],
83+
"llm_ranking": final_state.llm_ranking.model_dump() if final_state.llm_ranking else None,
84+
"pr_files_count": len(final_state.pr_files),
85+
"pr_author": final_state.pr_author,
86+
"codeowners_team_slugs": final_state.codeowners_team_slugs,
87+
"pr_base_branch": final_state.pr_base_branch,
88+
},
89+
)
90+
91+
except TimeoutError:
92+
logger.error("agent_execution_timeout", agent="reviewer_recommendation", repo=repo_full_name)
93+
return AgentResult(success=False, message="Recommendation timed out after 45 seconds")
94+
except Exception as e:
95+
logger.exception("agent_execution_failed", agent="reviewer_recommendation", error=str(e))
96+
return AgentResult(success=False, message=str(e))
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# File: src/agents/reviewer_recommendation_agent/models.py
2+
3+
from typing import Any
4+
5+
from pydantic import BaseModel, Field
6+
7+
8+
class ReviewerCandidate(BaseModel):
9+
"""A candidate reviewer with a score and reasons for recommendation."""
10+
11+
username: str
12+
score: int = 0
13+
ownership_pct: int = 0 # % of changed files they own or recently touched
14+
reasons: list[str] = Field(default_factory=list)
15+
16+
17+
class RiskSignal(BaseModel):
18+
"""A single contributing factor to the PR risk score."""
19+
20+
label: str
21+
description: str
22+
points: int
23+
24+
25+
class RankedReviewer(BaseModel):
26+
"""A single reviewer entry in the LLM ranking output."""
27+
28+
username: str = Field(description="GitHub username of the reviewer")
29+
reason: str = Field(description="Short explanation of why this reviewer is recommended")
30+
31+
32+
class LLMReviewerRanking(BaseModel):
33+
"""Structured output from the LLM reviewer ranking step."""
34+
35+
ranked_reviewers: list[RankedReviewer] = Field(description="Ordered list of reviewers, best match first")
36+
summary: str = Field(description="One-line overall recommendation summary")
37+
38+
39+
class RecommendationState(BaseModel):
40+
"""Shared state (blackboard) for the ReviewerRecommendationAgent graph."""
41+
42+
# --- Inputs ---
43+
repo_full_name: str
44+
pr_number: int
45+
installation_id: int
46+
47+
# --- Collected Data ---
48+
pr_files: list[str] = Field(default_factory=list)
49+
pr_author: str = ""
50+
pr_additions: int = 0
51+
pr_deletions: int = 0
52+
pr_commits_count: int = 0
53+
pr_author_association: str = "NONE"
54+
codeowners_content: str | None = None
55+
contributors: list[dict[str, Any]] = Field(default_factory=list)
56+
# file_path -> list of recent committer logins
57+
file_experts: dict[str, list[str]] = Field(default_factory=dict)
58+
# Matched Watchflow rules (description, severity) loaded from .watchflow/rules.yaml
59+
matched_rules: list[dict[str, str]] = Field(default_factory=list)
60+
# Recent review activity: login -> count of reviews on recent PRs (for load balancing)
61+
reviewer_load: dict[str, int] = Field(default_factory=dict)
62+
# Reviewer acceptance rates: login -> approval rate (0.0–1.0) from recent PRs
63+
reviewer_acceptance_rates: dict[str, float] = Field(default_factory=dict)
64+
# PR title (for revert detection)
65+
pr_title: str = ""
66+
67+
# --- Risk Assessment ---
68+
risk_score: int = 0
69+
risk_level: str = "low" # low / medium / high / critical
70+
risk_signals: list[RiskSignal] = Field(default_factory=list)
71+
72+
# --- Recommendations ---
73+
candidates: list[ReviewerCandidate] = Field(default_factory=list)
74+
llm_ranking: LLMReviewerRanking | None = None
75+
76+
# PR base branch (used when writing .watchflow/expertise.json)
77+
pr_base_branch: str = "main"
78+
# Team slugs extracted from CODEOWNERS (@org/team entries) — used to split
79+
# reviewer assignment into `reviewers` vs `team_reviewers` GitHub API fields
80+
codeowners_team_slugs: list[str] = Field(default_factory=list)
81+
# Persisted expertise profiles loaded from .watchflow/expertise.json
82+
expertise_profiles: dict[str, Any] = Field(default_factory=dict)
83+
84+
# --- Execution Metadata ---
85+
error: str | None = None

0 commit comments

Comments
 (0)