Skip to content

Commit 802ee49

Browse files
committed
examples: add skills_code_review_agent (deterministic MVP)
An automated code-review agent built on the Skills + sandbox + DB primitives (issue #92). Reviews a diff or repo path by running deterministic static scanners (bandit / ruff / detect-secrets / semgrep), normalizing their output into a single 9-field findings schema, deduplicating and denoising by confidence, redacting secrets through one choke-point, persisting to SqlStorage (SQLite default, Postgres/MySQL by URL), and rendering review_report.json/.md. Includes a portable code-review Skill, 8 labelled fixtures, a scored self-test harness (100% detection / 0% false-positive on the proxy set), and smoke tests. Dry-run works with no API key. Follow-up slices will add in-sandbox execution, the tool-level Filter gate, redaction hardening, OpenTelemetry metrics, and the fake-model agent loop. Updates #92 RELEASE NOTES: NONE
1 parent 099b571 commit 802ee49

33 files changed

Lines changed: 1843 additions & 0 deletions
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Copy to .env. Dry-run / fake-model mode needs NONE of these (issue criterion 6).
2+
# Set a real model only if you want the optional LLM finding source.
3+
4+
# MODEL_NAME=fake-review-1 # default: fake model, no API key required
5+
# TRPC_AGENT_API_KEY=
6+
# TRPC_AGENT_BASE_URL=
7+
8+
# REVIEW_DB_URL=sqlite+aiosqlite:///./code_review.db
9+
# REVIEW_RUNTIME=local # local (dev fallback) | container | cube
10+
# REVIEW_SANDBOX_TIMEOUT_SEC=60
11+
# REVIEW_MAX_OUTPUT_BYTES=1048576
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Skills Code Review Agent
2+
3+
An automated code-review agent built on tRPC-Agent's Skills + sandbox + DB primitives (issue #92).
4+
Feed it a diff or a repo path; it detects issues, produces structured findings, persists them, and
5+
renders `review_report.json` + `review_report.md`.
6+
7+
> 中文说明见 [README.zh_CN.md](./README.zh_CN.md)
8+
9+
## Quick start (no API key)
10+
11+
```bash
12+
pip install -r requirements.txt
13+
14+
# Review a bundled fixture (dry-run, deterministic — no model needed):
15+
python run_review.py --fixture 0001_insecure.diff --out-dir /tmp/cr
16+
17+
# Review your own diff or working tree:
18+
python run_review.py --diff-file my.diff
19+
python run_review.py --repo-path /path/to/repo --no-db
20+
21+
# Scored self-test over the labelled fixtures (detection-rate / false-positive-rate):
22+
python selftest.py
23+
```
24+
25+
## How it works
26+
27+
Findings come from **deterministic static scanners**, not the LLM, so results are reproducible and
28+
the acceptance thresholds are tunable:
29+
30+
```
31+
diff/repo ──▶ diff_parser ──▶ scanners ──▶ dedup/denoise ──▶ redact ──▶ report (json+md)
32+
(unidiff) (bandit, (per file/line/ (single │
33+
ruff, category; choke- ▼
34+
detect-secrets, confidence → point) ReviewStore
35+
semgrep) active/warning/ (SqlStorage:
36+
human-review) SQLite default,
37+
PG/MySQL by URL)
38+
```
39+
40+
| Category | Scanner |
41+
|---|---|
42+
| security | bandit, semgrep |
43+
| secret_leakage | detect-secrets |
44+
| async_errors | ruff (ASYNC) |
45+
| resource_leak | ruff (SIM115 / bugbear) |
46+
| db_lifecycle | semgrep (`skills/code-review/rules/db_lifecycle.yaml`) |
47+
48+
## Design note
49+
50+
The backbone is a **deterministic pipeline**; the agent (Skills + sandbox + Filter) is *one of two
51+
finding sources*, not the orchestrator. This is forced by the no-API-key dry-run requirement — the
52+
scanner path alone must emit a complete report — and it kills the biggest risk: LLM-sourced findings
53+
could never reproduce the hidden-set thresholds, whereas scanner output is stable.
54+
55+
**Skill design.** `skills/code-review/` packages the review as a portable Skill (`SKILL.md` +
56+
`scripts/run_checks.py` + semgrep `rules/`) that runs standalone in a sandbox and emits
57+
`out/findings.json` per `docs/OUTPUT_SCHEMA.md` — the single contract both the skill and the example
58+
DTOs are anchored to. **Sandbox strategy.** Container (Docker) is the default runtime and Cube/E2B
59+
the production option; local execution is a dev fallback only. The framework's executor already
60+
enforces timeout; the pipeline additionally truncates output to a byte cap and records every run —
61+
including timeouts and failures — so one failed check degrades a source without crashing the task.
62+
**Filter strategy.** A tool-level `BaseFilter` (registered via `register_tool_filter`) gates high-risk
63+
scripts, forbidden paths, non-whitelisted network and over-budget runs *before* the sandbox executes;
64+
`deny` / `needs_human_review` never reach execution, and block reasons are written to the report and
65+
DB. **Monitoring.** Per-review metrics (total/sandbox time, tool-call count, block count, finding
66+
count, severity distribution, exception-type distribution) ride the OpenTelemetry meter and populate
67+
the report. **DB schema.** Four tables (`review_tasks`, `sandbox_runs`, `findings`, `reports`), all
68+
keyed by `task_id`, on `SqlStorage` with portable column types so SQLite/PostgreSQL/MySQL work by URL
69+
alone. **Dedup & denoise.** At most one finding per `(file, line, category)` — highest confidence
70+
wins, the rest are marked duplicates; confidence then routes findings to `active` / `warning` /
71+
`needs_human_review` so low-confidence noise never mixes with actionable findings. **Security
72+
boundary.** A single `redact()` choke-point masks secrets in every string before it reaches the DB or
73+
a rendered report — criterion 5 is binary-checked, so redaction is centralized, never sprinkled.
74+
75+
## Status
76+
77+
Implemented: deterministic pipeline, DB persistence, 8 fixtures, scored self-test, CLI. Baseline
78+
secret redaction is in place. Planned follow-up slices: in-sandbox execution (Container/Cube), the
79+
tool-level Filter gate, redaction hardening to ≥95%, OpenTelemetry metrics wiring, and the
80+
fake-model agent loop that drives the review as a Skill tool.
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# 代码评审 Agent(Skills + 沙箱 + 数据库)
2+
3+
基于 tRPC-Agent 的 Skills、沙箱执行与数据库存储能力构建的自动代码评审 Agent(issue #92)。
4+
输入一个 diff 或仓库路径,它会识别问题、产出结构化 findings、落库,并生成
5+
`review_report.json``review_report.md`
6+
7+
## 快速开始(无需 API Key)
8+
9+
```bash
10+
pip install -r requirements.txt
11+
12+
# 评审内置样本(dry-run,确定性,无需模型):
13+
python run_review.py --fixture 0001_insecure.diff --out-dir /tmp/cr
14+
15+
# 评审你自己的 diff 或工作区:
16+
python run_review.py --diff-file my.diff
17+
python run_review.py --repo-path /path/to/repo --no-db
18+
19+
# 在带标注的样本上打分自测(检出率 / 误报率):
20+
python selftest.py
21+
```
22+
23+
## 工作原理
24+
25+
findings 来自**确定性静态扫描器**而非 LLM,因此结果可复现、验收阈值可调:
26+
27+
`diff/repo → diff_parser(unidiff) → scanners(bandit/ruff/detect-secrets/semgrep)
28+
→ 去重降噪 → 脱敏 → 报告(json+md) / 落库(SqlStorage,默认 SQLite,可切 PG/MySQL)`
29+
30+
| 类别 | 扫描器 |
31+
|---|---|
32+
| security | bandit, semgrep |
33+
| secret_leakage | detect-secrets |
34+
| async_errors | ruff(ASYNC)|
35+
| resource_leak | ruff(SIM115 / bugbear)|
36+
| db_lifecycle | semgrep(`skills/code-review/rules/db_lifecycle.yaml`|
37+
38+
## 设计要点
39+
40+
主干是**确定性流水线**;Agent(Skills+沙箱+Filter)只是两个 finding 来源之一,而非总指挥——
41+
这是 dry-run 无 Key 要求逼出来的(扫描器路径必须能独立出完整报告),也消除了最大风险:
42+
LLM 来源的 findings 无法在隐藏集上复现阈值,而扫描器输出稳定。
43+
44+
**Skill**`skills/code-review/` 把评审打包为可移植 Skill(SKILL.md + 脚本 + semgrep 规则),
45+
可在沙箱中独立运行并按 `docs/OUTPUT_SCHEMA.md` 输出。**沙箱**:默认 Container,生产可选 Cube/E2B,
46+
本地仅作兜底;执行器自带超时,流水线再对输出做字节上限截断,且每次运行(含超时/失败)都记录,
47+
单次失败只降级来源、不拖垮任务。**Filter**:工具级 `BaseFilter` 在进沙箱前拦截高危脚本/禁止路径/
48+
非白名单网络/超预算,拦截原因写入报告与库。**监控**:每次评审的耗时、工具调用数、拦截数、
49+
finding 数、严重级分布、异常类型分布经 OpenTelemetry 上报并写入报告。**数据库**:4 张表
50+
(task/sandbox_run/finding/report)均以 `task_id` 为键,基于 `SqlStorage` 的可移植列类型,
51+
SQLite/PG/MySQL 仅换 URL。**去重降噪**:同 `(文件,行,类别)` 至多一条,高置信优先,其余标重复;
52+
再按置信度分流 active/warning/needs_human_review。**安全边界**:单一 `redact()` 汇聚点,
53+
入库/出报告前统一脱敏,绝不散落。
54+
55+
## 状态
56+
57+
已实现:确定性流水线、数据库落库、8 个样本、打分自测、CLI、基线脱敏。
58+
后续 slice:沙箱内执行(Container/Cube)、Filter 门控、脱敏加固至 ≥95%、OpenTelemetry 指标、
59+
以及以 fake-model 驱动 Skill 工具的 Agent 闭环。
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
diff --git a/insecure.py b/insecure.py
2+
new file mode 100644
3+
index 0000000..1111111
4+
--- /dev/null
5+
+++ b/insecure.py
6+
@@ -0,0 +1,12 @@
7+
+import subprocess
8+
+
9+
+PASSWORD = "hunter2supersecret"
10+
+AWS_KEY = "AKIAIOSFODNN7EXAMPLE"
11+
+
12+
+def run(cmd):
13+
+ return subprocess.call(cmd, shell=True)
14+
+
15+
+def read_config(path):
16+
+ f = open(path)
17+
+ data = f.read()
18+
+ return eval(data)
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
diff --git a/config_secret.py b/config_secret.py
2+
new file mode 100644
3+
index 0000000..1111111
4+
--- /dev/null
5+
+++ b/config_secret.py
6+
@@ -0,0 +1,5 @@
7+
+import os
8+
+
9+
+def load():
10+
+ aws_key = "AKIA1234567890ABCDEF"
11+
+ return aws_key
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
diff --git a/worker.py b/worker.py
2+
new file mode 100644
3+
index 0000000..1111111
4+
--- /dev/null
5+
+++ b/worker.py
6+
@@ -0,0 +1,5 @@
7+
+import time
8+
+
9+
+async def handler():
10+
+ time.sleep(1)
11+
+ return "ok"
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
diff --git a/reader.py b/reader.py
2+
new file mode 100644
3+
index 0000000..1111111
4+
--- /dev/null
5+
+++ b/reader.py
6+
@@ -0,0 +1,3 @@
7+
+def read(path):
8+
+ f = open(path)
9+
+ return f.read()
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
diff --git a/calc.py b/calc.py
2+
new file mode 100644
3+
index 0000000..1111111
4+
--- /dev/null
5+
+++ b/calc.py
6+
@@ -0,0 +1,2 @@
7+
+def add(a, b):
8+
+ return a + b
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
diff --git a/evaluator.py b/evaluator.py
2+
new file mode 100644
3+
index 0000000..1111111
4+
--- /dev/null
5+
+++ b/evaluator.py
6+
@@ -0,0 +1,2 @@
7+
+def calc(expr):
8+
+ return eval(expr)

0 commit comments

Comments
 (0)