Skip to content

Commit 6f3e739

Browse files
leo-aa88leo-aa88
authored
Merge pull request #92 from LAA-Software-Engineering/feat/pr-review-demo-example
feat: PR review demo with offline GitHub tools, policy trace, and integration tests
2 parents 55c0a76 + 89c1bb4 commit 6f3e739

15 files changed

Lines changed: 376 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ Most agent stacks today bury prompts, tool wiring, and permissions in applicatio
2525

2626
The full product vision, YAML spec v0, and architecture are documented in [**`docs/DESIGN_DOC.md`**](docs/DESIGN_DOC.md).
2727

28+
**Featured walkthrough:** declarative PR review with a **policy-blocked** (simulated) GitHub comment — no API keys required — in [**`examples/pr-review-demo/README.md`**](examples/pr-review-demo/README.md).
29+
2830
---
2931

3032
## Mental model
@@ -228,6 +230,7 @@ The **recommended implementation phases** are outlined in **section 20** of [`do
228230
## Documentation
229231

230232
- **[`docs/DESIGN_DOC.md`](docs/DESIGN_DOC.md)** — design document v0 (problem statement, spec, CLI, engine, state model, testing strategy, MVP vs end state, section 23 recommendation).
233+
- **[`examples/pr-review-demo/README.md`](examples/pr-review-demo/README.md)** — end-to-end demo: structured review output, traceable run, **approval-gated** write (`validate` → `plan` → `apply` → `run` → `logs`).
231234
- **[`docs/EXAMPLES.md`](docs/EXAMPLES.md)** — copy-paste YAML and CLI examples (`init`, mock vs OpenAI, workflows, environment overlays).
232235
- **[`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md)** — Contributor Covenant 2.1; participation expectations and reporting.
233236
- **License:** [MIT](LICENSE)

‎examples/pr-review-demo/README.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# PR review demo (policy-gated GitHub comment)
2+
3+
This example shows **why a declarative control plane beats ad-hoc glue code** for agent workflows that touch the outside world.
4+
5+
## What it demonstrates
6+
7+
- **Workflow as data** — `workflows/pr-review.yaml` reads like a runbook: fetch PR context → review → post a comment.
8+
- **Structured agent output** — the reviewer must return JSON validated by `schemas/review-output.json` (summary + findings), not an unparseable blob.
9+
- **First-class policy** — `policies/guarded-writes.yaml` lists which tool `uses` strings require explicit approval.
10+
- **Safe-by-default writes** — the `post_comment` step calls a **simulated** native GitHub tool (`tools/github.yaml`). Without approval, **policy blocks the call** before any side effect.
11+
- **Traceable behavior** — `agentctl logs` shows normal step progress plus a **`policy.denied`** event on the blocked step.
12+
13+
## Why this matters
14+
15+
In a typical script, “call the model, then maybe post to GitHub” is buried in code paths that are hard to review, diff, and audit. Here, **permissions and sequencing are YAML** you can put in code review, and the runtime **enforces** them with **exit codes** and **SQLite traces**.
16+
17+
## Project layout
18+
19+
| Path | Role |
20+
|------|------|
21+
| `project.yaml` | Imports resources; defaults to `mock` model (no API keys). |
22+
| `workflows/pr-review.yaml` | Three steps: `fetch_pr`, `review_diff`, `post_comment`. |
23+
| `agents/reviewer.yaml` | Reviewer instructions + output schema. |
24+
| `tools/github.yaml` | `native` tool; operations are implemented offline in the binary. |
25+
| `policies/guarded-writes.yaml` | Requires `--approve` for `tool.github.pull_request.post_comment`. |
26+
| `schemas/*.json` | JSON Schema for workflow input and agent output. |
27+
| `fixtures/sample-pr.json` | Sample input (no GitHub network or tokens). |
28+
29+
## Prerequisites
30+
31+
Build `agentctl` from the repo root (`make build`) or use a release binary on your `PATH`.
32+
33+
## How to run
34+
35+
From the **repository root** (paths below assume that):
36+
37+
```bash
38+
agentctl validate --project examples/pr-review-demo
39+
agentctl plan --project examples/pr-review-demo --state /tmp/pr-review-state.db
40+
agentctl apply --project examples/pr-review-demo --state /tmp/pr-review-state.db --auto-approve
41+
```
42+
43+
### Default run (comment blocked)
44+
45+
```bash
46+
agentctl run workflow/pr-review \
47+
--project examples/pr-review-demo \
48+
--state /tmp/pr-review-state.db \
49+
--input-file examples/pr-review-demo/fixtures/sample-pr.json
50+
```
51+
52+
- Exit code **5** = policy denial (by design).
53+
- Inspect the trace (use the printed **Run ID**):
54+
55+
```bash
56+
agentctl logs --project examples/pr-review-demo --state /tmp/pr-review-state.db --run <run-id>
57+
```
58+
59+
You should see steps through `review_diff`, then **`policy.denied`** on `post_comment` with reason **`approval_required`**.
60+
61+
### Optional: allow the write (full success)
62+
63+
```bash
64+
agentctl run workflow/pr-review \
65+
--project examples/pr-review-demo \
66+
--state /tmp/pr-review-state.db \
67+
--input-file examples/pr-review-demo/fixtures/sample-pr.json \
68+
--approve tool.github.pull_request.post_comment
69+
```
70+
71+
This records a simulated comment result (still **no** real GitHub traffic).
72+
73+
## Expected highlights
74+
75+
1. **`fetch_pr`** completes — native tool normalizes the `pr` object from input JSON.
76+
2. **`review_diff`** completes — mock model returns fixed structured JSON that satisfies the schema.
77+
3. **`post_comment`** is **blocked** unless approved — CLI prints a clear **policy** line naming the gated `uses` string.
78+
79+
## Design note (no real GitHub)
80+
81+
`pull_request.fetch` and `pull_request.post_comment` are **offline** native operations: they exist so the workflow and policy strings look like a real integration while the demo stays repeatable in CI and on laptops without tokens.
82+
83+
## Compared to one-off code
84+
85+
| Concern | This demo | Typical script |
86+
|--------|-----------|----------------|
87+
| Order of operations | Workflow YAML | Implicit control flow |
88+
| “Can we post?” | Policy resource + trace | Easy to forget a guard |
89+
| Model output shape | JSON Schema on the agent | String parsing / hope |
90+
| Audit trail | `agentctl logs` | Printf / none |
91+
92+
For broader patterns, see [`docs/EXAMPLES.md`](../../docs/EXAMPLES.md) and [`docs/DESIGN_DOC.md`](../../docs/DESIGN_DOC.md).

‎examples/pr-review-demo/agents/reviewer.yaml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
apiVersion: agentic.dev/v0
2+
kind: Agent
3+
metadata:
4+
name: reviewer
5+
spec:
6+
model: mock/gpt-4
7+
policy: guarded-writes
8+
constraints:
9+
timeoutSeconds: 120
10+
instructions: |
11+
You review pull requests for risky changes. The user message is JSON with a "pull_request"
12+
object (title, body, files_changed, etc.).
13+
14+
Respond with one JSON object only (no markdown, no code fences). Use exactly this shape:
15+
{
16+
"summary": "<one line overview>",
17+
"findings": [
18+
{
19+
"severity": "high" | "medium" | "low" | "info",
20+
"file": "<path or label>",
21+
"title": "<short issue title>",
22+
"evidence": "<what in the PR suggests this>"
23+
}
24+
]
25+
}
26+
output:
27+
schema: ./schemas/review-output.json

‎examples/pr-review-demo/fixtures/sample-pr.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"pr": {
3+
"title": "Add user CSV export",
4+
"number": 42,
5+
"head_ref": "feature/export",
6+
"base_ref": "main",
7+
"body": "Adds an admin endpoint that exports users to CSV.",
8+
"files_changed": ["db/query.py", "api/handlers.go"]
9+
}
10+
}

‎examples/pr-review-demo/policies/guarded-writes.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: agentic.dev/v0
2+
kind: Policy
3+
metadata:
4+
name: guarded-writes
5+
spec:
6+
execution:
7+
maxWallClockSeconds: 300
8+
maxTotalCostUsd: 5
9+
approvals:
10+
requiredFor:
11+
- tool.github.pull_request.post_comment

‎examples/pr-review-demo/project.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
apiVersion: agentic.dev/v0
2+
kind: Project
3+
metadata:
4+
name: pr-review-demo
5+
spec:
6+
imports:
7+
- ./policies/guarded-writes.yaml
8+
- ./tools/github.yaml
9+
- ./agents/reviewer.yaml
10+
- ./workflows/pr-review.yaml
11+
defaults:
12+
policy: guarded-writes
13+
model: mock/gpt-4
14+
runtime: local
15+
providers:
16+
models:
17+
mock:
18+
type: mock

‎examples/pr-review-demo/schemas/pr-review-input.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"type": "object",
4+
"required": ["pr"],
5+
"properties": {
6+
"pr": {
7+
"type": "object",
8+
"additionalProperties": true
9+
}
10+
},
11+
"additionalProperties": true
12+
}

‎examples/pr-review-demo/schemas/review-output.json‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"type": "object",
4+
"required": ["summary", "findings"],
5+
"properties": {
6+
"summary": { "type": "string" },
7+
"findings": {
8+
"type": "array",
9+
"items": {
10+
"type": "object",
11+
"required": ["severity", "file", "title", "evidence"],
12+
"properties": {
13+
"severity": {
14+
"type": "string",
15+
"enum": ["high", "medium", "low", "info"]
16+
},
17+
"file": { "type": "string" },
18+
"title": { "type": "string" },
19+
"evidence": { "type": "string" }
20+
},
21+
"additionalProperties": false
22+
}
23+
}
24+
},
25+
"additionalProperties": false
26+
}

‎examples/pr-review-demo/tools/github.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
apiVersion: agentic.dev/v0
2+
kind: Tool
3+
metadata:
4+
name: github
5+
spec:
6+
type: native

‎examples/pr-review-demo/workflows/pr-review.yaml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
apiVersion: agentic.dev/v0
2+
kind: Workflow
3+
metadata:
4+
name: pr-review
5+
spec:
6+
description: |
7+
Offline demo: load PR-shaped JSON, run a structured review, then attempt to post a comment.
8+
The comment step is gated by policy unless you pass --approve for the exact uses string.
9+
policy: guarded-writes
10+
input:
11+
schema: ./schemas/pr-review-input.json
12+
steps:
13+
- id: fetch_pr
14+
uses: tool.github.pull_request.fetch
15+
with:
16+
pr: ${input.pr}
17+
- id: review_diff
18+
agent: reviewer
19+
with:
20+
pull_request: ${steps.fetch_pr.output.pull_request}
21+
- id: post_comment
22+
uses: tool.github.pull_request.post_comment
23+
with:
24+
body: |
25+
## Automated review
26+
27+
${steps.review_diff.output.summary}
28+
29+
${steps.review_diff.output.findings}
30+
output:
31+
value:
32+
pull_request: ${steps.fetch_pr.output.pull_request}
33+
review: ${steps.review_diff.output}
34+
comment: ${steps.post_comment.output}

0 commit comments

Comments
 (0)