You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .agents/verification.md
+20-8Lines changed: 20 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,18 @@ This file expands [AGENTS.md](../AGENTS.md) for testing, manual UAT, CLI and bro
8
8
- The `CI` workflow runs build, typecheck, lint, tests, marketplace checks, docs link checks, and eval schema validation on pushes to `main`, pull requests to `main`, and manual dispatches.
9
9
- The CI build job publishes a short-lived, commit-addressed build artifact after `bun run build`. It is a reuse aid for workers and workflows only when the manifest's commit SHA, `bun.lock` hash, runner OS/architecture, Bun version source/value, and included output paths match the consuming checkout.
10
10
- The build artifact is intentionally limited to compiled outputs such as `packages/core/dist/**`, `packages/sdk/dist/**`, `apps/cli/dist/**`, `apps/dashboard/dist/**`, plus its manifest. It must not contain `node_modules`, Bun caches, `.turbo`, `.cache`, `.tsbuildinfo`, tracker state, evidence, or generated runtime artifacts.
11
-
- Run the same core checks locally when you need fast feedback:
11
+
- Use local checks for fast, targeted feedback while developing. Prefer the
12
+
smallest command that exercises the changed code, such as a package-specific
13
+
test, a single test file, `bun run typecheck` for type-facing changes, or
14
+
`bun run validate:examples` for eval example changes.
15
+
16
+
- Do not default to full local workspace validation before every PR. Push/open
17
+
the PR so GitHub Actions runs the broad build, typecheck, lint, test,
18
+
marketplace, link, and eval validation suite. The full local `bun run test` or
19
+
`bun run verify` pass is reserved for debugging, explicit operator request,
20
+
lack of usable CI, or changes where focused validation cannot cover the risk.
21
+
22
+
- When you do need the broad local checks, use:
12
23
13
24
```bash
14
25
bun run verify
@@ -182,15 +193,16 @@ The copied file is local-only and must remain uncommitted. If there is no
182
193
primary/main `.env`, record that as a live-provider blocker before claiming
183
194
grader dogfood is unavailable.
184
195
185
-
2. Run unit tests with `bun run test`.
186
-
3. Blocking manual red and green UAT:
196
+
2. Run focused local validation for the changed surface and record what you ran. Use `bun run test` only when debugging, explicitly requested, CI is unavailable, or focused validation cannot cover the risk.
197
+
3. Push/open the PR so GitHub Actions runs the authoritative full-suite merge gate.
198
+
4. Blocking manual red and green UAT:
187
199
188
200
- Red: run the scenario on `main` or the pre-change state and confirm the bug or missing feature is observable.
189
201
- Green: run the identical scenario on your branch and confirm the fix or feature works from the end user's perspective.
190
202
- Document both red and green evidence in the PR description or comments.
191
203
192
-
4. Verify no regressions in adjacent areas.
193
-
5. For scoring, threshold, or grader changes, run at least one real eval with a live provider and verify the output JSONL.
194
-
6. For Dashboard config, scoring-display, or dashboard API changes, use `agent-browser` to verify the UI still renders and behaves correctly.
195
-
7. If visual evidence was captured, push it to an `agentv-private` evidence branch and include the resulting branch, commit, or PR link in the handoff.
196
-
8. Mark the PR ready only after the checklist is complete and the red or green evidence is attached.
204
+
5. Verify no regressions in adjacent areas.
205
+
6. For scoring, threshold, or grader changes, run at least one real eval with a live provider and verify the output JSONL.
206
+
7. For Dashboard config, scoring-display, or dashboard API changes, use `agent-browser` to verify the UI still renders and behaves correctly.
207
+
8. If visual evidence was captured, push it to an `agentv-private` evidence branch and include the resulting branch, commit, or PR link in the handoff.
208
+
9. Mark the PR ready only after the checklist is complete, the red or green evidence is attached, and required GitHub Actions are passing.
Copy file name to clipboardExpand all lines: .agents/workflow.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -95,6 +95,7 @@ Issue and tracker workflow:
95
95
- Push focused commits to the assigned branch and open or update the PR requested by the tracker item or user.
96
96
- A branch, pushed commit, or draft PR is not done for ordinary scoped work.
97
97
- Mark tracker items complete only after the scoped work is complete, verified, merged to `main` through a PR, and documented with verification evidence.
98
+
- Run focused local validation that matches the changed code and helps debug or build confidence, then push/open the PR so GitHub Actions runs the broad suite. Do not spend local capacity on a full `bun run test` pass before every PR unless the operator asks for it, CI is unavailable, you are debugging a suite-level failure, or focused validation cannot cover the change.
98
99
- If the work intentionally remains on an ongoing branch, open a draft PR and record the branch name, PR URL, worktree path, current head commit, and remaining scope in the parent tracker item. Keep the child item open or in progress until the PR is merged or explicitly superseded.
99
100
- If a commit is a self-contained unit of completed, verified work, push it directly to its assigned remote branch instead of leaving it local for handoff. This does not override the rule against pushing directly to `main`.
100
101
- Do not merge feature, worker, or integration branches into local `main` to stage completion. If multiple branches need integration, create an integration branch, push it, and review it through a PR.
Copy file name to clipboardExpand all lines: AGENTS.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -49,6 +49,7 @@ These baseline rules apply to every repo change. They summarize the most common
49
49
- Non-trivial work needs a plan or task list. If the implementation surface starts to balloon, stop and re-plan.
50
50
- Large or high-risk PRs need meaningful, reviewable commits for each coherent change. Rewrite only the PR branch with `git push --force-with-lease` when needed to replace WIP or accidental squashed history before review.
51
51
- Manual red/green UAT is blocking before a branch is ready for review. GitHub Actions is the authoritative merge gate.
52
+
- Use focused local validation for changed code, debugging, and confidence. Do not default to the full local `bun run test` suite before every PR; push/open the PR to trigger GitHub Actions for broad full-suite validation. Full local `bun run test` is reserved for debugging, explicit operator request, lack of usable CI, or changes that cannot be covered by focused validation.
52
53
- For eval execution, experiments, repeat runs, providers, graders, or artifact-layout changes, dogfood with a live provider and a real LLM grader before marking ready. `agentv validate`, mock targets, replay/frozen transcript runs, and deterministic-only smoke tests are useful checks, but they are not live dogfood. Use canonical `.agentv/results/<run_id>/` output and publish private evidence. See [.agents/verification.md](.agents/verification.md).
53
54
- For browser or screenshot UAT, keep evidence out of the public repo and publish reviewable artifacts to an `agentv-private` evidence branch. See [.agents/verification.md](.agents/verification.md).
54
55
- When dogfood or review reveals a durable workflow lesson, capture it in this guide or the relevant `.agents/*.md` guide before merge; do not leave durable agent instructions only in PR comments, Bead comments, or private evidence. Use `docs/solutions/` for fuller reusable writeups.
0 commit comments