Skip to content

Commit 64d0435

Browse files
committed
Create Agent QC v0.1.0 standard
1 parent e9cb3da commit 64d0435

59 files changed

Lines changed: 7496 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/pages.yml

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
name: Deploy documentation to GitHub Pages
2+
3+
on:
4+
push:
5+
branches: [main]
6+
workflow_dispatch:
7+
8+
permissions:
9+
contents: read
10+
pages: write
11+
id-token: write
12+
13+
concurrency:
14+
group: pages
15+
cancel-in-progress: false
16+
17+
jobs:
18+
build:
19+
runs-on: ubuntu-latest
20+
steps:
21+
- name: Checkout
22+
uses: actions/checkout@v4
23+
24+
- name: Setup Node
25+
uses: actions/setup-node@v4
26+
with:
27+
node-version: 22
28+
cache: npm
29+
30+
- name: Install dependencies
31+
run: npm ci
32+
33+
- name: Validate examples
34+
run: npm run check:schemas
35+
36+
- name: Build site
37+
run: npm run build
38+
env:
39+
VITEPRESS_BASE: "/${{ github.event.repository.name }}/"
40+
41+
- name: Upload artifact
42+
uses: actions/upload-pages-artifact@v3
43+
with:
44+
path: docs/.vitepress/dist
45+
46+
deploy:
47+
environment:
48+
name: github-pages
49+
url: ${{ steps.deployment.outputs.page_url }}
50+
runs-on: ubuntu-latest
51+
needs: build
52+
steps:
53+
- name: Deploy to GitHub Pages
54+
id: deployment
55+
uses: actions/deploy-pages@v4

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
node_modules/
2+
docs/.vitepress/dist/
3+
.DS_Store
4+
*.log
5+
!package-lock.json

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Changelog
2+
3+
## v0.1.0 - 2026-05-10
4+
5+
- Initial Lime-focused Agent QC draft standard.
6+
- Added QC object model for plans, cases, gates, runs, verdicts, evidence, and reports.
7+
- Added Lime gate matrix for local checks, contract tests, GUI smoke, Playwright, qcloop, and review.
8+
- Added qcloop integration rules and public JSON examples.
9+
- Added public schemas for `qc_plan`, `qc_case`, and `qc_verdict`.
10+
- Added bilingual documentation and LLM entrypoints.
11+
- Added GitHub Pages deployment workflow for the documentation site.

README.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Agent QC
2+
3+
Agent QC is a Lime-focused draft standard for evidence-driven testing plans, quality gates, qcloop batches, verdicts, and review reports.
4+
5+
Lime is a GUI desktop product. For Lime, "tests passed" is not enough unless the result proves the relevant product path: local checks, command contracts, bridge readiness, GUI smoke, Playwright interaction, qcloop batch results, and human or LLM judge evidence all need a shared language.
6+
7+
Agent QC defines that language.
8+
9+
## Core boundary
10+
11+
| System | Owns | Agent QC owns |
12+
| --- | --- | --- |
13+
| Lime source repo | Product code, scripts, runtime commands, GUI, docs, and mocks. | Which Lime gates are required for a change type. |
14+
| qcloop | Batch execution, worker/verifier/repair loop, SQLite state, attempts, and qc rounds. | How tests are planned, itemized, judged, and reported. |
15+
| Agent Evidence | Durable traces, provenance, review, replay, and export facts. | Evidence references required by QC verdicts. |
16+
| Agent UI | How testing tasks and evidence are projected to users. | Behavior-level acceptance scenarios and honest pass/fail semantics. |
17+
| Human reviewers | Final product judgment when needed. | How manual review decisions become explicit verdict evidence. |
18+
19+
## What v0.1.0 defines
20+
21+
- `qc_plan`: a Lime test plan with change type, risk, required gates, cases, and evidence policy.
22+
- `qc_case`: one testable item that can map to a qcloop item.
23+
- `qc_gate`: a required validation boundary such as `verify:local`, `test:contracts`, or `verify:gui-smoke`.
24+
- `qc_run`: one execution record with command, executor, environment, and output references.
25+
- `qc_verdict`: pass/fail judgment backed by evidence refs, not agent prose.
26+
- `qc_evidence`: references to command logs, GUI smoke output, Playwright traces, qcloop attempts, screenshots, reports, or review notes.
27+
- `qc_report`: final aggregate result with remaining risk and next-action guidance.
28+
29+
## Lime testing stance
30+
31+
Agent QC uses evidence-driven gates:
32+
33+
1. Pick the Lime change type first.
34+
2. Select the minimum required gates from the Lime gate matrix.
35+
3. Write behavior-level `qc_case` items.
36+
4. Execute small deterministic gates directly and batch repetitive cases through qcloop.
37+
5. Let an independent verifier judge evidence using strict JSON.
38+
6. Report pass/fail/exhausted with evidence refs and remaining risk.
39+
40+
A qcloop batch finishing is not automatically a product pass. A Lime change is deliverable only when the required gates for its risk class have evidence-backed verdicts.
41+
42+
## Documentation
43+
44+
- [Specification](docs/en/specification.md)
45+
- [Lime gate matrix](docs/en/authoring/lime-gate-matrix.md)
46+
- [qcloop integration](docs/en/authoring/qcloop-integration.md)
47+
- [Evidence-driven verdicts](docs/en/authoring/evidence-driven-verdicts.md)
48+
- [Acceptance scenarios](docs/en/authoring/acceptance-scenarios.md)
49+
- [中文规范](docs/zh/specification.md)
50+
51+
## LLM entrypoints
52+
53+
- [`llms.txt`](llms.txt): concise navigation index for AI clients.
54+
- [`llms-full.txt`](llms-full.txt): concatenated current English documentation for model context.
55+
- [`llm.txt`](llm.txt) and [`llm-full.txt`](llm-full.txt): compatibility aliases.
56+
57+
## Local development
58+
59+
```bash
60+
npm install
61+
npm run dev
62+
```
63+
64+
## Build and validation
65+
66+
```bash
67+
npm run check:schemas
68+
npm run build
69+
```
70+
71+
The static site is generated at `docs/.vitepress/dist`.
72+
73+
## Related local sources
74+
75+
- Lime engineering rules: `/Users/coso/Documents/dev/ai/aiclientproxy/lime/docs/aiprompts/quality-workflow.md`
76+
- Lime GUI E2E rules: `/Users/coso/Documents/dev/ai/aiclientproxy/lime/docs/aiprompts/playwright-e2e.md`
77+
- qcloop reference: `/Users/coso/Documents/dev/ai/limecloud/qcloop/llms-full.txt`

RELEASE_NOTES.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Release Notes
2+
3+
## v0.1.0
4+
5+
Agent QC v0.1.0 is the first Lime-focused draft standard for evidence-driven testing plans, quality gates, qcloop batches, verdicts, and release-ready reports.
6+
7+
### Highlights
8+
9+
- Defines the core Agent QC objects: `qc_plan`, `qc_case`, `qc_gate`, `qc_run`, `qc_verdict`, `qc_evidence`, and `qc_report`.
10+
- Establishes Lime's evidence-driven testing stance: a pass verdict requires inspectable evidence, not agent self-reporting.
11+
- Adds the Lime gate matrix for `verify:local`, `test:contracts`, `verify:gui-smoke`, Playwright product flows, qcloop batches, and manual or LLM judge review.
12+
- Documents qcloop integration, including JSON item shape, worker prompt requirements, verifier prompt requirements, and status mapping for `success`, `failed`, and `exhausted`.
13+
- Adds bilingual English and Simplified Chinese documentation with VitePress navigation, version pages, and LLM-friendly entrypoints.
14+
- Adds public JSON schemas and examples for QC plans, QC cases, QC verdicts, and qcloop job requests.
15+
- Adds GitHub Pages deployment through GitHub Actions.
16+
17+
### Public docs
18+
19+
After the Pages workflow completes, the documentation site is expected at:
20+
21+
- https://limecloud.github.io/agentqc/
22+
- https://limecloud.github.io/agentqc/llms.txt
23+
- https://limecloud.github.io/agentqc/llms-full.txt
24+
25+
### Validation
26+
27+
Local validation performed before release:
28+
29+
- `npm run check:schemas`
30+
- `npm run build`
31+
- `npm audit --omit=dev`
32+
- `npm audit --audit-level=high`
33+
34+
`npm audit --audit-level=high` passes with no high or critical vulnerabilities. npm still reports a dev-only moderate advisory in the VitePress/Vite/esbuild chain with no current fix available.
35+
36+
### Scope
37+
38+
This release is intentionally scoped to Lime. It does not claim to be a generic industry testing standard yet. qcloop remains the execution loop; Agent QC defines the testing standard, evidence requirements, and reporting contract.

docs/.vitepress/config.ts

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
import { defineConfig } from 'vitepress'
2+
3+
const base = process.env.VITEPRESS_BASE || '/'
4+
const versions = ['0.1.0']
5+
6+
const enVersionItems = versions.flatMap((version) => [
7+
{ text: `v${version} overview`, link: `/en/versions/v${version}/overview` },
8+
{ text: `v${version} specification`, link: `/en/versions/v${version}/specification` },
9+
{ text: `v${version} changelog`, link: `/en/versions/v${version}/changelog` }
10+
])
11+
12+
const zhVersionItems = versions.flatMap((version) => [
13+
{ text: `v${version} 概览`, link: `/zh/versions/v${version}/overview` },
14+
{ text: `v${version} 规范`, link: `/zh/versions/v${version}/specification` },
15+
{ text: `v${version} 变更记录`, link: `/zh/versions/v${version}/changelog` }
16+
])
17+
18+
const enNav = [
19+
{ text: 'Guide', link: '/en/what-is-agent-qc' },
20+
{ text: 'Specification', link: '/en/specification' },
21+
{ text: 'Examples', link: '/en/examples/lime-gui-change' },
22+
{ text: 'Ecosystem', link: '/en/reference/agent-ecosystem' },
23+
{ text: 'Version', items: [{ text: 'latest', link: '/en/specification' }, ...enVersionItems] }
24+
]
25+
26+
const zhNav = [
27+
{ text: '指南', link: '/zh/what-is-agent-qc' },
28+
{ text: '规范', link: '/zh/specification' },
29+
{ text: '示例', link: '/zh/examples/lime-gui-change' },
30+
{ text: '生态', link: '/zh/reference/agent-ecosystem' },
31+
{ text: '版本', items: [{ text: 'latest', link: '/zh/specification' }, ...zhVersionItems] }
32+
]
33+
34+
const enSidebar = [
35+
{
36+
text: 'Start here',
37+
items: [
38+
{ text: 'Overview', link: '/en/' },
39+
{ text: 'What is Agent QC?', link: '/en/what-is-agent-qc' },
40+
{ text: 'Specification', link: '/en/specification' }
41+
]
42+
},
43+
{
44+
text: 'For test authors',
45+
items: [
46+
{ text: 'Quickstart', link: '/en/authoring/quickstart' },
47+
{ text: 'Lime gate matrix', link: '/en/authoring/lime-gate-matrix' },
48+
{ text: 'qcloop integration', link: '/en/authoring/qcloop-integration' },
49+
{ text: 'Evidence-driven verdicts', link: '/en/authoring/evidence-driven-verdicts' },
50+
{ text: 'Acceptance scenarios', link: '/en/authoring/acceptance-scenarios' }
51+
]
52+
},
53+
{
54+
text: 'Reference',
55+
items: [
56+
{ text: 'Glossary', link: '/en/reference/glossary' },
57+
{ text: 'Lime quality source index', link: '/en/reference/lime-quality-source-index' },
58+
{ text: 'Agent standards ecosystem', link: '/en/reference/agent-ecosystem' }
59+
]
60+
},
61+
{
62+
text: 'Examples',
63+
items: [
64+
{ text: 'Lime GUI change', link: '/en/examples/lime-gui-change' },
65+
{ text: 'Lime command bridge change', link: '/en/examples/lime-command-bridge-change' },
66+
{ text: 'qcloop batch', link: '/en/examples/qcloop-batch' }
67+
]
68+
},
69+
{ text: 'Versions', items: enVersionItems }
70+
]
71+
72+
const zhSidebar = [
73+
{
74+
text: '开始',
75+
items: [
76+
{ text: '概览', link: '/zh/' },
77+
{ text: '什么是 Agent QC', link: '/zh/what-is-agent-qc' },
78+
{ text: '规范', link: '/zh/specification' }
79+
]
80+
},
81+
{
82+
text: '测试作者',
83+
items: [
84+
{ text: '快速开始', link: '/zh/authoring/quickstart' },
85+
{ text: 'Lime 门禁矩阵', link: '/zh/authoring/lime-gate-matrix' },
86+
{ text: 'qcloop 集成', link: '/zh/authoring/qcloop-integration' },
87+
{ text: '证据驱动判定', link: '/zh/authoring/evidence-driven-verdicts' },
88+
{ text: '验收场景', link: '/zh/authoring/acceptance-scenarios' }
89+
]
90+
},
91+
{
92+
text: '参考',
93+
items: [
94+
{ text: '术语表', link: '/zh/reference/glossary' },
95+
{ text: 'Lime 质量事实源索引', link: '/zh/reference/lime-quality-source-index' },
96+
{ text: 'Agent 标准生态', link: '/zh/reference/agent-ecosystem' }
97+
]
98+
},
99+
{
100+
text: '示例',
101+
items: [
102+
{ text: 'Lime GUI 改动', link: '/zh/examples/lime-gui-change' },
103+
{ text: 'Lime 命令桥接改动', link: '/zh/examples/lime-command-bridge-change' },
104+
{ text: 'qcloop 批次', link: '/zh/examples/qcloop-batch' }
105+
]
106+
},
107+
{ text: '版本', items: zhVersionItems }
108+
]
109+
110+
export default defineConfig({
111+
base,
112+
title: 'Agent QC',
113+
description: 'A Lime-focused standard for evidence-driven agent quality control.',
114+
cleanUrls: true,
115+
lastUpdated: true,
116+
themeConfig: {
117+
logo: '/logo.svg',
118+
nav: [
119+
{ text: 'English', items: enNav },
120+
{ text: '中文', items: zhNav }
121+
],
122+
sidebar: {
123+
'/en/': enSidebar,
124+
'/zh/': zhSidebar
125+
},
126+
search: { provider: 'local' },
127+
footer: {
128+
message: 'Draft Lime-focused standard for evidence-driven agent quality control.',
129+
copyright: 'Copyright © 2026'
130+
}
131+
},
132+
markdown: { lineNumbers: true }
133+
})
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
title: Acceptance scenarios
3+
description: Behavior-level validation scenarios for Lime Agent QC.
4+
---
5+
6+
# Acceptance scenarios
7+
8+
Agent QC scenarios validate behavior and evidence, not file presence.
9+
10+
## 1. Ordinary frontend change
11+
12+
1. Classify change as `frontend`.
13+
2. Run `npm run verify:local`.
14+
3. Record command output evidence.
15+
4. Report pass only if the command passes and no higher-risk boundary was touched.
16+
17+
Passing condition: local validation evidence supports the change and no GUI or bridge gate is required.
18+
19+
## 2. User-visible UI change
20+
21+
1. Classify change as `user-visible-ui`.
22+
2. Run `npm run verify:local`.
23+
3. Add or run a stable UI assertion when an existing test surface exists.
24+
4. Record evidence for user-visible expected state.
25+
26+
Passing condition: UI behavior is asserted, not just typechecked.
27+
28+
## 3. GUI shell or Workspace change
29+
30+
1. Classify change as `gui-shell-workspace`.
31+
2. Run `npm run verify:local`.
32+
3. Run `npm run verify:gui-smoke`.
33+
4. Record DevBridge and workspace readiness evidence.
34+
35+
Passing condition: GUI main path is proven usable.
36+
37+
## 4. Tauri command / Bridge / mock change
38+
39+
1. Classify change as `tauri-command-bridge-mock`.
40+
2. Check command surfaces as a group.
41+
3. Run `npm run test:contracts` and `npm run verify:local`.
42+
4. Record failing command ids when contracts fail.
43+
44+
Passing condition: frontend calls, Rust registration, catalog, and mocks stay synchronized.
45+
46+
## 5. qcloop batch regression
47+
48+
1. Convert each behavior into one independent `qc_case`.
49+
2. Create a qcloop job with strict verifier JSON.
50+
3. Start the job and poll until no item is pending or running.
51+
4. Summarize `success`, `failed`, and `exhausted` separately.
52+
53+
Passing condition: every required item has verifier-backed evidence.
54+
55+
## 6. Manual or LLM judge review
56+
57+
1. Define a rubric before review.
58+
2. Provide the reviewer with outputs and evidence refs.
59+
3. Record verdict, reviewer identity or judge configuration, and remaining risk.
60+
61+
Passing condition: review decision is explicit and traceable.

0 commit comments

Comments
 (0)