Skip to content

Commit ff4ccd4

Browse files
diff added
1 parent 4c7908c commit ff4ccd4

30 files changed

Lines changed: 2652 additions & 21 deletions

.capabilities/core/assessment/assess-implementation-coverage.capability.yaml

Lines changed: 50 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,16 @@ acceptance:
1212
- Presents each acceptance criterion beside relevant implementation evidence.
1313
- Marks criteria as covered, uncovered, or uncertain.
1414
- Reports missing or unreadable references as uncovered evidence.
15+
- Provides an advisory report that groups coverage findings by recommended
16+
action.
17+
- Provides a developer-facing status summary that separates needs-action,
18+
needs-review, ok, and planned capabilities.
19+
- Advisory reports prefer saved agent.review criteria over deterministic text
20+
evidence.
21+
- Lists noisy capabilities that are good candidates for semantic Codex review.
22+
- Can update agent.review from current implementation evidence without
23+
changing capability status.
24+
- Allows accepted advisory findings to be ignored with an explicit reason.
1525
- Keeps the first version deterministic or review-assisted rather than
1626
silently trusting an AI judgment.
1727
guidance:
@@ -28,28 +38,64 @@ agent:
2838
- referenced source and documentation files
2939
outputs:
3040
- implementation coverage report
41+
- actionable assessment advice
42+
- synchronized review evidence
3143
- uncovered or uncertain acceptance criteria
3244
depends_on:
3345
- core/validation/verify-implementation-references
3446
implementation:
3547
references:
3648
- packages/core/src/assessImplementationCoverage.ts
49+
- packages/core/src/assessmentAdvice.ts
50+
- packages/core/src/capabilityStatus.ts
51+
- packages/core/src/syncReviewEvidence.ts
52+
- packages/core/src/schema.ts
53+
- packages/core/src/types.ts
3754
- packages/core/src/index.ts
3855
- packages/cli/src/index.ts
3956
- packages/core/tests/assessImplementationCoverage.test.ts
57+
- packages/core/tests/assessmentAdvice.test.ts
58+
- packages/core/tests/capabilityStatus.test.ts
59+
- packages/core/tests/syncReviewEvidence.test.ts
4060
verification:
4161
automated:
4262
- id: coverage-report-tests
4363
description: Unit tests cover report generation for covered, uncovered, and
4464
missing-reference cases.
4565
command: npm test -- packages/core/tests/assessImplementationCoverage.test.ts
66+
- id: advisory-report-tests
67+
description: Unit tests cover advisory classification and formatted recommended
68+
actions.
69+
command: npm test -- packages/core/tests/assessmentAdvice.test.ts
70+
- id: capability-status-tests
71+
description: Unit tests cover developer-facing capability health summaries.
72+
command: npm test -- packages/core/tests/capabilityStatus.test.ts
73+
- id: sync-review-tests
74+
description: Unit tests cover syncing agent.review without changing capability
75+
status.
76+
command: npm test -- packages/core/tests/syncReviewEvidence.test.ts
4677
- id: dogfood-coverage-report
4778
description: The CLI reports implementation coverage for this capability.
48-
command: node packages/cli/dist/index.js assess core/assessment/assess-implementation-coverage
79+
command: node packages/cli/dist/index.js assess
80+
core/assessment/assess-implementation-coverage
81+
- id: dogfood-advisory-report
82+
description: The CLI reports actionable advice for this capability.
83+
command: node packages/cli/dist/index.js advise
84+
core/assessment/assess-implementation-coverage
85+
- id: dogfood-sync-review-dry-run
86+
description: The CLI can preview review evidence synchronization for this
87+
capability.
88+
command: node packages/cli/dist/index.js sync-review
89+
core/assessment/assess-implementation-coverage --dry-run
4990
manual:
50-
- Run the coverage report against core/validation/verify-implementation-references
51-
and confirm the output gives a reviewer enough context to judge the
52-
implementation.
91+
- Run the coverage report against
92+
core/validation/verify-implementation-references and confirm the output
93+
gives a reviewer enough context to judge the implementation.
5394
gaps:
5495
- Deterministic text evidence can identify review targets but cannot prove
5596
semantic correctness without human or explicit AI review.
97+
ignore_gaps:
98+
- code: declared-gap
99+
message_contains: Deterministic text evidence can identify review targets
100+
reason: This is an accepted limitation of the deterministic assessor; semantic
101+
review is handled through agent-review and review-result.

.capabilities/core/graph/compile-capabilities.capability.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ agent:
2727
references:
2828
- packages/core/src/compileCapabilities.ts
2929
- packages/cli/src/index.ts
30+
- packages/core/tests/compile.test.ts
3031
verification:
3132
automated:
3233
- id: compile-tests
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
id: core/graph/diff-capabilities
2+
title: Diff capabilities
3+
status: implemented
4+
area: core
5+
summary: Compare current capability files with a Git base and summarize added,
6+
changed, and removed capabilities.
7+
intent: Help developers understand product and agent-facing intent changes
8+
without reading raw YAML diffs.
9+
acceptance:
10+
- Compares current capabilities against a configurable Git base ref.
11+
- Reports added, changed, and removed capabilities by capability ID.
12+
- Summarizes meaningful field changes such as status, intent, acceptance,
13+
dependencies, implementation references, verification, and ignore policy.
14+
- Excludes saved agent.review evidence churn by default while allowing it to
15+
be included explicitly.
16+
- Includes downstream impact context for changed capabilities.
17+
- Defaults to a concise PR-review-style summary and offers verbose
18+
field-level output.
19+
- Exposes the report through the CLI with human-readable and JSON output.
20+
guidance:
21+
- Compare normalized parsed capabilities, not raw YAML text.
22+
- Keep review evidence diffs out of the default view because they can be
23+
large and stale.
24+
- Include ignore policy changes because they affect assessment behavior.
25+
- Avoid raw JSON in the default human output.
26+
agent:
27+
inputs:
28+
- current capability files
29+
- Git base ref
30+
outputs:
31+
- capability diff report
32+
- added, changed, and removed capability IDs
33+
- field-level change summaries
34+
depends_on:
35+
- core/graph/compile-capabilities
36+
- core/graph/analyze-capability-impact
37+
implementation:
38+
references:
39+
- packages/core/src/capabilityDiff.ts
40+
- packages/core/src/index.ts
41+
- packages/cli/src/index.ts
42+
- packages/core/tests/capabilityDiff.test.ts
43+
verification:
44+
automated:
45+
- id: capability-diff-tests
46+
description: Unit tests cover Git-backed capability diff reporting.
47+
command: npm test -- packages/core/tests/capabilityDiff.test.ts
48+
- id: dogfood-capability-diff
49+
description: The CLI compares current CapabilityKit capabilities against HEAD.
50+
command: node packages/cli/dist/index.js diff --base HEAD
51+
manual:
52+
- Run capabilitykit diff during a working-tree change and confirm it
53+
explains capability intent changes more clearly than git diff.

.capabilities/core/validation/detect-verification-gaps.capability.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ acceptance:
1111
- Reports implemented or verified capabilities without
1212
agent.implementation.references values.
1313
- Reports declared agent.verification.gaps from capability files.
14+
- Allows capability files to suppress selected verification gap warnings with
15+
an explicit reason.
1416
guidance:
1517
- Prefer specific gap messages tied to capability IDs.
1618
- Keep gap detection rules deterministic and covered by automated tests.
@@ -25,6 +27,9 @@ agent:
2527
implementation:
2628
references:
2729
- packages/core/src/validateCapabilities.ts
30+
- packages/core/src/schema.ts
31+
- packages/core/src/types.ts
32+
- packages/core/tests/validation.test.ts
2833
verification:
2934
automated:
3035
- id: verification-gap-tests

.capabilities/core/validation/validate-capability-files.capability.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,13 @@ agent:
2626
- core/model/define-capability-format
2727
implementation:
2828
references:
29+
- packages/core/src/parseCapability.ts
30+
- packages/core/src/schema.ts
2931
- packages/core/src/validateCapabilities.ts
3032
- packages/core/src/loadCapabilities.ts
3133
- packages/cli/src/index.ts
34+
- packages/core/tests/parser.test.ts
35+
- packages/core/tests/validation.test.ts
3236
verification:
3337
automated:
3438
- id: validation-tests

.capabilities/developer-experience/cli/cli-workflow.capability.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ summary: Provide CLI commands for init, create, validate, compile, and inspect.
66
intent: Make capability management fast enough to stay close to normal
77
development workflows.
88
acceptance:
9-
- Supports init, create, validate, compile, and inspect commands.
9+
- Supports init, create, status, diff, validate, compile, and inspect
10+
commands.
1011
- Returns a non-zero exit code for validation failures.
1112
- Prints verification gaps in validation and inspect output.
1213
guidance:

.capabilities/developer-experience/examples/example-project.capability.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ agent:
2323
references:
2424
- examples/basic-app/.capabilities/capabilitykit.yaml
2525
- examples/basic-app/.capabilities/account/login.capability.yaml
26+
- examples/basic-app/.capabilities/account/profile.capability.yaml
27+
- examples/basic-app/.capabilities/checkout/cart.capability.yaml
28+
- examples/basic-app/.capabilities/checkout/payment.capability.yaml
29+
- packages/core/tests/parser.test.ts
2630
verification:
2731
automated:
2832
- id: parser-tests

.capabilities/docs/reference/capability-format-documentation.capability.yaml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,22 @@ agent:
3232
manual:
3333
- Read README from a new user's perspective and confirm the format is
3434
clear.
35+
review:
36+
ignore_findings:
37+
- status: weak-evidence
38+
criterion: README explains what a capability is.
39+
reason: README documentation evidence has been manually reviewed for this
40+
criterion.
41+
- status: weak-evidence
42+
criterion: README includes an example capability file.
43+
reason: README documentation evidence has been manually reviewed for this
44+
criterion.
45+
- status: weak-evidence
46+
criterion: README explains verification gaps.
47+
reason: README documentation evidence has been manually reviewed for this
48+
criterion.
49+
- status: weak-evidence
50+
criterion: README explains how folder hierarchy and capability IDs should
51+
align.
52+
reason: README documentation evidence has been manually reviewed for this
53+
criterion.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
node_modules/
22
dist/
33
coverage/
4+
tmp/
45
.capabilities/dist/
56
*.tsbuildinfo

README.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,17 +89,64 @@ agent:
8989
- `capabilitykit init` creates a starter `.capabilities/` folder.
9090
- `capabilitykit create <name> --area <area>` creates a capability file.
9191
- `capabilitykit skill` creates or updates CapabilityKit skill files and agent entrypoints.
92+
- `capabilitykit status [capability-id]` shows a developer-friendly capability health summary.
9293
- `capabilitykit validate` validates capability files and reports verification gaps.
9394
- `capabilitykit compile` writes normalized JSON to `.capabilities/dist/capabilities.json`.
9495
- `capabilitykit inspect <capability-id>` prints one capability and its relationships.
9596
- `capabilitykit impact <capability-id>` reports direct and transitive downstream capabilities plus suggested verification.
97+
- `capabilitykit diff [capability-id]` compares capability changes against a Git base ref.
98+
- `capabilitykit assess <capability-id>` compares acceptance criteria with referenced implementation evidence.
99+
- `capabilitykit advise [capability-id]` groups assessment findings into recommended next actions.
100+
- `capabilitykit review-noisy --limit 5` lists high-value capabilities for semantic Codex or human review.
101+
- `capabilitykit sync-review [capability-id]` updates `agent.review` from current implementation evidence without changing capability status.
102+
103+
`status` is the best first command when you want to understand what the
104+
capability map says about the project:
105+
106+
```bash
107+
capabilitykit status
108+
capabilitykit status core/graph/compile-capabilities
109+
capabilitykit diff --base HEAD
110+
capabilitykit diff --base HEAD --verbose
111+
```
96112

97113
## Verification Gaps
98114

99115
CapabilityKit treats missing confidence as a first-class signal. Missing automated checks, vague acceptance criteria, broken references, missing `agent.implementation.references`, and manual review gaps are reported as verification gaps.
100116

101117
Gaps are warnings by default. They should be fixed or intentionally documented so humans and agents know what still needs review.
102118

119+
When a warning is intentionally accepted, suppress it in the capability with an explicit reason:
120+
121+
```yaml
122+
agent:
123+
verification:
124+
ignore_gaps:
125+
- code: missing-automated-checks
126+
reason: Manual review is the accepted verification path for this documentation-only capability.
127+
- code: declared-gap
128+
message_contains: Known external dependency.
129+
reason: Tracked outside CapabilityKit for this release.
130+
```
131+
132+
Use `code: "*"` only when every verification gap for that capability is intentionally handled elsewhere.
133+
134+
Advisory assessment findings can also be ignored when a maintainer accepts the
135+
deterministic assessor's limitation for a specific criterion. Ignored findings
136+
are removed from recommended actions and `review-noisy` scoring, but remain
137+
auditable in the capability file:
138+
139+
```yaml
140+
agent:
141+
review:
142+
ignore_findings:
143+
- status: weak-evidence
144+
criterion: README explains what a capability is.
145+
reason: Documentation wording was manually reviewed and accepted.
146+
```
147+
148+
Use `criterion_contains` for a small family of related findings, and `status: "*"` only for intentionally accepted findings across statuses.
149+
103150
## Dogfooding
104151
105152
CapabilityKit uses its own `.capabilities/` folder from the first usable version. Each MVP feature has a matching capability spec, and the project verification loop validates and compiles those specs.

0 commit comments

Comments
 (0)