Skip to content

Commit e39e7c7

Browse files
sdieguez-wotcclaude
andcommitted
fix: move benchmark schema to public dir and require result_kind
Applies the remaining code-review changes that the test suite already expected but the source did not satisfy: - Move results.schema.json into docs/public/benchmarks/ so VitePress serves it at /neuron-js/benchmarks/results.schema.json (matching its $id); previously it lived in a content dir and was never served and the contract test could not find it. - Add result_kind to the per-row required array in the schema. - Update path references in methodology.md and benchmarks/README.md, and fix the index.md link to the served public URL. Fixes the failing CI quality job (3 benchmark-contract tests). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 415d4ec commit e39e7c7

4 files changed

Lines changed: 8 additions & 7 deletions

File tree

benchmarks/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@ This directory contains benchmark data artifacts for NJS-GROWTH-07.
44

55
- `sample-results.placeholder.json` is deterministic placeholder data for chart and playground wiring.
66
- It is not benchmark output and must not be used for public performance claims.
7-
- Real benchmark output must match `docs/benchmarks/results.schema.json` and set `result_kind` to `actual_benchmark`.
7+
- Real benchmark output must match `docs/public/benchmarks/results.schema.json` and set `result_kind` to `actual_benchmark`.
88

99
See `docs/benchmarks/methodology.md` for the competitor, scenario, input-size, and metric contract.

docs/benchmarks/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ This section is a design and generation foundation. It does not publish benchmar
77
## Files
88

99
- [Benchmark methodology and result contract](./methodology.md): competitor set, scenario matrix, input-size matrix, metric definitions, and placeholder-data policy.
10-
- [Result JSON schema](./results.schema.json): machine-readable contract for benchmark output and downstream chart data.
10+
- [Result JSON schema](/benchmarks/results.schema.json): machine-readable contract for benchmark output and downstream chart data.
1111
- [Visual proof system](./visual-proof-system.md): palette, typography, composition, diagram style, chart rules, social-card constraints, and README proof strip guidance.
1212
- [Visual proof prompt kit](./prompt-kit.md): reusable prompts for benchmark infographics, explainability trace diagrams, playground README GIF storyboards, AI-rule safety carousels, and README proof strips.
1313
- [Asset folder recommendation](./assets/): recommended `docs/benchmarks/assets/` structure and data-integrity rules.

docs/benchmarks/methodology.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ The harness/result contract covers exactly these engines for the first public pr
1616
| Hand-coded TypeScript | `hand-coded-typescript` | Baseline for direct conditional logic without engine overhead. |
1717
| rule-engine-js | `rule-engine-js` | Smaller modern competitor selected because it installs/builds in this repository. |
1818

19-
`rulepilot` remains an alternate candidate only if `rule-engine-js` becomes infeasible later. Do not mix both in the same first chart set without updating `docs/benchmarks/results.schema.json`.
19+
`rulepilot` remains an alternate candidate only if `rule-engine-js` becomes infeasible later. Do not mix both in the same first chart set without updating `docs/public/benchmarks/results.schema.json`.
2020

2121
## Scenario matrix
2222

@@ -37,7 +37,7 @@ The harness/result contract covers exactly these engines for the first public pr
3737

3838
## Stable result fields
3939

40-
Every result row must contain these fields. Units and source definitions are duplicated in `docs/benchmarks/results.schema.json` for machine consumers.
40+
Every result row must contain these fields. Units and source definitions are duplicated in `docs/public/benchmarks/results.schema.json` for machine consumers.
4141

4242
| Field | Unit | Source |
4343
| --- | --- | --- |
@@ -79,14 +79,14 @@ A future executable harness should:
7979
3. Execute `smoke`, `small`, and `medium` profiles by default; gate `large` behind an explicit flag.
8080
4. Measure cold start separately from warm throughput.
8181
5. Measure validation and explanation overhead as delta timings against the same scenario/input profile.
82-
6. Emit JSON matching `docs/benchmarks/results.schema.json`.
82+
6. Emit JSON matching `docs/public/benchmarks/results.schema.json`.
8383
7. Set `result_kind: "actual_benchmark"`, `is_placeholder: false`, and `claims_allowed: true` only for real measured output.
8484

8585
## Visual asset consumers
8686

8787
The first visual asset bundle can safely bind to these paths:
8888

89-
- Schema: `docs/benchmarks/results.schema.json`
89+
- Schema: `docs/public/benchmarks/results.schema.json`
9090
- Placeholder sample: `benchmarks/sample-results.placeholder.json`
9191
- Methodology: `docs/benchmarks/methodology.md`
9292

docs/benchmarks/results.schema.json renamed to docs/public/benchmarks/results.schema.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,8 @@
7070
"explanation_overhead_ms",
7171
"node_version",
7272
"package_version",
73-
"commit_sha"
73+
"commit_sha",
74+
"result_kind"
7475
],
7576
"properties": {
7677
"engine": {

0 commit comments

Comments
 (0)