Build NJS-GROWTH-07 benchmark harness, measured result charts, and AI-rule-safety carousel#15
Conversation
…y carousel Unblocks NJS-GROWTH-07 proof assets by producing measured benchmark data and publishing it, plus a qualitative AI-rule-safety carousel. Harness (benchmarks/, consumes built dist/, no src changes): - 5 engine adapters (@sebasoft/neuron-js, json-rules-engine, json-logic-js, hand-coded-typescript, rule-engine-js) over 3 example-backed scenarios (pricing-discount, eligibility-approval, workflow-routing) x 3 sizes. - Fairness gate: every engine must reproduce the scenario's canonical decision before timing. - Metrics: batched throughput/p50/p95, fresh-process cold start, esbuild minified bundle size, and Neuron-JS validateScript/explainExecution overhead deltas (competitors have no equivalent step -> 0 with a note). - run.ts emits benchmarks/results/latest.actual.json (result_kind=actual_benchmark, is_placeholder=false, claims_allowed=true). - Scripts: yarn benchmark, yarn benchmark:charts. esbuild devDependency added; biome now lints benchmarks/. Proof assets: - benchmarks/charts/generate.ts renders the 5 benchmark-chart-*.svg and the generated docs/benchmarks/results.md showcase from the same source file, so charts/table can never drift from measured data. Every visible number exists verbatim in the result file. - 5-slide AI-rule-safety carousel SVGs (qualitative, no numbers) + asset metadata; added methodology-card metadata. - README + VitePress nav/sidebar + benchmarks index link the results; methodology documents how each metric is measured. Tests: tests/contracts/benchmark-results.test.ts validates the committed result file against the schema, the full matrix, and the overhead-attribution rule. Privacy: scrubbed private chaos-vault paths and internal agent notes from the seven published docs/assets that leaked them onto the site. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive benchmark harness for the neuron-js rules engine, comparing it against competitors across multiple scenarios and profiles, and integrates the generated charts and results into the documentation. The review feedback highlights several key improvement opportunities to make the harness more robust and maintainable, including resolving a constructor argument mismatch in neuron-plugins.ts, adding a default error-handling case to the scenario switch statement in the neuron-js adapter, replacing fragile dynamic export indexing with named exports in cold-start-child.ts, extracting duplicated state-reading logic into a shared utility, and refactoring scenario decision functions to avoid hardcoded values.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- neuron-plugins: drop extraneous 6th constructor arg in resolveParam
(StateNumberParameter takes 5).
- neuron-js adapter: add default case throwing on unhandled scenario.
- cold-start: adapters now expose a named `adapter` export; cold-start-child
imports `{ adapter }` instead of Object.values(...)[0].
- extract shared readPath into harness/read-path.ts; reuse in neuron-plugins
and the hand-coded adapter (removes duplication).
- scenarios: pricing decide/canonical derive from shared PRICING_* constants
instead of a hardcoded subtotal duplicated in data.
Re-ran yarn benchmark + benchmark:charts; lint/test/examples/build/docs:build
all green; no chaos-vault strings in dist.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Summary
Unblocks the NJS-GROWTH-07 proof assets by building a real benchmark harness, running it to produce measured
actual_benchmarkdata, and publishing a results showcase generated from that data — plus a qualitative AI-rule-safety carousel. Grounded in the internal NJS-GROWTH-07 proof-assets brief: nosrc/changes, no release, and the live playground / README GIF stay deferred until the playground is stable.Benchmark harness (
benchmarks/, consumes builtdist/)@sebasoft/neuron-js,json-rules-engine,json-logic-js,hand-coded-typescript,rule-engine-js— over 3 example-backed scenarios (pricing-discount,eligibility-approval,workflow-routing) × 3 sizes (smoke/small/medium).validateScript/explainExecutionoverhead deltas (competitors have no equivalent step →0with a row note).run.tsemitsbenchmarks/results/latest.actual.json(result_kind=actual_benchmark,is_placeholder=false,claims_allowed=true).yarn benchmark,yarn benchmark:charts. AddedesbuilddevDependency; biome now lintsbenchmarks/.Proof assets
benchmarks/charts/generate.tsrenders the 5benchmark-chart-*.svgand the generated showcase pagedocs/benchmarks/results.mdfrom the same source file, so charts and table can never drift from measured data. Every visible number exists verbatim in the result file.Privacy
Scrubbed private
chaos-vault/...paths and internal agent notes from the 7 published docs/assets that were leaking them onto the live site (these rendered as visible page text, not just source comments).Validation
git diff --check✅yarn lint✅ (49 files, now includingbenchmarks/)yarn test✅ (13 files / 68 tests, incl. newtests/contracts/benchmark-results.test.ts)yarn examples✅yarn build✅yarn docs:build✅ — charts + carousel ship in the build; nochaos-vaultstrings indocs/.vitepress/distGuardrails
src/changes.actual_benchmarkoutput; placeholder fixture untouched.🤖 Generated with Claude Code