Skip to content

coverage_scoreboard.md

Latest commit

 

History

History
47 lines (34 loc) · 2.16 KB

coverage_scoreboard.md

File metadata and controls

47 lines (34 loc) · 2.16 KB
Raw

Coverage Scoreboard

Where we stand (live, auto-generated)

These two reports are the authoritative answer to "what is modelled and how well is it validated." Both regenerate from the chip definitions + real CI runs and are auto-committed nightly — they do not drift:

  • Chip conformance scoreboard — per chip: conformance level (L0 estate · L1 +registers-vs-silicon · L2 +behavior), the number of modelled peripherals, register-match % against silicon captures, and the behavior gate test. Generated by the chip_conformance test from configs/chips/*.yaml; refreshed + committed by core-nightly.yml.
  • Tier-1 validation matrix — per chip × per subsystem (clock/gpio/uart/timer/dma/irq/i2c/spi/adc/pwm/wdt/rtc): pass / partial / not-modelled, each cell linking the CI run that produced it. Generated by tier1-matrix nightly.

The per-board docs/boards/*.md "Status at a glance" tables are hand-maintained prose snapshots — treat the two scoreboards above as the source of truth when they disagree.

Matrix smoke gate (live)

The core-coverage-matrix-smoke.yml workflow runs a deterministic smoke matrix over the executable targets and publishes a scoreboard artifact on every run:

  • coverage-matrix-scoreboard/scoreboard.md
  • coverage-matrix-scoreboard/scoreboard.json

Both are summarized in the GitHub Actions step summary. The hard gate: the executable deterministic targets must maintain a ≥80% pass rate, enforced in the workflow's scoreboard job. The per-run pass/fail numbers live in the artifact and step summary — this doc no longer freezes a snapshot of them.

Update Protocol

  1. Matrix CI uploads per-target artifacts under out/coverage-matrix/<target-id>/.
  2. Matrix CI aggregates and publishes scoreboard artifacts via scripts/generate_coverage_matrix_scoreboard.py.
  3. Update this file when the Top-5 scope, thresholds, or status policy changes.
  4. Do not mark green without deterministic re-run evidence.

Tier-1 Validation Matrix

Per-chip, per-peripheral real-firmware validation: tier1-scoreboard.