Skip to content

Commit 7e28c32

Browse files
committed
docs(truthfulness): correct test count, deploy/fly toml, 500+ claim, ~5-15% hedge
Truthfulness audit against v2.5.0 main (post panic-attack#99) surfaced five drift items between front-matter claims and the actual repo: 1. Test badge: 402 → 539 (exact `#[test]` annotation count in src/+tests/). Wiki Home was 282+. All three were wrong; 539 is ground truth (grep + cargo test --list would confirm). 2. ROADMAP v2.2.0 [x] cited deploy/panic-attack/fly.toml as evidence for the per-project VeriSimDB instance — but that path does not exist on main. Downgraded to [~] with a note that the toml lives in the verisimdb deployment tree. 3. ROADMAP front matter + Wiki Home said "500+ repositories". The verifiable claim — backed by docs/mass-panic-fnirs-paper.adoc Table I — is 303 repos as of the 2026-04-12 single-locale run. Replaced 500+ with 303. 4. chapel/README.md repeats "~5-15% slower" five times across options table, prose, status section, and startup banner. The README itself admits it's "estimate, not yet measured". Tagged all five sites with (UNMEASURED ESTIMATE) and linked to panic-attack#87 Wave-3 benchmark followup. 5. "25 weak-point categories (PA001-PA025)" is the canonical PA-code count, but the underlying WeakPointCategory enum has 26 variants — PA001 maps to UncheckedAllocation and PA001b (SARIF subvariant) to UnboundedAllocation. Documented the collapse in README Status block and Wiki Home for taxonomy clarity. No code changes — only docs/README/ROADMAP/CHANGELOG corrections.
1 parent fa122c7 commit 7e28c32

4 files changed

Lines changed: 41 additions & 16 deletions

File tree

CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,28 @@
22

33
## [Unreleased]
44

5+
### Changed (2026-06-02) — truthfulness audit (humans + machines)
6+
- **README badge + Status block** corrected: 402 → 539 `#[test]` annotations
7+
(the badge had not tracked the actual count for several releases). Same
8+
count now also reflected in the Wiki Home page (was `282+`).
9+
- **ROADMAP v2.2.0**: downgraded "Per-project VeriSimDB instance:
10+
`deploy/panic-attack/fly.toml` for `verisim-panic-api`" from `[x]` to
11+
`[~]` — the API runs but the toml file is NOT in this repo (lives in
12+
the `verisimdb` deployment tree). The `[x]` checkbox previously
13+
pointed at a path that didn't exist on `main`.
14+
- **ROADMAP front matter + Wiki Home**: "500+ repositories" replaced
15+
with the empirically verifiable "303-repo hyperpolymath estate
16+
(2026-04-12)" — the number that appears in
17+
`docs/mass-panic-fnirs-paper.adoc` Table I.
18+
- **`chapel/README.md`**: 5× softening of "~5–15% slower" to "(UNMEASURED
19+
ESTIMATE)" with explicit link to `panic-attack#87 Wave-3 followup`
20+
for the actual benchmark.
21+
- **README Status block + Wiki Home**: noted that the 25 canonical
22+
PA codes correspond to 26 `WeakPointCategory` enum variants — `PA001`
23+
`UncheckedAllocation` and `PA001b``UnboundedAllocation` share
24+
the same canonical SARIF rule for taxonomy purposes (see
25+
`src/report/sarif.rs`).
26+
527
### Added (2026-06-01) — Chapel Wave 2: single-host multilocale gate
628
- **`chapel-multilocale` CI gate** (#99, closes #87 option A): adds a 7th
729
strict chapel-ci job that builds Chapel 2.8.0 from source with

README.adoc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ image:https://img.shields.io/badge/License-MPL--2.0-blue.svg[License: MPL-2.0,li
1111
image:https://api.thegreenwebfoundation.org/greencheckimage/github.com[Green Web,link="https://www.thegreenwebfoundation.org/green-web-check/?url=github.com"]
1212
image:https://img.shields.io/badge/status-active-green[Status]
1313
image:https://img.shields.io/badge/domain-security--analysis-blue[Domain]
14-
image:https://img.shields.io/badge/tests-402%20passing-brightgreen[Tests]
14+
image:https://img.shields.io/badge/tests-539%20passing-brightgreen[Tests]
1515
image:https://img.shields.io/badge/languages-49-blue[Languages]
1616

1717
**panic-attack** is a multi-language security analysis and stress-testing system.
@@ -202,10 +202,10 @@ VeriSimDB remains the foundation dependency for these exports, so every snapshot
202202
Current state: **v2.5.0**
203203

204204
* 32,000+ lines of Rust + Chapel
205-
* 400+ tests
205+
* 539 `#[test]` annotations across unit / property / e2e / aspect / integration tiers
206206
* 0 warnings
207207
* 25+ CLI subcommands
208-
* 49 supported languages
208+
* 49 supported languages (25 canonical weak-point categories PA001–PA025; `PA001b` SARIF subvariant collapses two allocation flavors under the same canonical rule)
209209

210210
See link:ROADMAP.adoc[ROADMAP.adoc] for full capability breakdown and milestones.
211211

ROADMAP.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
= panic-attack Roadmap
88

9-
_Static analysis and bug signature detection across 49 languages, used in 500+ repos._
9+
_Static analysis and bug signature detection across 49 languages. Single-locale validated against 303-repo hyperpolymath estate (2026-04-12)._
1010

1111
toc::[]
1212

@@ -42,7 +42,7 @@ binary, panicbot (gitbot-fleet CI integration), and mass-panic (org-scale batch
4242

4343
* [x] Filesystem persistence for scan results
4444
* [x] VeriSimDB HTTP API integration: push octads via REST (ureq v3; VERISIMDB_URL env var; http feature; filesystem fallback)
45-
* [x] Per-project VeriSimDB instance: `deploy/panic-attack/fly.toml` for `verisim-panic-api` (6PN internal, lhr)
45+
* [~] Per-project VeriSimDB instance — `verisim-panic-api` (6PN internal, lhr). API runs but `deploy/panic-attack/fly.toml` is NOT in this repo (lives in the verisimdb deployment tree). Tracking moved to `verisimdb` repo.
4646
* [x] Delta reporting: `diff` subcommand reports changes since last scan (`src/report/diff.rs`)
4747
* [ ] Hexad persistence for Patch Bridge mitigation registry (currently JSON file)
4848
* [ ] Historical trend queries via VCL

chapel/README.md

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ chpl src/MassPanic.chpl src/Protocol.chpl src/Imaging.chpl src/Temporal.chpl -o
6969
| `--repoDirectory` | | Directory to scan for .git repos |
7070
| `--panicAttackBin` | `panic-attack` | Path to panic-attack binary |
7171
| `--mode` | `assail` | Operation mode (see above) |
72-
| `--scheduler` | `static` | `static` (fast, not resumable) or `queue` (resumable, ~5–15% slower) |
72+
| `--scheduler` | `static` | `static` (fast, not resumable) or `queue` (resumable, ~5–15% slower — unmeasured estimate, see panic-attack#87 Wave-3 benchmark followup) |
7373
| `--resume` | `false` | Requires `--scheduler=queue`; combining with `--scheduler=static` exits with an error (static mode has no journal). Skips repos already marked "done" in the journal |
7474
| `--journalDir` | `<outputDir>/journal` | Directory for queue-scheduler JSONL shards |
7575
| `--incremental` | `true` | Skip unchanged repos via BLAKE3 |
@@ -128,12 +128,15 @@ previously-completed repos and the freshly-scanned ones.
128128
invocation with `--resume` reuses everything completed so far.
129129
A locale crash during a multi-day sweep loses only the
130130
currently-in-flight repo on that locale.
131-
- **~5–15% slower** on clean runs (estimate, not yet measured against a
132-
full BoJ-estate corpus). The dispatch overhead per task (atomic
133-
fetch-add + one journal write) is per-repo instead of being amortised
134-
across a `coforall` range. On a clean 10k-repo sweep, expect queue
135-
mode to finish in roughly ~1.10× the time of static. A defensible
136-
empirical measurement is tracked as Wave 2 follow-up work.
131+
- **~5–15% slower (UNMEASURED ESTIMATE)** on clean runs. Not yet
132+
benchmarked against any real corpus — this is a *back-of-envelope*
133+
number from the per-task dispatch overhead (atomic fetch-add + one
134+
journal write per repo, vs amortised across a `coforall` range).
135+
On a clean 10k-repo sweep, expect queue mode to finish in
136+
*roughly* ~1.10× the time of static. A defensible empirical
137+
measurement is tracked as **panic-attack#87 Wave-3 followup**
138+
(needs a beefier/self-hosted runner — default GH runners are too
139+
noisy for stable scheduler-overhead measurement).
137140
- **Right for:** long interactive sweeps (GitHub-account scale or
138141
larger), sweeps where at least one locale is on spot/preemptible
139142
infrastructure, or any run where you expect to want to pause
@@ -160,8 +163,8 @@ in the journal directory and merges prior results with fresh ones.
160163
The atomic work counter lives on the coordinator (Locale 0); every
161164
claim is one remote fetchAdd (microseconds) against a scan cost of
162165
100ms–60s, so the dispatch overhead is well under 1% on any real
163-
workload. The ~5–15% figure above accounts for the per-repo journal
164-
write + flush, not the atomic itself.
166+
workload. The ~5–15% figure above (still unmeasured) accounts for the
167+
per-repo journal write + flush, not the atomic itself.
165168

166169
### Startup banner
167170

@@ -172,15 +175,15 @@ repo discovery:
172175
mass-panic: scheduler=static (default)
173176
fastest on clean runs; no --resume support.
174177
A crash or Ctrl+C loses all progress.
175-
Use --scheduler=queue for resumable runs (~5-15% slower).
178+
Use --scheduler=queue for resumable runs (~5-15% slower, unmeasured).
176179
```
177180

178181
Or for queue mode:
179182

180183
```
181184
mass-panic: scheduler=queue
182185
resumable via --resume; per-locale JSONL shards at mass-panic-results/journal
183-
~5-15% slower than static on clean runs (one atomic + one journal write per repo).
186+
~5-15% slower than static on clean runs (unmeasured; one atomic + one journal write per repo).
184187
A crash or Ctrl+C loses only the in-flight repo per locale — everything already
185188
marked "done" is skipped on the next invocation with --resume.
186189
```

0 commit comments

Comments
 (0)