Skip to content

Commit 6d6f47e

Browse files
author
Akkari
committed
docs: update README and CHANGELOG for crash resilience, cost tracking, and audit reports
All v0.5.3 features now documented before validation run: - Crash-resilient batch validation (progressive saves, --resume, graceful shutdown) - Cost tracking (--max-cost, pre-run estimates, per-call tracking) - CSV audit reports (--audit-report, SHA-256, timing, cost breakdown) - PSScriptAnalyzer command injection fix - SEC-02 additions (severity calibration, manifest example, depth mapping) - Full CLI flag documentation
1 parent df541ef commit 6d6f47e

1 file changed

Lines changed: 25 additions & 1 deletion

File tree

docs/CHANGELOG.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,10 +32,34 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
3232
- All three tools follow the DevSkim pattern: detect if installed, run if available, graceful degradation if not
3333
- Pre-screen now has 5 layers: built-in regex (PS-001–010) → DevSkim → Ruff → Bandit → PSScriptAnalyzer
3434

35+
#### Crash-Resilient Batch Validation
36+
- **Progressive manifest**: `batch-manifest.json` updates atomically after each file completes (not just at the end). Tracks `completed_files`, `failed_files`, `status` (running/completed/interrupted)
37+
- **Resume**: `quorum run --resume <batch-dir>` reads manifest, skips completed files, re-validates crashed entries, only pays for remaining work
38+
- **Graceful shutdown**: SIGTERM/SIGINT handler saves all in-flight work with `status='interrupted'` before exiting
39+
- **Progressive report**: `batch-report.md` grows as files complete instead of one big write at the end
40+
41+
#### Cost Tracking and Estimation (Milestone #16)
42+
- **Per-call tracking**: CostTracker records prompt/completion tokens and USD cost for every LLM call via `litellm.completion_cost()`
43+
- **Budget cap**: `--max-cost $10` stops gracefully when cumulative cost exceeds threshold — saves all work collected so far
44+
- **Pre-run estimate**: prints "Estimated cost: $X.XX (Y critic calls across Z files). Proceed? [Y/n]" — auto-skips under $0.50 or with `--yes`
45+
- **Cost report**: CLI output shows per-file cost breakdown; `run-manifest.json` includes full cost summary
46+
- Hardcoded rates for Claude, GPT, Gemini, Mistral families; graceful fallback for unknown models
47+
48+
#### CSV Audit Reports (Milestone #17)
49+
- **`audit-detail.csv`**: one row per file — SHA-256 hash, file size, token estimate, analysis start/end/duration, input/output tokens, tokens/sec, models used, verdict, finding count, cost
50+
- **`audit-summary.csv`**: single aggregate row — totals, avg/median tokens/sec, cost by model (JSON), pass/fail counts
51+
- `--audit-report` flag, auto-enabled at `--depth thorough`
52+
- SHA-256 artifact hashes stored in `run-manifest.json`
53+
54+
### Fixed
55+
- **PSScriptAnalyzer command injection**: single quotes in file paths now escaped before passing to `pwsh -Command`
56+
- **Learning memory save resilience**: `save()` wrapped in try/except to prevent crash on write failure
57+
3558
### Documentation
3659
- **docs/README.md** — new documentation index with categorized links to all framework docs
60+
- **docs/SEC02_BUSINESS_LOGIC_VALIDATION.md** — severity calibration table, shared manifest example, Option C success/failure criteria, depth-level mapping (per Devola's review)
3761
- **SPEC.md** — Security Critic links to SEC-02 workflow; Fixer updated to reflect re-validation loops; learning memory sections updated from "planned" to "shipped"
38-
- **README.md** — updated to v0.5.3: re-validation loops, learning memory, multi-layer pre-screen, PyPI install, new CLI flags
62+
- **README.md** — updated to v0.5.3: all new features, CLI flags, cost tracking, audit reports, crash resilience
3963

4064
### Roadmap Status
4165
- [x] Re-validation loops (Milestone #5c)

0 commit comments

Comments
 (0)