Polish cache-hit telemetry + spec reconcile & archive#52
Merged
Conversation
…eans
Audit found regen-pipeline was marked "complete" with all 24 tasks
checked, but none of its named symbols ever shipped in either repo
(attune-author: _regen, regen_template(corpus_root=...),
_resolve_corpus_root, atomic_write, _patch_summaries_json;
attune-gui: /api/config, /api/templates/refresh-all,
/api/browse/directory, CorpusSetup, App.jsx). A bogus "Shipped" note
had conflated it with the unrelated hash-mismatch regenerate CLI.
The 3 user stories are all satisfied by a more evolved architecture:
- regen: POST /api/living-docs/docs/{id}/regenerate (Jobs +
generate_feature_templates)
- corpus config: multi-corpus registry + workspace config
- bulk: make regen-all
No genuine product gaps remain. This commit corrects the spec docs:
- requirements.md: status -> reconciled, with user-story->reality map
- design.md: marked obsolete (assumes React/JSX + single corpus_root)
- tasks.md: flags the false done-marks and corrects the Shipped note
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…0.14.2) Status said "Implementation TBD" but the fix shipped in PR #48 (commit 1b1c7c5), released in 0.14.2: apply_polish_results now re-injects deterministic frontmatter via _replace_polished_frontmatter, with regression test tests/unit/test_polished_frontmatter_reinjection.py and a CHANGELOG entry. Status corrected to reflect shipped reality. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…metrics) Each polish run now tracks Anthropic prompt-cache token usage and logs a one-line summary at the end of `attune-author regenerate`: Polish cache hit: 87% (1241 read / 1421 total tokens, 6 call(s)) A WARNING is appended when the run's hit rate < 50% (with >=1 cacheable token), surfacing silent cache regressions (prompt edits, model alias drift). Hit rate = read / (read + creation) cacheable input tokens. Implementation: - doc_gen/_anthropic.call_anthropic gains an optional on_cache_usage(creation, read, model) callback; _log_cache_usage now returns (creation, read). Backward compatible — doc-gen passes nothing. - polish.py: PolishCacheStats dataclass, in-process accumulator (_polish_cache_telemetry / reset_polish_cache_telemetry, mirroring generator._faithfulness_telemetry), polish_cache_stats(), and format_polish_cache_summary(). _call_llm wires the callback. - maintenance.py: reset at run start, log summary at run end alongside the faithfulness summary. Deviation from the written spec: attune-author has no telemetry JSONL, so the metric follows the existing in-process faithfulness-counter pattern instead of a new JSONL subsystem; the threshold warning is current-run, not cross-run. Acceptance criteria in decisions.md all met. Tests: 16 new in tests/unit/test_polish_cache_metrics.py (callback firing incl. zero case, hit-rate math, accumulator, summary, warning). Docs: README "Cache hit rate" subsection; CHANGELOG [Unreleased]. Spec docs updated to DONE with the deviation noted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s change Auto-regenerated by the pre-commit help-freshness hook following the polish prompt-cache telemetry work (d4af5a3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move terminal specs into docs/specs/archive/ so they stop inflating the active count: polish-fact-check (v0.14.0), polish-cache-hit-metrics (done), regen-staleness-hash-mismatch (#48/0.14.2), regen-pipeline (superseded). skill-export-evangelism kept active (open). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e-metrics change" This reverts commit b9edfc4.
The _state() helper hardcoded submitted_at=2026-05-08, which silently expired past the 29-day retention window on 2026-06-06 and broke the status/cancel tests (they read batch state without an injected now=). Default to now-1day so the fixture stays inside the window. Fixes the 3 date-bomb failures in test_maintenance_batch.py (TestStatus/TestCancel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This session's attune-author work.
Feature — polish prompt-cache hit-rate telemetry (spec
polish-cache-hit-metrics):call_anthropicgains optionalon_cache_usagecallback;polish.pyaddsPolishCacheStats+ in-process accumulator; end-of-run summary logsPolish cache hit: NN% (...)with a <50% regression warning.Spec reconciliation:
regen-pipeline→ marked satisfied-by-different-means (never built as designed; need met by living-docs regen + corpus registry).regen-staleness-hash-mismatch→ marked DONE (shipped in fix: prevent LLM polish from laundering source_hash frontmatter #48 / v0.14.2).Archive: moved 4 terminal specs (polish-fact-check, polish-cache-hit-metrics, regen-pipeline, regen-staleness-hash-mismatch) into
docs/specs/archive/.skill-export-evangelismkept active (open, awaiting topic decision).🤖 Generated with Claude Code