Optimize test-coverage-improver prompt caching and run cadence#3298
Conversation
test-coverage-improver prompt caching and run cadence
There was a problem hiding this comment.
Pull request overview
Optimizes the test-coverage-improver workflow to improve Claude prompt prefix caching, reduce tool surface, and lower run cadence from twice daily to once daily.
Changes:
- Moved run-specific coverage context to the end of the prompt and removed the full
COVERAGE_JSONinterpolation so static guidance can be cached as a prefix. - Dropped redundant
npm run buildandnpm run test:coveragebash permissions; keptnpm run test/lint. - Changed schedule from
0 8,20 * * *to0 8 * * *and regenerated the lock file accordingly.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/test-coverage-improver.md | Reordered prompt, removed COVERAGE_JSON step/interpolation, trimmed bash tools, updated schedule and validation guidance. |
| .github/workflows/test-coverage-improver.lock.yml | Regenerated lock to reflect frontmatter changes (cron, allow-tool list, removed coverage-summary step/env). |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 0
Smoke Test Results
Status: 2/3 tests passed
|
🔍 Smoke Test Results
Overall: FAIL — Workflow template variables (
|
Smoke Test: Copilot BYOK (Offline) Mode
Overall: FAIL — pre-computed smoke data was not injected (template variables unexpanded); GitHub MCP returned 401.
|
Smoke TestMerged PRs: chore: recompile test-coverage-improver lock file; chore: run test coverage improver twice daily Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
|
Gemini Smoke Test: FAIL (SSL 35, MCP 401) Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
Smoke Test Results
Overall: FAIL —
|
test-coverage-improverwas spending most of its token budget on uncached ambient prompt context: dynamic coverage data was injected into the middle of the prompt, forcing every request to miss Claude prefix caching. The workflow also exposed redundant expensive bash commands and ran twice daily despite low incremental value.Prompt cacheability
## Current Coverage Statusblock to the end of the workflow body so the static guidance becomes a reusable prompt prefix across runs.COVERAGE_JSONinterpolation from the prompt and from the pre-step outputs. The workflow now keeps only the human-readable summary and the prioritized low-coverage list in prompt context.Tool surface reduction
npm run buildnpm run test:coveragenpm run test,npm run lint, and file inspection commands so the agent can still iterate on tests without re-running the full precomputed pipeline.Run frequency
Prompt guidance
coverage/coverage-summary.jsonavailable only as an on-demand file read.Example of the prompt reshaping: