test: property-check forecast recommendation and summary contracts#599
test: property-check forecast recommendation and summary contracts#599ndycode wants to merge 2 commits into
Conversation
Seven fast-check properties over recommendForecastAccount, summarizeForecast, and buildForecastExplanation with generated result pools (0-8 accounts, full availability/risk/wait/flag space): - a recommendation always points at a recommendable result (not disabled, hard-failed, exhausted, or unavailable); null only when no such candidate exists - a ready candidate always wins with the minimal risk score among ready candidates, and the reason says so - with no ready candidate, the shortest delayed wait wins - an empty candidate pool names the actual blocker class: 'blocked or exhausted' guidance iff some account is blocked/exhausted rather than disabled/hard-failed (the #exhausted-flag regression guard) - the recommendation is invariant under input order - the summary partitions availability exactly (ready + delayed + unavailable === total) and counts high-risk rows - the explanation mirrors inputs in order and marks selected on exactly the recommended index Companion to the property suites in #574/#575/#592-#598. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
Warning Review limit reached
More reviews will be available in 6 minutes and 59 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more credits in the billing tab to continue. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ations Greptile flagged that availability and exhausted were drawn independently (evaluateForecastAccount only emits exhausted accounts as delayed) and that order-invariance only probed reversal. The generator now derives the pairing, and the invariance property checks an arbitrary generated permutation plus the reversal. Validated at FAST_CHECK_NUM_RUNS=1000. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB
Summary
recommendForecastAccount,summarizeForecast, andbuildForecastExplanationinlib/forecast.ts. These drive thecodex-multi-auth forecastrecommendation and its--explainoutput; the suite pins the selection policy over the full availability/risk/wait/flag space instead of hand-picked pools.What Changed
New
test/property/forecast.property.test.ts(7 properties over generatedForecastAccountResultpools, 0–8 accounts):exhausted-flag fix the code comments document (an all-exhausted pool must not produce a bogus "shortest wait" pick).ready + delayed + unavailable === totalexactly, with high-risk counted by risk level.consideredmirrors the inputs in order, andselectedis true on exactly the recommended index.No SUT bugs found.
Validation
npm test -- test/property/forecast.property.test.ts test/forecast.test.ts— 42/42 (new 7 + existing 35 untouched)npm run typecheck(also via pre-commit hook)npx eslint test/property/forecast.property.test.ts --max-warnings=0Docs and Governance Checklist
Risk and Rollback
test/property/suites.https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB
Generated by Claude Code
note: greptile review for oc-chatgpt-multi-auth. cite files like
lib/foo.ts:123. confirm regression tests + windows concurrency/token redaction coverage.Greptile Summary
adds a 7-property fast-check suite for the forecast engine's pure decision layer (
recommendForecastAccount,summarizeForecast,buildForecastExplanation), covering soundness, ready-dominance, delayed-fallback, blocker guidance, order-invariance, summary partition, and explanation fidelity. addresses both items from the previous review cycle: theexhausted → availability = "delayed"domain constraint is now enforced inarbResults, and order-invariance now usesfc.shuffledSubarrayfor arbitrary permutations rather than reversal alone.arbResultsgenerator derivesavailabilityfromexhaustedto restrict the test space to the reachable domain, preventing false positives if a future fast-path relies onexhausted ⟹ availability !== "ready".ready + delayed + unavailable === totaland pinssummary.ready, but does not individually assertsummary.delayedorsummary.unavailable, leaving a transposition of those two counts undetected.Confidence Score: 5/5
additive test-only file with no behavior or api surface changes; safe to merge.
the change is a single new test file exercising pure, side-effect-free functions. no production code is modified, no new dependencies are introduced, and the suite's 7 properties are internally consistent with the sut. the one gap (missing individual delayed/unavailable assertions in the summary test) is a coverage nit that doesn't risk a regression.
test/property/forecast.property.test.ts — the summary partition test could be tightened with individual delayed and unavailable count assertions.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[results pool] --> B{filter candidates} B -->|not disabled, not hardFailure, not exhausted, not unavailable| C[candidates] B -->|empty| D{hasBlockedOrExhausted?} D -->|yes| E[null + blocked-or-exhausted reason] D -->|no| F[null + no-healthy-accounts reason] C --> G[sort via compareForecastResults] G --> H{best availability} H -->|ready| I[pick best index - Lowest risk ready account] H -->|delayed| J[pick best index - pick shortest wait]Prompt To Fix All With AI
Reviews (2): Last reviewed commit: "test: keep forecast generator in the rea..." | Re-trigger Greptile