Skip to content

Gate pdd checkup review-loop on claim evidence and unproven material claims #1066

@Serhan-Asad

Description

@Serhan-Asad

Parent: #1059
Depends on: #1064, #1065

Goal

pdd checkup --review-loop must report and gate claim evidence. A clean reviewer report is not enough when material claims are unproven.

Problem

Current checkup_review_loop.py can render issue_aligned: true when no remaining findings exist. It has no structured knowledge of claim evidence or UNPROVEN_CLAIM markers.

Required change

Extend review-loop state, parsing, prompts, and final report to include:

  • ### Claim Evidence Map
  • ### Unproven Claims
  • machine-readable unproven claim data in final state
  • issue_aligned: false when any material claim is pending or unproven

Reviewers must explicitly inspect:

  • what user claim could still fail in production
  • what edge case would make the PR look correct but still be wrong
  • whether tests prove user-visible behavior rather than implementation plumbing
  • multi-step, repeated-call, async, concurrent, aggregation paths when relevant

Parser/gate requirement

Do not rely only on markdown headings. The final report can render markdown, but the review-loop state/final JSON must carry structured unproven-claim information.

Acceptance criteria

  • Final report includes claim evidence and unproven claims.
  • Any material UNPROVEN_CLAIM forces issue_aligned: false.
  • Adapter-facing markdown still exposes enough information for downstream systems.
  • Tests cover parsing/rendering/gating of unproven claims.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions