Why
evaluate_claim currently returns the verification_mode and the result, but not the per-mode parameters Cluster A chose (json_path, expected_json_schema, range_min, range_max, query_record_id).
That makes downstream audit/replay incomplete: a consumer storing the verdict can record that a claim CAUGHT but not how the comparison was framed. Without those fields, replaying a failed run requires keeping the original HandoffClaim somewhere external to the verdict.
What
Extend the dict returned by provably.handoff.eval_modes.evaluate_claim (and the corresponding return shape used in handoff/_http.py) to include:
json_path
expected_json_schema (nullable)
range_min, range_max (nullable)
query_record_id
verification_mode is already there. Existing result, claimed, indexed, indexed_at_path, detail keys stay as-is.
Acceptance
- New keys present on every returned verdict (null when not applicable).
- Tests in
tests/unit/test_eval_modes* updated to assert the new keys.
- No change to
HandoffClaim shape; this is purely the eval response.
Downstream (out of scope here)
Once this lands, the monorepo can extend its agent_steps audit table to persist these fields. Tracked separately if needed.
Why
evaluate_claimcurrently returns theverification_modeand the result, but not the per-mode parameters Cluster A chose (json_path,expected_json_schema,range_min,range_max,query_record_id).That makes downstream audit/replay incomplete: a consumer storing the verdict can record that a claim CAUGHT but not how the comparison was framed. Without those fields, replaying a failed run requires keeping the original
HandoffClaimsomewhere external to the verdict.What
Extend the dict returned by
provably.handoff.eval_modes.evaluate_claim(and the corresponding return shape used inhandoff/_http.py) to include:json_pathexpected_json_schema(nullable)range_min,range_max(nullable)query_record_idverification_modeis already there. Existingresult,claimed,indexed,indexed_at_path,detailkeys stay as-is.Acceptance
tests/unit/test_eval_modes*updated to assert the new keys.HandoffClaimshape; this is purely the eval response.Downstream (out of scope here)
Once this lands, the monorepo can extend its
agent_stepsaudit table to persist these fields. Tracked separately if needed.