Skip to content

Add H2 order-control seed policy#311

Merged
DeliciousBuding merged 1 commit into
mainfrom
h2-order-control-scout-20260525
May 24, 2026
Merged

Add H2 order-control seed policy#311
DeliciousBuding merged 1 commit into
mainfrom
h2-order-control-scout-20260525

Conversation

@DeliciousBuding

Copy link
Copy Markdown
Owner

Summary

  • add a bounded H2 seed-offset policy for shared-position order-control response-cache scouts
  • mark non-default seed policies as control-cache scouts in summary output
  • document that this does not admit Platform or Runtime rows

Tests

  • python -X utf8 -m unittest tests.test_run_h2_response_strength_validation
  • python -X utf8 tests/test_review_h2_output_cloud_geometry_script.py
  • python -X utf8 -m unittest tests.test_run_h2_response_strength_validation tests.test_review_h2_output_cloud_geometry_script tests.test_evaluate_h2_response_cache_script tests.test_h2_response_strength

Copilot AI review requested due to automatic review settings May 24, 2026 21:56
@DeliciousBuding DeliciousBuding merged commit 6912fbc into main May 24, 2026
2 of 3 checks passed
@DeliciousBuding DeliciousBuding deleted the h2-order-control-scout-20260525 branch May 24, 2026 21:57

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a --seed-offset-policy parameter to the H2 response strength validation script, enabling a 'shared-position' mode for order-control scouting. The changes include updates to the documentation, seed offset calculation logic, and the final summary output, along with new unit tests. Review feedback suggests refining the scout detection logic to be more explicit and enhancing the verdict string to include validation results for scout runs to improve debugging clarity.

raw_best_simple_auc=raw_best_simple_auc,
)
validation_passed = bool(validation_gate["validation_passed"])
is_order_control_scout = args.seed_offset_policy != "class-ordered"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic for determining if a run is an order_control_scout is slightly brittle. Currently, it assumes any policy other than the default class-ordered is a scout. If additional policies are added in the future that are intended for admission (not just scouting), this logic will incorrectly mark them as scouts. It would be safer to explicitly check for the known scout policies.

Suggested change
is_order_control_scout = args.seed_offset_policy != "class-ordered"
is_order_control_scout = args.seed_offset_policy == "shared-position"

Comment on lines +289 to +295
"verdict": (
"order-control scout generated"
if is_order_control_scout
else "positive but bounded validation"
if validation_passed
else "negative but useful"
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The verdict string currently prioritizes the scout status over the validation result. While scouts are indeed not for admission, it might be useful for debugging and clarity to include whether the scout run actually passed the validation gate in the verdict string itself, rather than just in the validation_passed boolean field.

Suggested change
"verdict": (
"order-control scout generated"
if is_order_control_scout
else "positive but bounded validation"
if validation_passed
else "negative but useful"
),
"verdict": (
f"order-control scout generated ({'passed' if validation_passed else 'failed'})"
if is_order_control_scout
else "positive but bounded validation"
if validation_passed
else "negative but useful"
),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant