-
Notifications
You must be signed in to change notification settings - Fork 0
Add manifest-driven layered artifact generation command #131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,39 @@ | ||||||
| """Deterministic entrypoint for layered admissibility artifact regeneration.""" | ||||||
|
|
||||||
| from __future__ import annotations | ||||||
|
|
||||||
| import sys | ||||||
| from pathlib import Path | ||||||
|
|
||||||
| REPO_ROOT = Path(__file__).resolve().parents[1] | ||||||
| if str(REPO_ROOT) not in sys.path: | ||||||
| sys.path.insert(0, str(REPO_ROOT)) | ||||||
|
|
||||||
| from src.validation.degradation_curve_generator import DegradationCurveGenerator | ||||||
|
|
||||||
| CURVE_ID = "coding_workflow_pr_review_curve_v1" | ||||||
| OUTPUT_PATH = Path("artifacts/layered_admissibility_results.json") | ||||||
|
|
||||||
|
|
||||||
| def generate_layered_admissibility_artifact( | ||||||
| output_path: Path = OUTPUT_PATH, | ||||||
| *, | ||||||
| curve_id: str = CURVE_ID, | ||||||
| ) -> Path: | ||||||
| """Regenerate layered admissibility artifact from manifest-ordered fixtures.""" | ||||||
|
|
||||||
| generator = DegradationCurveGenerator() | ||||||
| fixtures = generator.fixtures_for_layered_admissibility_curve() | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The call to
Suggested change
|
||||||
| curve = generator.generate(fixtures, curve_id=curve_id) | ||||||
| generator.write_json(curve, output_path) | ||||||
| return output_path | ||||||
|
|
||||||
|
|
||||||
| def main() -> int: | ||||||
| output_path = generate_layered_admissibility_artifact() | ||||||
| print(output_path.as_posix()) | ||||||
| return 0 | ||||||
|
|
||||||
|
|
||||||
| if __name__ == "__main__": | ||||||
| raise SystemExit(main()) | ||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,39 @@ | ||||||||||||||||||||
| from __future__ import annotations | ||||||||||||||||||||
|
|
||||||||||||||||||||
| import json | ||||||||||||||||||||
| from pathlib import Path | ||||||||||||||||||||
|
|
||||||||||||||||||||
| from scripts.generate_layered_admissibility_artifact import ( | ||||||||||||||||||||
| CURVE_ID, | ||||||||||||||||||||
| generate_layered_admissibility_artifact, | ||||||||||||||||||||
| ) | ||||||||||||||||||||
|
Comment on lines
+6
to
+9
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Import
Suggested change
|
||||||||||||||||||||
| from src.validation.degradation_curve_generator import DegradationCurveGenerator | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ARTIFACT_PATH = Path("artifacts/layered_admissibility_results.json") | ||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Anchor
Suggested change
|
||||||||||||||||||||
| CANONICAL_FIXTURE_ID = "coding_workflow_pr_review_moderate_v1" | ||||||||||||||||||||
| CANONICAL_MODERATE_SCORE = 0.8333333333333334 | ||||||||||||||||||||
|
|
||||||||||||||||||||
|
|
||||||||||||||||||||
| def test_generated_artifact_matches_committed_artifact(tmp_path: Path) -> None: | ||||||||||||||||||||
| output_path = tmp_path / "layered_admissibility_results.json" | ||||||||||||||||||||
| generate_layered_admissibility_artifact(output_path) | ||||||||||||||||||||
|
|
||||||||||||||||||||
| generated = json.loads(output_path.read_text(encoding="utf-8")) | ||||||||||||||||||||
| committed = json.loads(ARTIFACT_PATH.read_text(encoding="utf-8")) | ||||||||||||||||||||
| assert generated == committed | ||||||||||||||||||||
|
|
||||||||||||||||||||
|
|
||||||||||||||||||||
| def test_generated_artifact_has_no_new_top_level_fields() -> None: | ||||||||||||||||||||
| generator = DegradationCurveGenerator() | ||||||||||||||||||||
| curve = generator.generate(generator.fixtures_for_layered_admissibility_curve(), curve_id=CURVE_ID) | ||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pass the absolute manifest path to
Suggested change
|
||||||||||||||||||||
|
|
||||||||||||||||||||
| generated_top_level = set(generator.to_dict(curve).keys()) | ||||||||||||||||||||
| committed_top_level = set(json.loads(ARTIFACT_PATH.read_text(encoding="utf-8")).keys()) | ||||||||||||||||||||
| assert generated_top_level == committed_top_level | ||||||||||||||||||||
|
|
||||||||||||||||||||
|
|
||||||||||||||||||||
| def test_moderate_fixture_score_remains_canonical() -> None: | ||||||||||||||||||||
| payload = json.loads(ARTIFACT_PATH.read_text(encoding="utf-8")) | ||||||||||||||||||||
| points = payload["points"] | ||||||||||||||||||||
| moderate_point = next(point for point in points if point["fixture_id"] == CANONICAL_FIXTURE_ID) | ||||||||||||||||||||
| assert moderate_point["overall_admissibility_score"] == CANONICAL_MODERATE_SCORE | ||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
OUTPUT_PATHis currently defined as a relative path. Since this script is intended to be a deterministic entrypoint and already calculatesREPO_ROOT, it is more robust to anchor the output path to the repository root. This ensures the script behaves consistently regardless of the current working directory.