Record H2 img2img output-cloud portability boundary#325
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a portability review for H2 output-cloud geometry on image-to-image (img2img) response caches. It includes a new evaluation script, review_h2_img2img_output_cloud_portability.py, and detailed documentation of the findings, which conclude that the geometry is weak or unstable on SD/CelebA caches compared to simple-distance baselines. The project roadmap, reproduction status, and implementation queue have been updated to reflect that this method remains a research-side diagnostic rather than a production candidate. A review comment correctly identified a potential runtime error in the new script when reference metrics are missing and provided a code suggestion to improve robustness.
| max_simple_distance_auc_delta = max( | ||
| float(packet["comparison"]["output_cloud_minus_best_simple_distance"]["auc"]) | ||
| for packet in packets | ||
| if packet["comparison"]["output_cloud_minus_best_simple_distance"] is not None | ||
| ) |
There was a problem hiding this comment.
The max() function will raise a ValueError if the generator expression is empty, which occurs if the reference simple-distance-review.json files are missing for both the admission and stability caches. Adding a default value ensures the script remains robust even when reference metrics are unavailable.
| max_simple_distance_auc_delta = max( | |
| float(packet["comparison"]["output_cloud_minus_best_simple_distance"]["auc"]) | |
| for packet in packets | |
| if packet["comparison"]["output_cloud_minus_best_simple_distance"] is not None | |
| ) | |
| max_simple_distance_auc_delta = max( | |
| ( | |
| float(packet["comparison"]["output_cloud_minus_best_simple_distance"]["auc"]) | |
| for packet in packets | |
| if packet["comparison"]["output_cloud_minus_best_simple_distance"] is not None | |
| ), | |
| default=0.0, | |
| ) |
Summary: Adds a CPU-only existing-cache review for H2 output-cloud geometry on the existing SD/CelebA img2img response caches. The result is weak or unstable on the admission cache and not distinct from simple distance, so this narrows H2 output-cloud to a Research-side response-strength diagnostic only. No Runtime runner, Platform row, admitted bundle change, model download, GPU task, or img2img sweep is selected.
Checks: review_h2_img2img_output_cloud_portability.py smoke run; python -X utf8 scripts/check_markdown_links.py; python -X utf8 scripts/check_public_surface.py; python -X utf8 scripts/export_admitted_evidence_bundle.py --check; python -X utf8 scripts/run_pr_checks.py.