Supply an initial xhat from a file (--xhat-from-file)#672
Merged
Conversation
Introduces --xhat-from-file, a single-flag feature on the xhat spokes that reads a .npy first-stage vector (via the existing ciutils.read_xhat helper), evaluates it across scenarios via Xhat_Eval.evaluate, and reports the resulting inner bound as the spoke's first candidate — before the normal xhatter exploration loop. Two uses motivate it: 1. User-supplied warm start / heuristic candidate. 2. End-to-end testing of xhat feasibility cuts (PR Pyomo#671): feed a known-infeasible binary xhat from a file, assert the cut lands and the xhat is not revisited. Design document covers the precedence rule with Jensen's --*-try-jensens-first (file beats Jensen's beats normal), multi-stage deferral (V1 is two-stage only, matching ciutils.read_xhat), the hook point (right after xhat_prep, mirrors Jensen's), CLI/cfg plumbing, and the first-milestone scope. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Design: doc/xhat_from_file_design.md. Every xhat spoke (xhatlooper, xhatshufflelooper, xhatspecific, xhatxbar) that descends from XhatInnerBoundBase now optionally reads a .npy file holding a first-stage xhat vector, evaluates it across all scenarios once via Xhat_Eval.evaluate, and reports the resulting inner bound before its normal exploration loop starts. - cfg.xhat_from_file_args() registers --xhat-from-file (str, default None = off). - XhatInnerBoundBase._try_file_xhat() loads via ciutils.read_xhat, validates (length vs root-node nonant count, two-stage only, file exists), calls self.opt.evaluate, calls self.update_if_improving if Eobj is finite, and restores nonants either way. Hard-fails on any precondition violation rather than silently skipping. - xhat_prep() calls the helper after _save_nonants, so every xhat spoke picks it up for free. - shared_options carries the path into every xhat spoke's options. - generic_cylinders wiring via mpisppy/generic/parsing.py. V1 is two-stage only, matching ciutils.read_xhat; multi-stage raises with a clear error and is listed as a follow-up. Tests: 11 unit tests in mpisppy/tests/test_xhat_from_file.py covering off-by-default, missing file, multi-stage reject, length mismatch, happy path with finite Eobj, infeasible-style (inf / None) Eobj, and sanity checks on the math.isfinite predicate. Existing xhat / PH tests (34 tests) still pass. User-facing doc/src/xhat_from_file.rst: covers the three use cases (warm start from prior run on a similar instance, user heuristic, testing infeasibility cuts), the flag, the format, precedence with Jensen's --*-try-jensens-first, and the interaction with --xhat-feasibility-cuts-count (PR Pyomo#671). Wired into index.rst. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #672 +/- ##
==========================================
+ Coverage 70.74% 70.81% +0.06%
==========================================
Files 153 153
Lines 19011 19040 +29
==========================================
+ Hits 13450 13483 +33
+ Misses 5561 5557 -4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Add three focused tests that bring patch coverage on Pyomo#672's diff to full: - TestConfigArgRegistration: direct call to Config.xhat_from_file_args plus a drive of generic.parsing.parse_args with a stub module so the cfg.xhat_from_file_args() call inside parse_args is exercised. - TestXhatPrepCallSite: stub self.opt with the minimum that xhat_prep touches and verify xhat_prep reaches self._try_file_xhat() (gated off, so the real helper early-returns). This covers the call site inside xhat_prep. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bknueven
approved these changes
Apr 24, 2026
The 14 unit tests in mpisppy/tests/test_xhat_from_file.py exercise every branch of XhatInnerBoundBase._try_file_xhat, but neither the regression job in .github/workflows/test_pr_and_main.yml nor the local run_coverage.bash invoked the file under coverage run. As a result Codecov saw the helper as 36.67% covered on PR Pyomo#672 — the covered lines being only the early-return path reached incidentally by other tests. Add a coverage-instrumented pytest invocation of test_xhat_from_file.py to both runners alongside the other serial pytest phases. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DLWoodruff
added a commit
to DLWoodruff/mpi-sppy-1
that referenced
this pull request
Apr 26, 2026
PR Pyomo#672's --xhat-from-file feeds a file-supplied xhat through Xhat_Eval.evaluate before the xhatter's main loop. Until now, an infeasible file-supplied xhat just produced an "Eobj=None; not updating inner bound" log line and discarded the candidate. The PR Pyomo#672 description called out this exact integration as a motivating use case: feed a known-infeasible binary xhat to drive PR Pyomo#671's feasibility-cut emission end-to-end. Wire the two: - Extract pack_no_good_feasibility_cut(opt) as a module-level helper in mpisppy/extensions/xhatbase.py (the existing XhatBase._maybe_emit_feasibility_cut becomes a thin wrapper, so the original call site in _try_one is unchanged). - In XhatInnerBoundBase._try_file_xhat, after evaluate, check no_incumbent_prob() != 0 (same predicate _try_one uses) and call pack_no_good_feasibility_cut before _restore_nonants. evaluate and the prob-check are wrapped in try/except so a stub Xhat_Eval without those methods (or an evaluate that raises on infeasibility) doesn't break the helper. Tests in mpisppy/tests/test_xhat_from_file.py (TestFileXhatEmitsFeasibilityCut, three cases): - Infeasible file xhat with cuts_count=1 emits exactly one cut on Field.XHAT_FEASIBILITY_CUT, with the correct no-good row ([constant, coefs...]) for the supplied xhat. Asserts the cut violates at its own xhat (sanity). - Feature off (cuts_count=0): no put_send_buffer call. - Feasible file xhat (Eobj finite, infeasP=0): no cut emitted, inner-bound update happens normally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DLWoodruff
added a commit
to DLWoodruff/mpi-sppy-1
that referenced
this pull request
Apr 26, 2026
Two changes bundled here: 1. Restrict V1 to two-stage at hub setup time. The no-good cut row encodes coefficients positionally against each scenario's nonant_indices. In two-stage every scenario shares the same ROOT nonants under nonanticipativity, so the same row applied to every scenario is consistent. In multi-stage, scenarios on different branches have different per-stage-2+ variables at the deeper indices, so the same row applied positionally lands coefficients on unrelated variables — the resulting "cut" on a divergent scenario is meaningless. A multi-stage-correct installer must group cuts by branch (deferred to a follow-up alongside V2). XhatFeasibilityCutExtension._assert_two_stage raises at setup_hub when opt.multistage is true. The two-stage gate runs before the binary check, so a multi-stage all-binary model still hits the multi-stage error. The user-facing doc/src/xhat_feasibility_cuts.rst and the design doc/xhat_feasibility_cuts_design.md are updated to reflect this: the previous "no-good cuts work in multi-stage from day one" claim was overstated — the cut form is multi-stage-valid but the installer is not, and V1 hard-fails rather than silently produce invalid relaxations. The V1/V2/V3 scope table gains a "Multi-stage" row; first-milestone test list swaps the multi-stage positive test for a multi-stage rejection test. 2. Wire test_xhat_feasibility_cuts.py into both coverage runners (regression job in .github/workflows/test_pr_and_main.yml and the local run_coverage.bash). Same gap PR Pyomo#672 had pre-fix: the test file wasn't invoked under coverage run, so Codecov never saw the feature exercised. Tests: TestMultiNodeStartupCheck → TestMultiStageRejection. New cases: _assert_two_stage raises with the expected message; setup_hub hits the multi-stage error before reaching the binary check. All 56 tests across test_xhat_feasibility_cuts.py and test_xhat_from_file.py pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design document:
doc/xhat_from_file_design.md— please read that first.Summary
Every xhat spoke now optionally reads a first-stage
xhatvector from a.npyfile, evaluates it across all scenarios once viaXhat_Eval.evaluate, and reports the resulting inner bound — before the spoke's normal exploration loop starts.--xhat-from-file <path>.ciutils.read_xhat).xhatlooper,xhatshufflelooper,xhatspecific,xhatxbar) — the helper lives onXhatInnerBoundBase.xhat_prep, so all four inherit it for free.Use cases
--xhat-feasibility-cuts-count(PR Optional xhatter feasibility cuts, V1 (binary first-stage) — addresses #601 #671), supply a deliberately infeasible binary vector to exercise the feasibility-cut emission path end-to-end.What this PR adds
cfg.xhat_from_file_args()registering the CLI flag.XhatInnerBoundBase._try_file_xhat()— loads viaciutils.read_xhat, validates (file exists, length matches root-node nonants, two-stage), callsself.opt.evaluate, reports viaself.update_if_improvingif finite, restores nonants either way.cfg_vanilla.shared_optionspropagates the path to every xhat spoke.generic_cylindersviampisppy/generic/parsing.py.doc/src/xhat_from_file.rst, wired intoindex.rst.Hard-fail paths
The feature refuses to silently skip; all of these raise at
xhat_preptime:Precedence with Jensen's (#657)
If both
--xhat-from-fileand--*-try-jensens-firstare set, file-supplied is evaluated first, then Jensen's, then normal exploration. Correctness is order-invariant (update_if_improvingkeeps whichever is best); the ordering is a predictability / log-readability choice.Test plan
ruff check mpisppy/— clean.python -m unittest mpisppy.tests.test_xhat_from_file— 11 tests pass. Covers off-by-default, missing file, multi-stage reject, length mismatch, happy path (finite Eobj →update_if_improving), infeasible-style (inf/None) Eobj, andmath.isfinitesanity.python -m unittest mpisppy.tests.test_ef_ph mpisppy.tests.test_ph_extensions— 34 existing xhat/PH tests still pass.cd doc && make html— no new warnings.Follow-ups
run_all.pyentry combining--xhat-from-file(a pre-computed infeasible binary xhat) with--xhat-feasibility-cuts-count=1to drive the feasibility-cut emission path end-to-end.🤖 Generated with Claude Code