Skip to content

Sync launchable notebooks with source updates and add Mean-Variance s…#39

Merged
phuo-nv merged 4 commits into
mainfrom
features/launchable-mean-variance-and-cuopt-label
Apr 22, 2026
Merged

Sync launchable notebooks with source updates and add Mean-Variance s…#39
phuo-nv merged 4 commits into
mainfrom
features/launchable-mean-variance-and-cuopt-label

Conversation

@phuo-nv
Copy link
Copy Markdown
Collaborator

@phuo-nv phuo-nv commented Apr 22, 2026

Summary

  • Sync notebooks/launchable.ipynb and brev/launchable-content.ipynb with their source notebooks (cvar_basic, efficient_frontier, rebalancing_strategies) so updates introduced in Add Mean-Variance optimizer, refactor to shared base classes, and update notebooks #32 / Require python 3.11 in order to resolve conflicts with UV sync command #34 / fix notebook cuOpt presolve setting: False -> 0 for cuOpt 26.4 compat #36 / notebook API compat for cuOpt 26.4 + pydantic settings migration #38 propagate: new colored callout <div> styling on every section heading, regime_name rename, ApiSettings(...) pydantic model, presolve: 0, num_scen adjustments, etc.
  • Add a new Mean-Variance Portfolio Optimization section (mean_variance_basic.ipynb content) to both launchable variants as ## 10. Mean-Variance Portfolio Optimization, including Mathematical Formulation, GPU solve via CVXPY + cuOpt Python API, Results, CPU comparison, and Summary table.
  • Renumber Advanced topics continuously so Efficient Frontier = ## 8, Rebalancing = ## 9, Mean-Variance = ## 10 (was restarting at 1). Both ToCs updated.
  • Heading-level cleanup: Strategy 1 / Strategy 2 are now ### under section 9 (were ##); Summary is now ### under section 10 with its "Mean-Variance vs CVaR" subheading bumped to ####.
  • Disambiguate the duplicate ### Install Dependencies headings in notebooks/launchable.ipynb and brev/launchable-setup.ipynb: the first one (CUDA version detection) is now ### Detect CUDA Version, the second (the actual install) keeps ### Install Dependencies. ToC updated.
  • Fix QA-test regression introduced in e65a857: compare_results() now displays the cuOpt GPU solver as cuOpt (GPU) regardless of whether it was invoked via CVXPY (str(cp.CUOPT)"CUOPT") or the cuOpt Python API (already "cuOpt"). Other CVXPY backends are suffixed (CPU). The previous output printed raw CUOPT which broke the QA test pattern r"cuOpt.*?(\d+\.\d+)" for two cases:
    • [cvar_basic] Solve CVaR Optimization, Compare results between GPU and CPU solvers
    • [launchable_brev] Comprehensive validation - CVaR, Efficient Frontier, Rebalancing
  • Drop redundant --- separators from Mean-Variance subsection cells (they came from source where each was top-level H2; here they're nested H3), the empty preamble cell, and a 4-newline gap inside the Mean-CVaR Introduction.
  • Fix brev/launchable-content.ipynb ToC formatting: extra --- between items 9 and 10 removed; the kernel-spec note is now properly separated.
  • Build cvxpy from upstream master instead of the PyPI wheel (via [tool.uv.sources] git source). The Mean-Variance Markowitz QP solve through cp.CUOPT relies on the set_quadratic_objective_matrix() wiring in cuopt_conif.py that's on master ahead of the next cvxpy release. Verified locally: cp.CUOPT solves the Markowitz QP and matches cp.CLARABEL to 1e-9 on objective and weights. uv.lock pins commit 766bdb92 (= cvxpy 1.9.0.dev0); bump with uv lock --upgrade-package cvxpy.

Test plan

  • uv lock --locked passes (165 packages, no resolution drift)
  • ruff format --check src/ tests/ passes (17 files already formatted)
  • ruff check src/ tests/ passes
  • pytest tests/ passes — 41 passed, 1 documented skip (was 38 before, +3 new tests in TestCompareResultsLabels that pin the QA pattern contract)
  • Heading hierarchy in both launchable notebooks has 0 H-level skips (perfect H1 → H2 → H3 → H4 nesting); 0 empty markdown cells; 0 back-to-back --- separator cells
  • Body cells of notebooks/launchable.ipynb and brev/launchable-content.ipynb are byte-identical (only the install/verify preamble differs, by design)
  • Anchors (<a id="mean-variance"> etc.) reachable from both ToCs
  • Post-merge: main.yml papermill run on GPU runner regenerates fresh outputs with cuOpt (GPU) labels — QA notebook tests for [cvar_basic] and [launchable_brev] should pass

Files changed

File Change
src/utils.py Add _solver_display_label() helper, use it in table rows and Objective Differences lines
tests/test_core.py Add TestCompareResultsLabels (3 tests pinning the QA-pattern contract)
notebooks/launchable.ipynb Rebuilt body from source notebooks; new MV section; renumbered Advanced topics; heading + line-break cleanup; disambiguated install headings
brev/launchable-content.ipynb Same body as launchable (minus install/verify preamble); ToC + heading cleanup
brev/launchable-setup.ipynb Disambiguated Install DependenciesDetect CUDA Version for the first heading

Note on committed notebook outputs

The .ipynb files still embed outputs from the previous compare_results call (showing CUOPT). The QA tests parse the re-executed *_result.html produced by main.yml's papermill step, not the committed outputs, so the test will pass after merge. Re-executing locally to refresh the committed outputs would take ~30 min on the H200 — happy to do that as a follow-up if reviewers prefer fresh outputs in the diff.

…ection

The launchable.ipynb and brev/launchable-content.ipynb stitched-notebook
copies had drifted from the source notebooks (cvar_basic, efficient_frontier,
rebalancing_strategies) and were missing the new mean_variance_basic content
entirely. This commit brings them back in sync and standardizes structure.

notebooks/launchable.ipynb and brev/launchable-content.ipynb:
- Pull cvar_basic / efficient_frontier / rebalancing_strategies cells fresh
  from source so the new <div> styling, regime renames, ApiSettings pydantic
  model, presolve: 0, and num_scen tweaks all propagate.
- Add a new Mean-Variance Portfolio Optimization section embedding
  mean_variance_basic content as section 10 (subsections demoted from H2 to
  H3 to nest cleanly).
- Renumber Advanced topics continuously: Efficient Frontier 1 -> 8,
  Rebalancing 2 -> 9, Mean-Variance -> 10. ToCs updated to match.
- Heading-level cleanup: Strategy 1 / Strategy 2 demoted from H2 to H3
  (subsections of Rebalancing); Summary demoted from H2 to H3 with its
  "Mean-Variance vs CVaR" subheading from H3 to H4.
- Drop redundant '---' separators from Mean-Variance subsections, the empty
  preamble cell, and a 4-newline gap in the Mean-CVaR introduction.
- Fix brev ToC formatting (extra '---' between items 9 and 10 removed; the
  kernel-spec note now properly separated).

brev/launchable-setup.ipynb and notebooks/launchable.ipynb:
- Rename the first '### Install Dependencies' heading to '### Detect CUDA
  Version' to disambiguate it from the actual install step that follows.
- Update launchable.ipynb's ToC under '0. Environment Setup' to list all
  three steps.

src/utils.py:
- Fix QA-test regression: compare_results now displays the cuOpt GPU solver
  as 'cuOpt (GPU)' regardless of whether it was invoked via CVXPY (cp.CUOPT
  -> str -> "CUOPT") or the cuOpt Python API (already "cuOpt"). Other
  CVXPY backends are suffixed '(CPU)'. The previous output printed the raw
  uppercase 'CUOPT' from str(cp.CUOPT) which broke the QA notebook test
  pattern r"cuOpt.*?(\d+\.\d+)".

tests/test_core.py:
- Add TestCompareResultsLabels covering the three label paths and pinning
  the QA test pattern to prevent future regression.

Made-with: Cursor
phuo-nv added 3 commits April 22, 2026 19:09
pydantic is a new direct runtime dependency (added in pyproject.toml under
[project].dependencies as pydantic>=2.12.5) and is the only new package not
already covered by the existing Apache 2.0 / BSD sections. Add a new MIT
License section listing pydantic with the standard MIT text, matching the
formatting of the existing sections.

Made-with: Cursor
When rebuilding the launchable notebooks I dropped efficient_frontier.ipynb
cell ef[3] (user inputs) and ef[7] (results_df display) on the assumption
ef[3] duplicated cvar_basic's user inputs. It does not — the EF section uses
its own ef_-prefixed variables (ef_dataset_name, ef_regime, ef_regime_dict,
ef_dataset_path, ef_returns_compute_settings, ef_scenario_generation_settings,
and the computed ef_returns_dict) that are not defined elsewhere. Without
that cell, the next cell's f-string referencing ef_dataset_name and ef_regime
raises NameError on execution.

Insert ef[3] back between the EF parameters cell and the
'### Using create_efficient_frontier' markdown, and ef[7] back at the end of
the EF section so the results DataFrame is displayed.

Both notebooks/launchable.ipynb and brev/launchable-content.ipynb get the
same fix.

Made-with: Cursor
The Mean-Variance section in mean_variance_basic.ipynb / launchable.ipynb
solves a Markowitz QP through cvxpy + cp.CUOPT. cuOpt's QP support via
CVXPY (data_model.set_quadratic_objective_matrix) is wired up in
cvxpy/reductions/solvers/conic_solvers/cuopt_conif.py on master, but the
last cvxpy PyPI release that contains the relevant fixes is not yet cut.

Add a uv source override that pins cvxpy to the github master branch:

  [tool.uv.sources]
  cvxpy = { git = "https://github.com/cvxpy/cvxpy.git", branch = "master" }

uv.lock now records the resolved commit SHA (currently 766bdb92) for
reproducibility. To bump:

  uv lock --upgrade-package cvxpy

Verified locally:
  - uv sync builds cvxpy from source successfully (~30s on this box)
  - cvxpy 1.9.0.dev0+0.766bdb9 imported, exposes CUOPT and CUCLARABEL
  - Toy Markowitz QP solved via cp.CUOPT matches cp.CLARABEL to 1e-9
    on both objective and weight values
  - uv lock --locked passes
  - ruff check / ruff format --check on src/ + tests/ passes
  - pytest tests/: 41 passed, 1 skipped (no regression)

Made-with: Cursor
@phuo-nv phuo-nv merged commit 1db4684 into main Apr 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant