Legacy specification: This document describes the archived one-session live experiment. Do not add its outcomes to the new 20-session evidence record. See the horizon decision and the 20-session redesign draft.
The live-forward experiment tests whether two independently tuned ranking models can select stocks using only information available at each decision time. It is an evidence-collection workflow, not yet a production trading recommendation. Ridge is the stable linear benchmark and the XGBoost ranker is the nonlinear challenger. Both select 40 equal-weight names and share SPY as the market benchmark.
This dashboard track preserves the legacy one-session selections for the rebalancing-policy study; it is separate from the primary horizon-matched 5/10/20-session live-forward experiment. The July 20–27 Ridge and XGBoost selections are frozen as permanent cohorts. After Yahoo Finance publishes the official daily close, run:
python -m src.cli run-live-forward-daily --as-of YYYY-MM-DDMonitoring-only is the default. The command:
- requests adjusted daily prices only for SPY and securities present in the archived cohorts;
- verifies the requested official close exists and fails closed if Yahoo still has partial data;
- extends every unchanged July 20–27 portfolio through the new close;
- replays archived rebalance policies for every interval from 1 through 60 sessions;
- recalculates net return, SPY-relative return, cumulative turnover, and trades; and
- rebuilds the local dashboard and sanitized GitHub Pages bundle.
It does not fit a model or create a new stock list. The output explicitly reports
"new_model_lists": 0.
The dashboard's Holding-period laboratory supports selecting any July 20–27 list, displaying Ridge and XGBoost together with clickable model chips, or replaying a 1–60-session rebalance interval. Five-, 10-, and 20-session presets are provided, and the horizon comparison table reports every 1–20-session candidate side by side. SPY is permanent and begins from the same zero-return decision-close baseline as each portfolio.
The main Portfolio vs Benchmark chart uses the archived one-session policy while frozen-cohort monitoring is active. It therefore extends through every new official close even though no new stock list is generated after July 27. The date controls and KPI cards are rebased to the selected monitoring window. The stock-level table and experiment history remain immutable records of the original one-session snapshots.
The displayed replacement turnover excludes the unavoidable initial purchase. An unchanged cohort therefore correctly shows 0% replacement turnover. Scheduled policies use the same $1 million capital, 8 bps one-way cost, $10 fixed trade fee, and 0.25% minimum trade-weight rule.
The old production-style retraining path remains available for reproducibility, but now requires
the explicit --retrain flag:
python -m src.cli run-live-forward-daily \
--as-of YYYY-MM-DD \
--retrain \
--tuning-folds 5 \
--ridge-optuna-trials 60 \
--xgb-optuna-trials 50 \
--optuna-timeout-seconds 7200Do not use this flag during the frozen-cohort monitoring phase. The focused parameter ranges come from earlier completed live runs. The current decision date and its future return remain excluded, so narrowing the search does not introduce look-ahead leakage. Five purged temporal folds provide a more stable selection estimate, while the separate Ridge and XGBoost trial budgets reflect their very different fitting costs.
Do not pay for the full audit automatically on every unchanged trading day. A routine daily refresh may use fewer trials inside the same validated ranges; repeat the five-fold audit after a material feature/data change or on a predeclared schedule. Any future warm-start or pruning optimization must preserve the same temporal validation and frozen-snapshot rules.
XGBoost also uses fold-safe early stopping:
- the last 21 dates of each fold's training block are reserved as an inner stopping window;
- the ranker stops after 50 rounds without validation-NDCG improvement;
- the outer temporal validation block remains untouched for trial scoring;
- the final all-data refit uses the median effective tree count learned across folds.
A trial is now eligible only when every temporal validation fold produces non-constant scores. After the final refit, the live snapshot also fails closed if every security receives the same score. This prevents tied scores from silently becoming an alphabetical ticker portfolio.
model_selection.csv records the requested estimator cap, best iteration, effective estimator
count, and stopping-window settings for each fold. The aggregate row records the median and range
of effective trees. Early stopping reduces unnecessary boosting rounds without using the live
decision date or its future return.
- A portfolio is evaluated only after its holdings have been frozen.
- Same-session partial prices cannot enter model fitting or stock selection.
- Training labels end early enough that their complete forward horizon is known at the decision time.
- Hyperparameters and feature counts are selected using temporal validation, not the live session.
- Sector observations are joined backward by effective date; no later inferred sector is copied into an earlier feature row.
- Split features use only completed split events on or before the feature date. Upcoming splits require a separate timestamped announcement source and are not inferred from future event dates.
- Ridge, XGBoost, and the shadow ensemble are displayed together; none is chosen retrospectively using realized live returns.
- The ensemble uses 25% Ridge / 75% XGBoost, selected by expanding historical OOS folds. Each fold's reported weight used only earlier completed folds; no live realized return selected the weight.
- The July 20 session is a setup/test observation. The official comparison baseline begins at the July 20 SPY close, and portfolio performance begins on July 21.
Yahoo's hourly timestamps identify the start of each bar. The dashboard displays the corresponding bar-end time, so a Yahoo 15:30 bar appears as 16:00. Daily compounded returns are anchored to the previous official close, not to the session open.
The date controls aggregate every completed observation in the selected range. Because the top-40 holdings can change each day, each session uses the list frozen before that session.
The Portfolio vs Benchmark chart uses the union of all evaluated dates. A model introduced later starts at its own first frozen observation and does not truncate the earlier history of established models. Click a model chip above the chart to hide or restore that series. Visibility choices and the selected date range persist across manual refreshes.
The stock-level view defaults to the newest decision date with a completed holding-level evaluation.
The newest frozen portfolios remain available in the selector with a Pending label until their next
session closes. Experiment history defaults to the newest evaluation date and can be filtered by
evaluation date or selected model, limited to a compact number of rows, and sorted by any column.
The dashboard does not automatically reload on a timer because public data changes only after a pipeline publication. Use Refresh data when a new deployment is expected. Date ranges, stock/model choices, history filters, row limits, and table sorting are stored in the browser and restored after a manual reload, so checking for new data does not reset the analyst's view.
All dashboard insights are deterministic calculations; no LLM API is required.
For two 40-stock sets, A and B:
- Overlap rate:
|A intersection B| / 40 - Jaccard similarity:
|A intersection B| / |A union B| - Model disagreement:
1 - overlap rate - Day-over-day replacement:
1 - |A_today intersection A_prior| / 40 - Rank stability: Spearman correlation of ranks among names retained on both dates
- Average rank movement: mean absolute rank change among retained names
The dashboard also reports top-10 retention, entrants, exits, current Ridge-XGBoost consensus, and persistent consensus across the available sessions. These describe model agreement and stability; they do not by themselves prove forecast quality.
Generated outputs remain under the ignored reports/live_forward/ directory:
- dated Ridge and XGBoost snapshot directories with manifests, tuning results, predictions, and selected holdings;
- dated
ridge_xgb_ensemble/shadow snapshots with normalized component ranks, selected holdings, and SHA-256 hashes of both source score files; - a model-aware
live_forward_results.csvrebuilt from the canonical per-snapshot summaries; - intraday and close-to-close evaluation files for each frozen portfolio;
- holding-level returns, benchmark-relative returns, and beat-SPY indicators for completed sessions;
selection_stability_YYYY-MM-DD.json;dashboard.html.
The daily command also refreshes the tracked site/ directory. It contains only the allowlisted
dashboard fields and split static assets required by GitHub Pages. See the
deployment guide for the privacy boundary, validation command, and
one-time repository setting.
Keep source code, configurations, tests, and documentation in Git. Do not commit cached prices, trained artifacts, generated reports, credentials, or temporary pytest directories.
The consolidated result key is (model, decision_date, evaluation date, horizon, benchmark).
Including model prevents one same-session XGBoost observation from overwriting Ridge. If the
table is missing or suspected to be stale, rebuild_live_forward_results() reconstructs it from
the intact canonical snapshot JSON files; archived experimental variants are not mixed into the
official live history.
The live sample is currently too short for a performance claim. Compare Ridge, XGBoost, the predeclared ensemble, and SPY over many independent sessions, then report active return, turnover, drawdown, and uncertainty. Rank IC remains a component-model diagnostic; it is not available for the ensemble until paired historical out-of-sample scores are persisted. Model overlap is diagnostically useful, but high consensus is neither necessary nor sufficient for outperformance.
The complete historical portfolio comparison is recorded in the Ridge-XGBoost ensemble experiment. It includes compounding, identical execution costs, SPY, risk metrics, moving-block confidence intervals, a shuffled-score placebo, and a final chronological-fold check.
run_live_rebalance_ablation() compares each model's actual daily frozen selections with two
controls beginning from the same decision date:
- Frozen names, buy and hold: buy the first top 40 once and allow weights to drift.
- Frozen names, equal weight: keep the first top 40 but apply the normal execution threshold around equal weights. This separates membership changes from weight drift.
- Daily selection: use the top 40 that was genuinely frozen after each completed close.
All arms use the same cached adjusted closes, starting capital, spread/slippage assumption, fixed trade fee, and completed close-to-close sessions. The report intentionally emphasizes cumulative return, cost drag, turnover, trades, and drawdown; annualized statistics are not meaningful for a handful of live observations.